Java EE apps as first-class citizens

Juliano Viana writes that he hopes to see Java EE applications promoted to first-class citizens in Java EE 8.

His comments are focused on protecting apps running in a container from each other, and the container from the apps. The primary focus of the post is on resource leaks. He's interested in making it possible to cleanly and totally unload an app, much as you can kill a process on a regular OS to completely clean it up.

This lead me to an interesting realization, though one I'm sure others have had before:

Java EE containers are in some ways a lot like MS-DOS, Mac OS 6/7/8/9, Symbian, etc ... applications share the same memory space and have access to each other's innards. There's polite agreement not to poke where they don't belong, but no strong enforcement of that. Without pointers and the resulting fun with corrupted stacks and dangling pointers leading apps to trample all over each other's memory the JVM and Java EE don't have it quite so bad - but there are still real issues with access control, resource ownership, etc.

Currently a Java EE app can delve into the innards of the container if it wants. It's not *meant* to, but it's not easily prevented and it makes the container's integrity less trustworthy. An app can break the container or cause it to misbehave. Expecting apps to be well-written is, as demonstrated by the OSes listed above, a great way to get a reputation for being unstable and generally crap. More importantly, as the referenced article notes, Java EE apps can cause the container to leak resources, or can disrupt resources the apps shares with other apps via the container, like pooled connections. That makes it hard to share one container among many different apps, let alone different users/customers.

This (and garbage collection + resource use issues with big JVMs) makes admins reluctant to have many apps hosted in a single container, so you land up doing all sorts of icky inter-JVM work to make your environment managable. Shared-hosting services with Java are expensive and uncommon due to related issues.

I certainly think that much stronger protection is needed to isolate the container's private core from apps, and apps from each other. I'm not at all convinced that trying to build that isolation entirely in software in the JVM is best, though. Modern successful OSes isolate apps using *hardware* features to do most of the grunt-work and protect them from implementation bugs; apps and the kernel are isolated from each other by default and can only interact by configured exception. Protected mode, separate logical address spaces, and the security ring system in x86 provide rather strong OS/app isolation. Trying to implement similarly capable isolation purely in software in the JVM would be bug-prone and hard to get right, kind of like the half-baked and buggy app isolation in Windows 3.x or even 9x.

Perhaps a more viable future approach would be to let multiple JVMs running on a host integrate more tightly via shared memory, high performance IPC, or whatever, so the container could run in its own JVM and each app could have a private JVM, with only those resources *explicitly* shared between them accessible across JVMs. That way, apps cleanup would be as simple as killing the app's jvm and letting the OS+hardware clean up the mess. The exposed surface for problems would be restricted to that part of the container that's explicitly accessible via the shared resources.

Or ... maybe the JVM will need more OS co-operation than that, so the JVM its self can use hardware features to isolate apps running within the JVM. I can't imagine OS designers wanting to let the JVM get its hooks into the kernel's memory mapping and process management, but with the advent of VMs and VM-friendly CPU extensions like VT-X and VT-IO I wonder if the JVM could use those instructions, like the kernel of a guest OS does in a VM, to isolate apps running within the JVM.

Much as I'd love the JVM to be able to isolate apps properly, in my admittedly pretty ignorant opinion I don't see it happening without some kind of major re-design. I imagine a half-assed solution that works about as well as Windows 9x did is possible with the current JVM, but giving the impression apps can be isolated and cleaned up without being able to really do a comprehensive job of it is IMO in many ways worse than the current situation. Without some really powerful, low level isolation features I don't see that happening. Look at how well "shared" OSes have typically achieved app isolation for examples of just how hard the problem is.

I find the idea of using hardware VT-X support to help the JVM isolate apps in an EE container quite intriguing. I wonder if it's something anyone's done any investigation of.

Three hours in

I'm three hours or so into my real work, after being side-tracked isolating and reporting Glassfish and Mojarra bugs. I just found another one in Mojarra, albeit a trivial one.

(Edit) I'd written a little rant about the problem I was trying to solve, in the assumption that I'd hit another limitation/bug in JSF2. Well, this time I was just plain wrong; I'd used an incorrect test to draw an invalid conclusion and from there ran down the path - from my previous experience, admittedly - of assuming that this, too, was a tools issue.

The correct answer is here: To permit null/empty/none selection in a h:selectOneMenu driven by a f:selectItems , just add a f:selectItem with no value before the f:selectItems.

Converting a NetBeans-generated JSF2 project to use CDI

I started playing with Netbeans 6.9's JSF2 interface generation for database apps, and quickly noticed that (a) it uses JSF2 injection, not CDI and (b) it generates some verbose, ugly converters.

I decided to switch it over, as a bit of a learning experience. Here's what landed up being necessary.

Untangling Java EE 6 - a broad conceptual overview

WARNING

This document was written by someone who's only learning Java EE 6. On one hand, I remember the frustrations clearly and still encounter issues of understanding (and bugs) regularly, so I'm in a good position to think about how a newbie needs to have things explained. On the other hand, a bunch of this is probably wrong because my own understanding is still developing. Beware.

This is also a work in progress. It's been published for comment and review, and isn't a fully edited and checked final work.

If you're coming from Spring, rather than starting with Java EE cold, you should start here.

Java EE 6

Java EE 6 is not a product you can download and install. It is a specification that describes how implementing software should behave. It refers to many sub-specifications for particular features, each of which has one or more implementations in software. Some of these implementations can be used outside a full Java EE 6 environment as standalone products.

Java EE 6 doesn't "just work" in even trivial real-world uses

Lincoln Baxter III writes that with Java EE 6, It all “Just works”.

Personally, I have not found that to be the case in even simple, toy applications.

I suspect it'll be true in a few years, probably after a much needed "Java EE 6.2" and some major bug-fixing. Right now it all "just works" ... except when it doesn't, which is frequently, due to spec oversights, bugs, and a general lack of real-world use and testing.

Simple Java EE 6 project fails in different ways on Glassfish and JBoss AS 6

(Update: this issue was caused by a Glassfish bug. See Bug 13040 in the Glassfish tracker.)

I'm coming to really like Java EE 6 ... or at least, I would if anything worked properly. There seem to be an awful lot of quirks and bugs in even fairly basic functionality, though.

Take this simple project, where a @Stateless EJB that inherits from an abstract base is injected into a regular JSF2 backing bean. It's done two different ways - once using the new CDI/Weld injection facilities, and once with old-style JSF2 injection and EJB container management.

One works on glassfish, the other on JBoss. Neither works on both.

The glassfish failure is caused by a problem resolving methods inherited from superclasses in the Weld-generated local no-interface EJB proxies. The JBoss issue appears to be caused by issues with its local no-interface EJB support, too, though I'm not sure exactly what yet.

The point: Local no-interface EJBs were supposed to save time and pain for developers, yet all I'm finding so far is bugs, or at least quirks and inconsistent behaviour.

Sources to the sample can be found here and a deployable war is here.

Posted on StackOverflow and The Glassfish forums

.

Linux (ubuntu 10.04) ppp maxfail 0 and persist not working? (WORKAROUND)

I've been wondering for a while why, after an upgrade to Ubuntu 10.04, the ppp daemon on my router/server had stopped automatically reconnecting after it lost ADSL line sync or otherwise had its connection dropped.

The culprit turns out to be a helper script (/etc/init/network-interface.conf) in upstart that tries to handle hotplugging of network interfaces, automatically bringing them up when plugged in and down when unplugged.

Unfortunately, it views ppp interfaces as hot-plugged, and helpfully calls ifdown on them when they dissappear after a connection is lost. This kills pppd, preventing it from creating a new ppp interface and trying to connect again.

Unlike most interfaces, ppp interfaces are created and destroyed as a consequence of ifup and ifdown calls. Well, really pppd invocation, but that's usually done via ifup/ifdown these days. Calling ifdown when a ppp interface vanishes isn't always wrong, but it's wrong if the pppd is set to "maxfail 0 persist".

As the script isn't smart enough to know that, and I don't hotplug interfaces on my router anyway, I've opted to simply disable the script in question by renaming it to /etc/init/network-interface.conf.disabled. That'll be broken whenever the upstart package is updated, though, so a better solution is required.

Unconditionally ignoring all ppp interfaces in the script isn't necessarily right, as it would be nice to ifdown them cleanly when the pppd has exited and is thus no long retrying. It's hard to do that reliably as the interface is destroyed before pppd actually terminates, and it's hard to query pppd to find out if it plans to quit or retry. Arguably the retry logic should be moved out of pppd and into upstart or network-manager, but with the state of those tools at this point that's a recipe for pain and suffering.

In the mean time, you can work around it by disabling the problematic init script.

simple script to extract Stanza epub ebooks from an iTunes backup

iTunes and the iPhone don't like to let go of data they have their claws into. Sometimes, however, you might need that data outside the Apple Garden, in which case you're going to have to get your hands dirty diving in iTunes backups, as they're the easiest way to regain control of your files.

(Tip: if you need to fight your software to access your own files, your platform is hostile. I don't use the iPhone myself and loathe the way Apple does things, but sometimes I have to work with it for other people. Hence this post.)

I needed to recover some Stanza ebooks from an iPhone. It's hard enough to get them *on* to the phone, and getting them off is nigh on impossible, as Apple continuously changes things to make it hard to access the phone via anything but iTunes. In this case, though, the latest change (to the backup format) made it easier to work with, not harder, so the extraction wasn't too hard.

I'm not smart enough to use Java EE

I don't seem to be smart enough to use Java EE.

Reading the Java EE tutorial certainly helps. It's very good at explaining a lot of the how, though not as much of the why; there isn't a strong overview of what EE is and how it fits together, but there's plenty on how the parts work and how to use them. Before sitting down for some quality reading time, I was totally lost. This is not something you can dive head-first into. After reading the tutorial I'm only very confused. [Edit 2012: The tutorial seems to have more overview info now, or I understood it better this time around. It's still hard to see what uses what, builds on what, and relates to what from the tutorial though.]

UPDATE 2011: I seem to be managing despite the learning curve and the quality challenges, and am trying to help others avoid the same suffering I went through by providing some overview level documentation on how it all fits together and what all the damn acronyms mean.

UPDATE mid-2012: It's two years since I wrote this, and sometimes I still struggle with the complexity of the Java EE stack. JSF2 in particular feels significantly over-engineered and doesn't have documentation to match its complexity; it feels as complicated as the rest of Java EE put together without close to as much useful documentation (and lots more bugs).

---

The remainder was written in August 2010, not long after I moved from Swing development over to Java EE 6 - unbeknownst to me, just as Java EE 6 implementations were in their infancy and not in any sense stable or production ready.

---

Problem 1: Acronym soup

Seam. Struts. Tapestry. Weave. Weld. JSF. JSF2. JPA. JPA2. EJB2. EJB3. CDI. Hibernate, EclipseLink, etc. J2EE. JARs in OSGi bundles in WARs in EARs. Glassfish. JBoss. Tomcat. Jetty. @Inject. @EJB. @Resource. @PersistenceContext. @javax.annotation.ManagedBean. @javax.faces.bean.ManagedBean. @Named. @Stateless. @Stateful. Spring. Servlet filters. @javax.faces.bean.RequestScoped, javax.enterprise.context.RequestScoped, @javax.faces.bean.SessionScoped, javax.enterprise.context.SessionsCoped, @javax.faces.bean.ApplicationScoped, javax.enterprise.context.ApplicationScoped, javax.enterprise.context.ConversationScoped, @javax.faces.bean.ViewScoped. @Singleton. DAOs and data access layers. JTA. JNDI. JMS. RMI. Deployment descriptors (web.xml, etc). Vendor deployment descriptors (sun-web.xml, sun-resources.xml, etc). Various config files, some of which have an effect even when empty (faces-config.xml, beans.xml). Maven plugins, oh god the maven plugins. RichFaces, MyFaces, IceFaces, PrimeFaces, SeamFaces.

Java EE makes me understand how "non computer people" feel when listening to programmers talk.

Problem 2: Bad or incomplete documentation, narrow-view docs without an overview

For a newbie, it's extremely hard to understand the layering, and made even more confusing by the mixing of specs (eg JAX-RS, JSF2) with implementations (eg Jackson or RestEASY, Mojarra or MyFaces). Each project's documentation talks largely about its self in isolation.

Most of the documentation, even the Java EE 6 tutorial, focuses on narrow parts of the spec without ever trying to help you understand how it all fits together. Some overview documentation that focuses on the basic concepts is sorely needed.

It doesn't help that the JSF2 spec is [wording edited] a very dense document that specifies but does not document or explain JSF2. The assumption seems to be that you'll get a book for JSF2[/end edit]. [Added 2012:] Many recommend Core JavaServerFaces over JavaServer Faces 2.0, The Complete Reference; I certainly found the former helpful.[/end added]

Problem 3: Legacy, duplication, and specs created in isolation

Part of the problem is that there are several generation of "technology" that are mashed together in an overlapping mess. For example, J2EE6 uses CDI with Weld, but everyone already does this with Spring, which also does all sorts of other things so you might want to still use it even though you're doing DI with Weld in a J2EE container. Your objects might be being managed by some combination of JSF2 @ManagedBean lifecycle management, @EJB lifecycle management, Weld, or (if you're using it) Spring.

Or does JSF2 actually use CDI to manage its beans via Weld? Who knows! (Update: the answer is "Maybe", btw. If you're using CDI, JSF2 beans are managed with CDI as implemented by Weld. Otherwise they're managed by JSF2 its self.) Every framework uses every other framework or has semi-transparent integration hooks for them - hooks that mostly work, but with some subtle bugs or limitations. Every framework claims to make everything simple, while adding yet another layer of nightmarish complexity. Specifications inevitably have a reference implementation from Sun/Oracle/JBoss, and an Apache implementation, so you have to figure out what the relationship between the implementation names and the spec name is for each, what they actually do, which implementation you might want to choose and why, etc.

When trying to learn about this stuff, it's hard to find materials that don't already assume you know the older technologies and just want to update to the newer ones. Hell, it's hard to figure out what the older ones and newer ones even are, or why you should want to use one over the other.

The J2EE platform releases are supposed to help with that, but seem to add complexity more than remove it, due to the need for backward compat, support for older frameworks and code that uses them, the messy overlap between different specs, the way different specs each added feature like injection in different and incompatible ways, etc.

Problem 4: Too much magic, not enough visibility when (not if) the magic breaks

There's so much transparent management of application state, object lifetimes, request handling, etc that it becomes quite hard to figure out what your app is actually doing, how and when to do something, or how to alter some part of the automatic behaviour when you need to. As for debugging... good luck following your app though a JSF2 request lifecycle.

Heaven save you if you forget to put an empty beans.xml magic marker file in a project. It'll be NullPointerException soup with absolutely NO indication that CDI is turned off. Obvious and simple with experience, hell when I first got started.

Struggling? You're not alone

On the other hand, it makes me feel better when really, really smart people point out that all this architecture makes it hard to see what you're actually trying to do.

... and all this means that paradoxically, even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.

.... and really, reading the manual makes it a little more approachable. Entirely avoiding JSF2 in favour of something sane like Vaadin or even Play if you're so inclined will help even more.

Great service providers I use

A few of the service providers I use deserve some credit for great work. They are:

• Linode.com, a great VPS host with excellent prices, good service, and really, really good tools for VPS management and control. You'll never need to call support because you broke SSH or installed a busted kernel ever again, since you have a web-based virtual serial console and full bootloader control over the web interface. Check them out, they're excellent.
• SimpleCDN.com, whose mirroring CDN service "Mirror Buckets" is truly incredible. I put about fifty bucks into it six months ago, and still haven't run out. It's about two thousand times cheaper than my old host's traffic charges - though admittedly that was an Australian VPS host with typically Australian bandwidth-prices-of-doom.
• Internode, who are the only Australian ISP at time of writing who're offering native IPv6 to regular ADSL customers. They're otherwise pretty decent within the limits of a cut-price cut-throat industry. Imperfect, but they're in telecommunications, and compared to the telco/ISP average they don't even rank on the suck chart.

All hail GNU Parted

A user bought in a dead laptop today. It was booting into Dell MediaDirect (an XP Embedded OS stored on the same disk as the main Windows XP install) instead of Windows XP, then bluescreening early in MediaDirect boot.

The user had accidentally hit the hardware button that boots the machine into MediaDirect instead of XP.

The Java needs language-level property support (and a reset)

I find myself increasingly bewildered by some of the design choices in the Java world. The core language's use of accessor methods is a notable example.

Not that properties are the only bewildering Java design choice.