Arquillian initially overpromises and frustrates, but delivers real benefits

Arquillian is changing fairly rapidly, and the Arquillian folks are paying a lot of attention to feedback. This post discusses Arquillian 1.0.0.Final, and more importantly the ShinkWrap Resolver and ShrinkWrap Dependency extensions. A lot has already improved since I wrote this, though most of it hasn't hit -Final versions yet.

---

After seeing a lot of talk, hype and excitement about Arquillian on twitter for several months, I finally got around to introducing it into a new project to give it a try. I'm told it'll make testing massively easier and save me tons of time, so using it is a no-brainer.

Tweet to @craigdevel After my recent experience I recommend that you start using it too - but you'll need to be prepared for some rough edges and the need for workarounds until a few point releases have gone by.

To jump straight to the summary of it all, click here, or read on for the whole experience.

Java EE 6 - Traps, pitfalls and warts list

Java EE 6 has the potential to be a great platform. It just needs plenty of cleanup, implementation bug fixing, and real-world use to get there.

One of the bigger issues is the significant array of spec inconsistencies and oversights that leave parts of the system working (or not working) in ways very different to one might expect. These issues can greatly increase debugging time and take important time away from productive development and into chasing issues in appservers, frameworks, and the specifications themselves.

I thought I'd make a list of some of the ones I've hit so far. I'm really hoping we'll see a Java EE 6.1 - a fixes-only revision that, unlike EE 7, focuses on polish and usability over adding new features. These are things I think are important to see covered by it.

The shortlist, explained in detail below:

You can't @Inject into a @FacesConverter. Use Seam 3 Faces to work around this. UPDATE: This should be fixed in JSF 2.1.

You can't @Inject into a JPA 2.0 EntityListener or an entity. Do a direct BeanManager lookup from JNDI or use Seam 3 Solder's BeanManagerAware to kind of hack around this; it's ugly. Lack of JPA 2.0 lifecycle hooks for EntityListener classes means Seam 3 Persistence can't fix this one. UPDATE Jan 2012: It appears that into EntityListener will be supported in JPA 2.1, but not injection into entities.

Completely avoid all use of javax.annotation.ManagedBean. It was obsolete before release, is only inconsistently recognised, duplicates JSF2 functionality that's also obsoleted by CDI's @Named, and should simply be removed.

Weld (CDI) before version 1.1.1 is extremely buggy. If using Glassfish 3.0, 3.0.1, or 3.1 you need to update weld-osgi-bundle to 1.1.1. See these instructions for Seam 3. UPDATE April 2012: These weld/glassfish issues are largely obsoleted by newer releases.

Mapping of application-local resource-ref resource names in web.xml to a global resource name defined in glasfish-web.xml or jboss-web.xml doesn't work for persistence.xml jta datasource references and isn't supposed to work. Ignore all the examples and official documentation that suggest it should, they work with direct JDBC and with Spring but not with JPA 2.0.

If you encounter mysterious silent failures or null pointer exceptions, you probably forgot to include beans.xml. Your container will not warn you about this at deployment or runtime even if CDI annotations are present on classes.

Many of these workarounds require the use of Seam 3 (ie: "let's finish the work EE 6 started") modules. Because of Glassfish bugs, some additional work is required to use Seam 3 on Glassfish versions prior to the yet-to-be-released 3.2. See this previous post for some information.

In addition to the genuine issues with EE 6, there's also the learning curve to contend with. A while ago I wrote a broad conceptual overview of EE 6 intended for those coming to it cold, rather than migrating from EE 5 or from Seam. I felt it was necessary because of the struggle I had getting started with EE 6 and the bugs that bit me and confused me during the learning process. Perhaps it'll benefit others here or even provide insight into areas of the documentation and tutorial that would benefit from improvement.

Now, for the details:

Someone else in Java EE 6 pain

I'm not alone!

Harald Wellmann writes about his experience moving a Tomcat-based app to Glassfish, JBoss AS 6, and Resin. Warts and all.

I think Java EE 6 really needs more real-world attitude and less hype. More bug fixing and more spec tidying, less focus on the next whizz-bang feature. It's nice to see somebody else who's had similarly painful experiences with it. Perhaps I'm not stupid after all, I simply make the foolish mistake of expecting released software like Glassfish 3.0.1 to have most features fairly solid.

Harald also notes that CDI was a huge problem with Java EE 6 until quite recently, largely because Glassfish 3.0 was half-baked and under-tested; JPA 2.0 is prone to exciting implementation differences and bugs, and more.

My favourite quote:

Legacy

Backward compatibility is a good thing for veteran users of a framework. Users don't want to change all of their application code just for upgrading to a new framework version.

On the other hand, backward compatibility can be extremely confusing to new users: There's two or three solutions for the same kind of problem, the legacy one is not marked as legacy or deprecated clearly enough, tutorials and example code still use the old style, and you can only resort to your favourite search engine or to trial and error to make things work consistently.

(Emphasis mine)

That's a perfect description of some of the problems I encountered when trying to get a handle on the released standard Java EE 6 and Glassfish 3.0, which I foolishly assumed was a stable platform to build my app on as I moved into the Java EE world

Java EE 7 really needs to modularize the backward compatibility crap into separate archives so that apps can depend on either javaee-7 or javaee-7-legacy as appropriate. That way we'll finally be free of the two copies of the obsolete-before-release @ManagedBean annotation and all the other horrible legacy crap.

Java EE application servers - learning from the past

(Edit 15 July 2011: JBoss AS 7 is here, and brings a huge improvement to class loading and memory management. It's not full isolation, but it limits the exposed contact surface between app server and app greatly, and massively improves class loading. Brilliant!)

If you've used a Java EE application server like Glassfish or JBoss AS for long, you will have experienced classloader leaks, though you may not have realized it.

If you've ever seen the error java.lang.OutOfMemoryError: PermGen space at deploy-time, read the linked article above. If you have ever worked on the JVM, on app servers, or on EE applications, please read on.

Even if you haven't hit classloader leaks, you should be aware of the various ways Java EE applications can cause memory leaks in the server and what to do about them.

For those coming here for help fixing an immediate issue with their app: Read the links above, and this article on using jhat's JavaScript interface to find likely leaks. More fancy JavaScript OQL tricks are here.

Useful things I've discovered about Glassfish (especially embedded glassfish) lately

I've been working with / struggling with Glassfish a lot lately. The documentation leaves much to be desired, so there are a few bits and pieces you might not know that might come in handy.

These notes refer to Glassfish 3.1.

File upload webapp

I recently became frustrated with the lack of easily self-hosted file upload app options. I needed to replace my employer's reliance on giant email attachments with something that wouldn't melt our mail server - and our clients could still understand how to use. This unfortunately ruled out the classic options like anonymous FTP.

In the end I wrote a new web application to accept files from clients over http and save them to a shared folder on the internal network. Many of the building blocks like uploadify were available, but there wasn't much around in terms of complete applications to do the job. Instead, people seemed to be using services like YouSendIt, Box.net, DropSend, etc ... which have downsides including:

• Inability to provide a canned recipient list, or high per-user per-month fees to support one
• Inability to customise the site's appearance and integrate it into your domain - without paying high fees for an "enterprise" version
• The need to download files once the client has sent them to the provider, often one-by-one via a clumsy web interface
• Windows-only client applications required to do batch downloads of files. Some services have Mac versions, but only one had a Linux client suitable for our advertising thin clients.
• Lack of a web services API - unless you pay high fees for the "enterprise" version.
• Lack of no-flash fallback upload methods
• Sometimes iffy browser compatibility

I wanted something really simple for clients - and staff - to use, that simply dropped files into a shared folder as they were uploaded and sent a notification email to staff when the upload was finished. Now that I've finished it, I'm releasing the result as open source software for the use of anybody who wants it. You can get it here:

https://github.com/ringerc/postupload

It's a Java EE 6 project that runs on Glassfish. Don't stress if you've never used any of that before, if you only know PHP, etc. The client-side is all familiar HTML and JavaScript, and the server side isn't especially complicated code. Detailed instructions for installing and running Glassfish are provided.

Handling file uploads with Java EE 6 / JAX-RS / Glassfish / Uploadify

UPDATE: This article is very obsolete. RichFaces 4, among other framworks, now handles multi-file upload out of the box. Use that instead.

Joeri Sykora wrote about using Jersey extensions to JAX-RS to handle file uploads in Java EE 6. His example is extremely handy, but needs some updates to handle Jersey 1.5.

Update: I've now put together a complete and self contained Java EE upload application using Uploadify, which you can grab from github.com/ringerc/postupload. See http://blog.ringerc.id.au/2011/03/file-upload-webapp.html

Untangling Java EE 6 - a broad conceptual overview

WARNING

This document was written by someone who's only learning Java EE 6. On one hand, I remember the frustrations clearly and still encounter issues of understanding (and bugs) regularly, so I'm in a good position to think about how a newbie needs to have things explained. On the other hand, a bunch of this is probably wrong because my own understanding is still developing. Beware.

This is also a work in progress. It's been published for comment and review, and isn't a fully edited and checked final work.

If you're coming from Spring, rather than starting with Java EE cold, you should start here.

Java EE 6

Java EE 6 is not a product you can download and install. It is a specification that describes how implementing software should behave. It refers to many sub-specifications for particular features, each of which has one or more implementations in software. Some of these implementations can be used outside a full Java EE 6 environment as standalone products.

Simple Java EE 6 project fails in different ways on Glassfish and JBoss AS 6

(Update: this issue was caused by a Glassfish bug. See Bug 13040 in the Glassfish tracker.)

I'm coming to really like Java EE 6 ... or at least, I would if anything worked properly. There seem to be an awful lot of quirks and bugs in even fairly basic functionality, though.

Take this simple project, where a @Stateless EJB that inherits from an abstract base is injected into a regular JSF2 backing bean. It's done two different ways - once using the new CDI/Weld injection facilities, and once with old-style JSF2 injection and EJB container management.

One works on glassfish, the other on JBoss. Neither works on both.

The glassfish failure is caused by a problem resolving methods inherited from superclasses in the Weld-generated local no-interface EJB proxies. The JBoss issue appears to be caused by issues with its local no-interface EJB support, too, though I'm not sure exactly what yet.

The point: Local no-interface EJBs were supposed to save time and pain for developers, yet all I'm finding so far is bugs, or at least quirks and inconsistent behaviour.

Sources to the sample can be found here and a deployable war is here.

Posted on StackOverflow and The Glassfish forums

.

I'm not smart enough to use Java EE

I don't seem to be smart enough to use Java EE.

Reading the Java EE tutorial certainly helps. It's very good at explaining a lot of the how, though not as much of the why; there isn't a strong overview of what EE is and how it fits together, but there's plenty on how the parts work and how to use them. Before sitting down for some quality reading time, I was totally lost. This is not something you can dive head-first into. After reading the tutorial I'm only very confused. [Edit 2012: The tutorial seems to have more overview info now, or I understood it better this time around. It's still hard to see what uses what, builds on what, and relates to what from the tutorial though.]

UPDATE 2011: I seem to be managing despite the learning curve and the quality challenges, and am trying to help others avoid the same suffering I went through by providing some overview level documentation on how it all fits together and what all the damn acronyms mean.

UPDATE mid-2012: It's two years since I wrote this, and sometimes I still struggle with the complexity of the Java EE stack. JSF2 in particular feels significantly over-engineered and doesn't have documentation to match its complexity; it feels as complicated as the rest of Java EE put together without close to as much useful documentation (and lots more bugs).

---

The remainder was written in August 2010, not long after I moved from Swing development over to Java EE 6 - unbeknownst to me, just as Java EE 6 implementations were in their infancy and not in any sense stable or production ready.

---

Problem 1: Acronym soup

Seam. Struts. Tapestry. Weave. Weld. JSF. JSF2. JPA. JPA2. EJB2. EJB3. CDI. Hibernate, EclipseLink, etc. J2EE. JARs in OSGi bundles in WARs in EARs. Glassfish. JBoss. Tomcat. Jetty. @Inject. @EJB. @Resource. @PersistenceContext. @javax.annotation.ManagedBean. @javax.faces.bean.ManagedBean. @Named. @Stateless. @Stateful. Spring. Servlet filters. @javax.faces.bean.RequestScoped, javax.enterprise.context.RequestScoped, @javax.faces.bean.SessionScoped, javax.enterprise.context.SessionsCoped, @javax.faces.bean.ApplicationScoped, javax.enterprise.context.ApplicationScoped, javax.enterprise.context.ConversationScoped, @javax.faces.bean.ViewScoped. @Singleton. DAOs and data access layers. JTA. JNDI. JMS. RMI. Deployment descriptors (web.xml, etc). Vendor deployment descriptors (sun-web.xml, sun-resources.xml, etc). Various config files, some of which have an effect even when empty (faces-config.xml, beans.xml). Maven plugins, oh god the maven plugins. RichFaces, MyFaces, IceFaces, PrimeFaces, SeamFaces.

Java EE makes me understand how "non computer people" feel when listening to programmers talk.

Problem 2: Bad or incomplete documentation, narrow-view docs without an overview

For a newbie, it's extremely hard to understand the layering, and made even more confusing by the mixing of specs (eg JAX-RS, JSF2) with implementations (eg Jackson or RestEASY, Mojarra or MyFaces). Each project's documentation talks largely about its self in isolation.

Most of the documentation, even the Java EE 6 tutorial, focuses on narrow parts of the spec without ever trying to help you understand how it all fits together. Some overview documentation that focuses on the basic concepts is sorely needed.

It doesn't help that the JSF2 spec is [wording edited] a very dense document that specifies but does not document or explain JSF2. The assumption seems to be that you'll get a book for JSF2[/end edit]. [Added 2012:] Many recommend Core JavaServerFaces over JavaServer Faces 2.0, The Complete Reference; I certainly found the former helpful.[/end added]

Problem 3: Legacy, duplication, and specs created in isolation

Part of the problem is that there are several generation of "technology" that are mashed together in an overlapping mess. For example, J2EE6 uses CDI with Weld, but everyone already does this with Spring, which also does all sorts of other things so you might want to still use it even though you're doing DI with Weld in a J2EE container. Your objects might be being managed by some combination of JSF2 @ManagedBean lifecycle management, @EJB lifecycle management, Weld, or (if you're using it) Spring.

Or does JSF2 actually use CDI to manage its beans via Weld? Who knows! (Update: the answer is "Maybe", btw. If you're using CDI, JSF2 beans are managed with CDI as implemented by Weld. Otherwise they're managed by JSF2 its self.) Every framework uses every other framework or has semi-transparent integration hooks for them - hooks that mostly work, but with some subtle bugs or limitations. Every framework claims to make everything simple, while adding yet another layer of nightmarish complexity. Specifications inevitably have a reference implementation from Sun/Oracle/JBoss, and an Apache implementation, so you have to figure out what the relationship between the implementation names and the spec name is for each, what they actually do, which implementation you might want to choose and why, etc.

When trying to learn about this stuff, it's hard to find materials that don't already assume you know the older technologies and just want to update to the newer ones. Hell, it's hard to figure out what the older ones and newer ones even are, or why you should want to use one over the other.

The J2EE platform releases are supposed to help with that, but seem to add complexity more than remove it, due to the need for backward compat, support for older frameworks and code that uses them, the messy overlap between different specs, the way different specs each added feature like injection in different and incompatible ways, etc.

Problem 4: Too much magic, not enough visibility when (not if) the magic breaks

There's so much transparent management of application state, object lifetimes, request handling, etc that it becomes quite hard to figure out what your app is actually doing, how and when to do something, or how to alter some part of the automatic behaviour when you need to. As for debugging... good luck following your app though a JSF2 request lifecycle.

Heaven save you if you forget to put an empty beans.xml magic marker file in a project. It'll be NullPointerException soup with absolutely NO indication that CDI is turned off. Obvious and simple with experience, hell when I first got started.

Struggling? You're not alone

On the other hand, it makes me feel better when really, really smart people point out that all this architecture makes it hard to see what you're actually trying to do.

... and all this means that paradoxically, even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.

.... and really, reading the manual makes it a little more approachable. Entirely avoiding JSF2 in favour of something sane like Vaadin or even Play if you're so inclined will help even more.