As a little background to this post, during the course of a recent client project involving image manipulation, I wrote some code to handle images in the tiff format. I used the Java Advanced Imaging (JAI) library's ImageIO class, which made reading from the image file super easy through the use of ImageIO.read(File) which automatically determines the file format based on the extension and performs the file read internally, returning a fully usable BufferedImage object. It really makes reading data from an image super easy, and I highly recommend using it.

While working on the project I needed to write a simple utility to count the number of unique colors in an image and print out how many pixels in the image were that color (useful for debugging the main application). Its a really simple Java program that loops through the image, incrementing the count in a hash map of color values to pixel counts. I copy/pasted the image reading code from the main program, and was surprised to see the test program start generating errors stating that the tiff file could not be read as there was no image reader associated with the tiff file format. Lacking a tiff reader is really surprising because this test utility is just another class file within the same eclipse project as the main application, and therefore has the same class path. Both the jai_core.jar and the jai_codec.jar (the two jars that make up the JAI library) are on the project's class path, so there should be no reason that one java file would have access to them, but another java file does not.

It turns out this was also a problem for gif and jpeg images in versions past, as evidenced by this FAQ question on the JAI home page:

On Solaris, Java Advanced Imaging complains about lack of access to an X server.
Java Advanced Imaging versions previous to JAI 1.1.1 used the AWT toolkit to load GIF and JPEG files. This problem is a manifestation of a JDK bug in which creation of the AWT Toolkit class results in an attempt to open the X display. To work around this problem in Java Advanced Imaging versions prior to 1.1.1, either make an X display available to the Java runtime using the DISPLAY environment variable (no windows will appear on the display), or consider running a dummy X server that will satisfy the AWT, such as the Xvfb utility included with the X11R6.4 distribution.

In the JAI 1.1.1 version, the GIF and JPEG decoders were improved to no longer have a dependency on the X server.

The answer it turns out is that my simple utility does not set up the AWT windowing system (since I wrote it as a CLI utilizing System.in) and therefore does not end up loading the tiff image reader plugin because of this fact. It turns out that in order to utilize the tiff reader plugin from the JAI library your code must perform at least one of the following calls:

1. Instantiate a JFrame
2. Call Toolkit.getDefaultToolkit()
3. Call Application.getApplication() (Mac OS X java extension)

While I still don't know the exact nature of the behavior, all evidence points to the fact that in a java program that needs to utilize the tiff image reader, you must set up the AWT windowing system in some manner. Even if your program (like my test utility) doesn't need to create a window or deal with the windowing system in any fashion, you must do one of the above methods in order to correctly register the tiff reader.

Over the last couple of days of development on a Java web application using the Wicket framework, I noticed a peculiar behavior on the server while in debug mode. After several redeployments of the application war file to the Tomcat server (without restarting the server), eventually a page reload would hang and the server would start spewing java.lang.OutOfMemoryError: PermGen space stack traces to the console. Clearly this is indicating a memory leak somewhere in my code or in the libraries my code relies on and on top of that, this isn't the sort of memory leak usually associated with poor memory management, since this is the PermGen space not the Heap space which is running out. Since PermGen is a special place in memory for ClassLoader objects to reside, its not something that most memory profilers will pick up since it shouldn't ever be a problem for normal code.

At this point, after searching for information on what would cause PermGen space to run out, and reading a bunch of mixed responses (most programmers that have blogged about this before me seem to take the head in the sand approach and just increase the PermGen size so that won't run out as fast), I found these two wonderful blog posts by Frank Kieviet. He's analysis of the situation is very enlightened and helped me understand what was going on behind the scenes.

So, using Sun's VisualVM tool, I started profiling the PermGen space of my local Tomcat install while continually deploying and redeploying the code base. Sure enough, every time I changed the code and eclipse automatically redeployed the code to the Tomcat instance, the PermGen size would increase the next time I requested a page and Tomcat started allocating objects. No matter how long I waited or clicked the garbage collect button, the PermGen space was never reclaimed and eventually I'd hit the memory ceiling and the server would start spewing those familiar OutOfMemoryError messages again.

Having read Frank's excellent analysis of static references, I took a trip through the code base in order to find offending static object references that might be causing the leak. Unfortunately, no where in the code was I creating static references that looked like they would have resulted in a ClassLoader leak (not to say that there aren't, as was pointed out in Frank's article, this is a tricky problem to track down and none of my references looked suspicious). Well, if its not in my code, where is it?

Since I'm using Wicket, I created a skeletal test project that creates a single webpage and dynamically sets a span tag on the page to a hard coded String object. I fired up VisualVM, and ran my same deploy and redeploy test and sure enough the PermGen space starts getting eaten up like clockwork. To be fair, the PermGen space didn't fill up nearly as fast since my test project is only allocating a few classes and they're associated ClassLoader objects every redeploy, but given enough time it did eventually result in the same OutOfMemoryErrors in the console.

"Aha!", I thought, "Wicket's developers have made a grievous mistake, and aren't watching their code for potential ClassLoader leaks". I figured that since I'm using Wicket, I'll just have to live with the knowledge that eventually I'll run out of PermGen space and have to restart my server. Certainly annoying to me as a developer, but its definitely a viable workaround. The only problem will be when this code hits production, and I'll have to be careful to restart the server after each code deployment or run the risk of having the server be brought to its knees from memory problems.

I can't remember what particular Google search netted me this link, but I did eventually find this gem of information:

The JDK's permanent memory behaves differently depending on whether a debugging is enable, i.e. if there is an active agent.

If there is an active agent, the JDK can fail to collect permanent memory in some cases. (Specifically, when some code introspects a class with a primitive array like, byte[] or char[].) This can cause the permanent space to run out, for example, when redeploying .war files.

I had no idea that the JVM's garbage collector behaved differently depending on whether the application is running in debug mode or not. Since I'm constantly running my code in debug mode and using Eclipse's hot fix code replace feature, and every code change redeploys the entire .war archive to the server, resulting in more class loaders on the class path, and another chunk of memory that will never be reclaimed by the server.

Therefore, the moral of the story is while trying to track down memory leaks, do not run your application (especially a web application) in debug mode. It is still true that you can leak PermGen space without your application being run in debug mode, so this isn't an excuse for poor coding practices. Rather, its a cautionary tale of my part of how even a simple thing like running your application in debug mode can mean rather drastically different program performance even outside of the debug instruction overhead.