Monday, July 12. 2010
The case of the missing tiff plugin
As a little background to this post, during the course of a recent client project involving image manipulation, I wrote some code to handle images in the tiff format. I used the Java Advanced Imaging (JAI) library's ImageIO class, which made reading from the image file super easy through the use of ImageIO.read(File) which automatically determines the file format based on the extension and performs the file read internally, returning a fully usable BufferedImage object. It really makes reading data from an image super easy, and I highly recommend using it.
While working on the project I needed to write a simple utility to count the number of unique colors in an image and print out how many pixels in the image were that color (useful for debugging the main application). Its a really simple Java program that loops through the image, incrementing the count in a hash map of color values to pixel counts. I copy/pasted the image reading code from the main program, and was surprised to see the test program start generating errors stating that the tiff file could not be read as there was no image reader associated with the tiff file format. Lacking a tiff reader is really surprising because this test utility is just another class file within the same eclipse project as the main application, and therefore has the same class path. Both the jai_core.jar and the jai_codec.jar (the two jars that make up the JAI library) are on the project's class path, so there should be no reason that one java file would have access to them, but another java file does not.
It turns out this was also a problem for gif and jpeg images in versions past, as evidenced by this FAQ question on the JAI home page:
On Solaris, Java Advanced Imaging complains about lack of access to an X server.
Java Advanced Imaging versions previous to JAI 1.1.1 used the AWT toolkit to load GIF and JPEG files. This problem is a manifestation of a JDK bug in which creation of the AWT Toolkit class results in an attempt to open the X display. To work around this problem in Java Advanced Imaging versions prior to 1.1.1, either make an X display available to the Java runtime using the DISPLAY environment variable (no windows will appear on the display), or consider running a dummy X server that will satisfy the AWT, such as the Xvfb utility included with the X11R6.4 distribution.In the JAI 1.1.1 version, the GIF and JPEG decoders were improved to no longer have a dependency on the X server.
The answer it turns out is that my simple utility does not set up the AWT windowing system (since I wrote it as a CLI utilizing System.in) and therefore does not end up loading the tiff image reader plugin because of this fact. It turns out that in order to utilize the tiff reader plugin from the JAI library your code must perform at least one of the following calls:
- Instantiate a JFrame
- Call Toolkit.getDefaultToolkit()
- Call Application.getApplication() (Mac OS X java extension)
While I still don't know the exact nature of the behavior, all evidence points to the fact that in a java program that needs to utilize the tiff image reader, you must set up the AWT windowing system in some manner. Even if your program (like my test utility) doesn't need to create a window or deal with the windowing system in any fashion, you must do one of the above methods in order to correctly register the tiff reader.
Sunday, July 11. 2010
Clone Only HEAD Using Git-SVN
A quick tip for git-svn users: When checking out an SVN repository where only the HEAD revision is desired, the following snippet may be useful:
git svn clone -$(svn log -q --limit 1 $SVN_URL | awk '/^r/{print $1}') $SVN_URL
The above snippet determines the most recent SVN revision number (using svn log) and passes it to git-svn-clone. This can also be added to git as an alias by adding the following to .gitconfig:
[alias]
svn-clone-head = "!f() { git svn clone -`svn log -q --limit 1 $1 | awk '/^r/{print $1}'` $1 $2; }; f"
For checking out the last $N commits, a similar convention can be used:
git svn clone -$(svn log -q --limit $N $SVN_URL | awk '/^r/{rev=$1};END{print rev}') $SVN_URL
Friday, July 9. 2010
HttpHandlers in Virtual Directories on IIS6
Background
I recently encountered an interesting problem related to how Virtual Directories interact with web.config when dealing with an HttpHandler. The website was virtually rooted at "/webapp", physically rooted at "C:\inetpub\wwwroot\webapp". Inside the application, I wanted "files" ("/webapp/files") to be physically rooted on another drive at "E:\files" so that large data files could be stored on a more appropriate drive. Furthermore, I wanted a custom Http Handler which would generate a few files inside "files" on the fly. Initially, this was setup by creating a virtual directory named "files" which pointed to "E:\files", the HttpHandler was placed in App_Code, and web.config contained the following:
<?xml version="1.0"?>
<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">
<system.web>
<httpHandlers>
<add verb="GET" path="files/generated.txt" type="WebApp.MyHttpHandler" />
</httpHandlers>
</system.web>
</configuration>
The Problem
As I quickly found out, this doesn't work (for several reasons, as we will see). First, requests for .txt files are not handled by isapi_aspnet.dll (by default), so whatever is done in the ASP.NET code is irrelevant because IIS will not call ASP.NET to handle the request. To fix this problem, the .txt extension can be added to the list of extensions handled by isapi_aspnet.dll (which will cause extra overhead as each request is run through the ASP.NET ISAPI handler, even when the file exists on disk) or the extension of the generated file can be changed to something mapped to isapi_aspnet.dll (like .aspx).
Next, unless the content of "files" is going to be substantially different from the rest of the application, the "files" Virtual Directory must not be a Virtual Application. If the "files" mapping is really a Virtual Application, it will not share code with the parent application so the HttpHandler class will not be found.
Finally, due to how the ASP.NET Configuration File Hierarchy and Inheritance works, web.config will be (essentially) re-applied in the virtual directory, so "files/generated.aspx" will be "files/files/generated.aspx" when considered from inside of the "files" virtual directory. To fix this (while not also creating a "/generated.aspx" alias as well), remove the httpHandlers section in the global web.config and create a web.config inside of the physical directory for "files" with path="generated.aspx"
Once all of the above steps are completed, the generated file should appear correctly and everything should be golden. If not, I strongly recommend replacing the real content of the custom HttpHandler with code that simply writes a string to the response and exits. This way any internal errors in the HttpHandler will not confound any issues with whether or not the handler is being called.
Thursday, July 8. 2010
Microsoft Sync Framework (v2) Not Thread Friendly
As a quick note for other developers that may be getting the same (difficult to understand) error, the Microsoft Sync Framework version 2 is not as thread-friendly as one might expect. The API documentation makes it clear that class instances in the framework are not thread-safe, however, this thread-unsafety goes farther. Even when the instance is protected by proper locking to prevent concurrent access, it may still error when accessed from multiple threads. For example, if a SyncOrchestrator and 2 FileSyncProviders are initialized on one thread and (later) Synchronize is called from another thread, the following exception will be thrown:
System.InvalidCastException: Specified cast is not valid.
at Microsoft.Synchronization.CoreInterop.SyncServicesClass.CreateSyncSession(ISyncProvider pDestinationProvider, ISyncProvider pSourceProvider)
at Microsoft.Synchronization.KnowledgeSyncOrchestrator.DoOneWaySyncHelper(SyncIdFormatGroup sourceIdFormats, SyncIdFormatGroup destinationIdFormats, KnowledgeSyncProviderConfiguration destinationConfiguration, SyncCallbacks DestinationCallbacks, ISyncProvider sourceProxy, ISyncProvider destinationProxy, ChangeDataAdapter callbackChangeDataAdapter, SyncDataConverter conflictDataConverter, Int32& changesApplied, Int32& changesFailed)
at Microsoft.Synchronization.KnowledgeSyncOrchestrator.DoOneWayKnowledgeSync(SyncDataConverter sourceConverter, SyncDataConverter destinationConverter, SyncProvider sourceProvider, SyncProvider destinationProvider, Int32& changesApplied, Int32& changesFailed)
at Microsoft.Synchronization.KnowledgeSyncOrchestrator.Synchronize()
at Microsoft.Synchronization.SyncOrchestrator.Synchronize()
at DigitalEngine.SyncMgrFileSync.SyncItem.Synchronize() in File.cs:line num
To work around such errors, make sure all sync instances are only accessed from a single thread.
