Running out of heap is never fun, and as applications grow, both in terms of usage and size, this has become a problem for many companies hosting big applications. A number of years ago, when EM64T/AMD64/x86-64 (or whatever you want to call it)-capable servers entered the ring, the most common way to overcome the 1,n to 2,n GB limit (depending on the OS) of the 32bit JVM was to install a 64bit JVM.
The newly gained nirvana, however, was short-lived. We saw the applications consume a lot more heap than they ever did before and still, we were running out of memory. The reason for this is quite simple: Pointers were now twice as big.
This may not a huge issue, you may think... yet, rethinking is necessary. Imagine an array. Every pointer in an array is now twice as big. Every object pointer (and you know that there are quite a lot of objects in your JVM) now consumed twice as much memory as it did before.
I often suggested to my customers to rethink their 64bit strategy unless they are willing to add a lot more memory to their machine in order to get an actual benefit. This was often combined with GC tweaking. After all, the larger the heap, the more there is to clean up once GC occours (a gross over-generalization, I know).
There is a solution to our pains, though. This year Sun released a Performance Release of the JRE, which adds a the UseCompressedOops flag. Don't confuse the acronym OOP with object-oriented programming, in this very case it stands for ordinary object pointers. By starting your JVM with this flag, you tell it to behave like a 32bit/64bit-hybrid, as it will use 32bit object pointers and thus conserves a respectable amount of memory (generally about 30%) which your application can use and at the same time allows you to break free from the 32bit JVM heap size and register limitations mentioned previously.
Why don't you give it a try yourself? Simply start your application with the command line addition -XX:+UseCompressedOops and see what happens. I am quite sure you will see an improvement.
The only downside with all this is, that you will have to register to get the aforementioned Performance Release.
