One problem with 64-bit Java is the amount of Memory it occupies. As of 2011 we feel that all application profiles we have encountered do not require 64-bit Java as they can not take advantage of using more than than 1.6GB of memory. In many cases it is better to scale horizontally (running multiple JVMs) as opposed to using one giant JVM.
That is not to say not to user 64-bit servers. In fact we are moving to use entirely 64-bit systems these days most importantly to take advantage of all the memories (most of our servers use over 4 GB).
More details explaining the mechanics can be put here or linked in another document.