I'm curious about the specification of the server that you run. I run Java application servers at work (Tomcat, Glassfish, and Solr) on RedHat Enterprise Linux and have considerable experience troubleshooting performance issues. This may be apples vs oranges, but maybe worth a shot. Maybe you could share the p.nerd.nu server config with me so I could see first if there is anything glaringly obvious. Second, if the server isn't too beefy, I might try to replicate it in a VM and see what some of my monitoring tools can shed light on what causes the lag.
Yours, dacracot