[Building Sakai] Java 6 garbage collection

[Charles Hedrick]

We're using Java 6 with Sakai 2.6. As the load has come on during Sept  
we've started seeing pauses due to garbage collection. Some are  
unavoidable. However some are a result of changes in defaults between  
Java 5 and Java 6. Most of the changes are good, but one is not: When  
using CMS (the concurrent low-pause collector), it used to be that a  
GC of old was by default started when old was 68% full. That's a bit  
too conservative, so many people set  - 
XX:CMSInitiatingOccupancyFraction to a higher value. In Java 6, the  
default is more complex, but in many cases it works out to be  
something over 90%. I've seen claims that it's 92%. That sounds about  
right. This is far too high a value. It can result in full GC's and  
also fragmentation of memory leading to long minor GCs. If you carry  
over your old Java 5 value for CMSInitiatingOccupancyFraction you  
should be fine, but if you let it default, you are likely to be in for  
trouble. I'm currently using 80%. Here are our current values. This is  
with 1.6.0u16.

JAVA_OPTS=" -d64 -Dsun.lang.ClassLoader.allowArraySyntax=true - 
Xmx13000m -Xms13000m -Xmn3g -XX:+UseConcMarkSweepGC -XX:+UseParNewGC - 
XX:CMSInitiatingOccupancyFraction=80 -XX:MaxPermSize=512m - 
XX:PermSize=64m "

I actually recommend setting PermSize to a value appropriate to your  
installation, by using jstat to see what perm space usage actually is.  
That will save you from a few long GCs early in your JVM's lifetime.

With these tunings, pauses are normally OK. However certain behavior  
can get a JVM into a state where minor GC's are 20 - 30 sec. In my  
opinion the new GC ("G1") is the only final solution to GC-related  
pauses in large JVMs. We're trying that on one system, but I'm not  
sure it's ready for production use.