[Building Sakai] Java 6 seems to be a problem for us

[Charles Hedrick]

On Oct 13, 2009, at 11:18:37 AM, Aaron Zeckoski wrote:

>> Have you tried running without the tuning params and seeing where the
>> auto tuning settles? You are currently forcing the JVM to run a  
>> series
>> of params that might be brutally inefficient and it can be eye- 
>> opening
>> to see how the JVM tunes itself. You may have already done this and
>> this might be how you settled on these numbers but I thought I would
>> ask anyway.
>



With 5 I started with recommendations from a Sun tuning paper. In 5  
more tuning is needed than in 6. Indeed 6 tends to default to  
something close to the recommendations we followed for tuning 5.

For 5:

-d64 -Xmx12000m -Xms12000m  is necessary.

-XX:+UseConcMarkSweepGC -XX:+UseParNewGC  is the basic choice of GCs,  
which won't default in 5.

-Xmn3g  was done fairly early, to give us a large enough eden to  
survive uploading large files. I believe we allow 1 GB uploads. We had  
problems without this.

-XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 - 
XX:MaxTenuringThreshold=15  is from a Sun tuning paper.  In 6 we tried  
without it, and in retrospect I'm not sure it's needed in 6. However  
in 6, it tends to default to one or two generations, which a number of  
people think is too small.

-XX:MaxPermSize=512m -XX:PermSize=512m    max is certainly needed.  
I've watched with jstat and seen both the full gc's caused by starting  
with a small permsize, and the pauses caused by those GCs.

-XX:+UseMembar -XX:-UseThreadPriorities     is probably unnecesary, as  
I've said. However I'd tried it both way and not seen any performance  
problem. I'd probably start without this, particularly in 6.

-XX:+DisableExplicitGC   is recommended commonly. We really don't want  
audio uploads in Samigo to trigger a full GC (this is fixed in trunk).


For 6, if we get the pauses fixed so we can go back I think the  
following would do:

JAVA_OPTS=" -d64 -Xmx12000m -Xms12000m  -XX:+UseConcMarkSweepGC -XX: 
+UseParNewGC -XX:CMSInitiatingOccupancyFraction=75 - 
XX:MaxPermSize=512m -XX:PermSize=512m  -XX:+DisableExplicitGC

For 6 I decided to let it tune as much as possible, so I started with

JAVA_OPTS=" -d64 -Xmx12000m -Xms12000m  -XX:+UseConcMarkSweepGC -XX: 
+UseParNewGC -XX:MaxPermSize=512m -XX:PermSize=64m "

I can tell you that none of the changes  caused a problem. Some are  
probably just unnecessary. The things I feel strongly about in 6 based  
on our experience are the CMSInitiatingOccupancyFraction and starting  
permsize larger. I think you'll find a number of experts that think  
the default initiating point is way too high in 6, and that too few  
survivor generations are used by default. At any rate, with the  
default CMSInitiatinngOccupancyFraction we saw slow full GCs. I  
suspected memory fragmentation, but it's possible that whatever  
problem is resulting in our slow minor GCs was actually at fault, and  
when that is fixed we won't need this either.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2421 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20091014/d541f88c/attachment.bin