[Building Sakai] JVM memory usage up after upgrading to 2.9.x

[Sam Ottenhoff]

So I guess the relevant questions are

a) is your Tomcat running out of memory?

b) what is taking up all of the memory?  Heap dump => Eclipse MAT will give
you a great idea of what is in your heap.


On Tue, Jul 30, 2013 at 6:31 PM, Kusnetz, Jeremy <JKusnetz at apus.edu> wrote:

>  We just upgraded from the rSmart 2.8 CLE to 2.9.x community (close to
> 2.9.3 rc01).****
>
> ** **
>
> I’m seeing a significant amount of JVM memory usage that we were not
> seeing previously, and the GC doesn’t seem to be happening as frequently.*
> ***
>
> ** **
>
> We switched from Tomcat 5.5 to Tomcat 7, but we are still running on
> Java6, just a slightly newer version.****
>
> ** **
>
> We pretty much kept the same tomcat startup options as before, but GC
> seems to be working very differently now.  Really the only thing changed is
> adding  -Dorg.apache.jasper.compiler.Parser.STRICT_WHITESPACE=false****
>
> ** **
>
> ** **
>
> -server -d64 -XX:+ForceTimeHighResolution -XX:+UseLargePages -Xms7g -Xmx7g
> -XX:PermSize=768m -XX:MaxPermSize=768m -XX:NewSize=1536m
> -XX:MaxNewSize=1536m -Djava.awt.headless=true -Dsakai.home=/etc/tomcat100
> -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing
> -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -XX:SurvivorRatio=6
> -XX:+PrintGCDetails -XX:+PrintHeapAtGC -Dnetworkaddress.cache.ttl=60
> -Dsun.rmi.dgc.client.gcInterval=3600000
> -Dsun.rmi.dgc.server.gcInterval=3600000
> -Dcom.sun.management.snmp.port=41003
> -Dcom.sun.management.snmp.acl.file=/usr/local/apus/tomcat_servers/etc/snmp100.acl
> -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false
> -Dhttp.agent=Sakai -XX:ParallelGCThreads=20
> -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=80
> -XX:CMSMaxAbortablePrecleanTime=60000 -XX:+UseCMSCompactAtFullCollection
> -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCTimeStamps -XX:+PrintGCDetails
> -Xloggc:/usr/local/apus/tomcat_servers/servers/tomcat100/logs/sakai.vgc
> -XX:TargetSurvivorRatio=90 -XX:+UseBiasedLocking
> -Dsun.lang.ClassLoader.allowArraySyntax=true
> -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSIncrementalSafetyFactor=75
> -Dorg.apache.jasper.compiler.Parser.STRICT_WHITESPACE=false****
>
> ** **
>
> Here is a 1 day graph.  You can see the CMS Old Gen is working very
> differently than before, it just seems to grow and  grow and then do a
> large GC, where before it was doing lots of GCs.****
>
> ** **
>
> [image:
> https://edge.apus.edu/monitor/graphs/stsakai5/stsakai5-javamemused-day.png]
> ****
>
> ** **
>
> Here is a full week of JVM usage.  Sundays are our big night and we had
> about double the number of users on the node, although you can see our
> memory usage for today is much higher than normal.****
>
> ** **
>
> [image:
> https://edge.apus.edu/monitor/graphs/stsakai5/stsakai5-javamemused-week.png]
> ****
>
> This message is private and confidential. If you have received it in
> error, please notify the sender and remove it from your system.
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20130801/4f870789/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 50345 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20130801/4f870789/attachment.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 47095 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20130801/4f870789/attachment-0001.png