[Deploying Sakai] Production Problems with Garbage Collection

[Beth Kirschner]

Hi all,

  We've been looking into some serious production problems which have required frequent restarting of our production servers due to the performance impact of the server being mired in continuous Garbage Collection. The only clue we have from the logs is "Concurrent Mode Failure" messages in logs. These problems started shortly after updating to Java 1.6 and Sakai 2.7.1.

  I'm looking for help and advice, specifically: Has anyone seen these types of issues in Sakai production?

  On a related note, I'm wondering if anyone has specific diagnostic tools that they use for production Sakai systems, to allow for better analysis of: heap dumps, thread dumps, tomcat monitoring (e.g. psi-probe, jstack, jconsole, visualvm), and log file analysis (e.g. swatch)?

Thanks!
- Beth