[Building Sakai] Monitoring Samigo in a big test

[Daniel Merino]

Hi everybody.

Time ago, we had a system crash while one Samigo test was being made by 
250+ users. System memory was filled up and our production environment 
gradually slowed down until it crashed.

This happened after using the audio recorder applet, when users uploaded 
the recorded file all at the same time. After a reboot, another part of 
the exam with other kind of questions also produced the same system 
slowness and halt.

We use Samigo 2.9.2 on Sakai 2.9.2 and the system which collapsed had 
the next configuration (1). Now we have extended that system to this one 
(2).

And soon we are going to have another test with a high number of users. 
We plan to monitor that test and try to look into the results to find 
what has happened in case it fails. Or maybe send these results to the 
list so other more expert users can have a look on it.

We are going to use YourKit to store the JVM internal use and JConsole 
to watch memory and CPU usage. If somebody can provide some advice about 
other tools or procedures that we could use to have better and more 
useful results, we would be very grateful.

Of course, if our configuration has any bottleneck or if any work has 
been done on this performance topic in Samigo's code, I would like to 
know in order to fix it here if I can.

Many thanks in advance.
Best regards.

-------------------------------

(1) 1 active Tomcat.

Processor: 
http://ark.intel.com/products/27219/Intel-Xeon-Processor-5160-4M-Cache-3_00-GHz-1333-MHz-FSB?wapkw=5160

JAVA_OPTS: '-server -d64 -Xms10000m -Xmx10000m -XX:PermSize=512m 
-XX:MaxPermSize=512m -Djava.awt.headless=true -Dhttp.agent=Sakai 
-Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false 
-Dsun.lang.ClassLoader.allowArraySyntax=true -Duser.language=es 
-Duser.region=ES -Dsakai.cookieName=SAKAISESSIONID 
-XX:+UseConcMarkSweepGC -XX:+ExplicitGCInvokesConcurrent 
-XX:+CMSIncrementalMode'

(2) 2 active Tomcat.

Processor 1: 
http://ark.intel.com/products/27219/Intel-Xeon-Processor-5160-4M-Cache-3_00-GHz-1333-MHz-FSB?wapkw=5160 

Processor 2: 
http://ark.intel.com/products/52583/Intel-Xeon-Processor-E5606-8M-Cache-2_13-GHz-4_80-GTs-Intel-QPI 


JAVA_OPTS: '-server -d64 -Xms11000m -Xmx11000m -Xmn3g -XX:PermSize=512m 
-XX:MaxPermSize=512m -Djava.awt.headless=true -Dhttp.agent=Sakai 
-Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false 
-Dsun.lang.ClassLoader.allowArraySyntax=true -Duser.language=es 
-Duser.region=ES -Dsakai.cookieName=SAKAISESSIONID 
-XX:+UseConcMarkSweepGC -XX:+ExplicitGCInvokesConcurrent 
-XX:+CMSIncrementalMode'

-- 
Daniel Merino Echeverría
daniel.merino at unavarra.es
Gestor de E-learning - Centro Superior de Innovación Educativa.
Tfno: 948-168489 - Universidad Pública de Navarra.