[Building Sakai] An odd behavior when adding participants under a high load

[Shoji Kajita]

Hi Matt,

Our server is also on a single node. I don't see any odd behavior under normal load.

Best regards,
Shoji

At Fri, 5 Oct 2012 10:13:45 -0400,
Matthew Jones wrote:
> 
> [1  <multipart/alternative (7bit)>]
> [1.1  <text/plain; ISO-8859-1 (7bit)>]
> Changing group membership fires SECURE_UPDATE_AUTHZ_GROUP which was caught
> by the original patch. i can see how load could slow it down a little but
> not minutes. Testing locally works for me, but I'm only on a single node.
> Is this also a single server? Do you see this under normal load?
> 
> On Fri, Oct 5, 2012 at 9:42 AM, David Horwitz <david.horwitz at uct.ac.za>wrote:
> 
> > Hi Shoji,
> >
> > What version of the Kernel are you running? This sounds like a caching
> > issue - perhaps related to KNL-600. I know there has been some work on
> > fixes in this code recently ...
> >
> > D
> > On 10/05/2012 03:37 PM, Shoji Kajita wrote:
> > > Hi Sakai Dev,
> > >
> > > We have started using our Sakai 2.9.x Pilot Server in real classes
> > > since October 1st for 2012 second semester at Kyoto University, but we
> > > have been struggling from an odd behavior.
> > >
> > > During a high load period (18:30-18:55 last evening as shown in the
> > > jconsole summary screen shot http://db.tt/y8uNXzWM) produced by almost
> > > 100 concurrent access from two physical class rooms, we saw the
> > > following odd behavior when two instructors independently tried to add
> > > students in their same course worksite:
> > >
> > >   1. the first instructor added about 50 students by using "Add
> > >      Participants" of Site Info, but no students could not see the
> > >      course worksite in their screen. Refreshing pages by Web browser
> > >      or re-loging in did not help. After several minutes (maybe seven
> > >      minutes or so), most of students could see the course worksite
> > >      suddenly.
> > >
> > >      During the not-shown period, I (using admin account) sudo-ed a
> > >      student who should have the right to access to the course
> > >      worksite, and I could see the course worksite properly.
> > >
> > >   2. After 10 minutes, the second instructor did the almost same thing
> > >      and no students could not see for five minutes or so.
> > >
> > > I have learned that the second instructor also had the same odd
> > > behavior in his class during this evening class.
> > >
> > > I really appreciate if someone gives me any suggestions or pointers
> > > for this odd behavior.
> > >
> > > Our Sakai 2.9.x Pilot Server is running under the following environment:
> > >
> > >    a. RedHat 6.2 on VMWare (16GB main memory, 2 CPUs)
> > >
> > >    b. Tomcat 7.0.29 with the following JAVA_OPTS:
> > >
> > >       #  12GB in total
> > >       #    Eden size (3GB) = New size (4GB) - 2 x Survivor space size
> > (512MB)
> > >       #    Survivor space size (512MB) = New size (4GB) / survivor ratio
> > (8)
> > >
> > >       export JAVA_OPTS='-server -Xms12288m -Xmx12288m -XX:PermSize=4096m
> > -XX:MaxPermSize=4096m
> > >       -XX:NewSize=4096m -XX:MaxNewSize=4096m -Djava.awt.headless=true
> > -Dsun.lang.ClassLoader.allowArraySyntax=true
> > >       -Dcom.sun.management.jmxremote
> > -Dorg.apache.jasper.compiler.Parser.STRICT_WHITESPACE=false
> > >       -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false
> > -Dhttp.agent=Sakai
> > >       -Duser.language=ja -Duser.region=JP -Dfile.encoding=UTF-8
> > -XX:+DisableExplicitGC -XX:+PrintGCTimeStamps
> > >       -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC
> > -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled
> > >       -XX:+UseParNewGC -verbose:gc -XX:+PrintClassHistogram
> > -XX:+HeapDumpOnOutOfMemoryError -XX:+PrintReferenceGC
> > >       -Xloggc:$CH/logs/loggc.out'
> > >
> > >    c. Sakai 2.9.x (rev 113440) but kernel (rev 113905 with
> > KNL-972.patch) and site-manage (rev 113912)
> > >
> > >    d. Oracle 11g R1 on RedHat 6.2/VMWare
> > >
> > > Best regards,
> > > Shoji Kajita
> > > ---
> > > Shoji Kajita
> > > IT Planning Office, IIMC, Kyoto University, Japan
> > > http://about.me/shojikajita
> > > _______________________________________________
> > > sakai-dev mailing list
> > > sakai-dev at collab.sakaiproject.org
> > > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> > >
> > > TO UNSUBSCRIBE: send email to
> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> > "unsubscribe"
> > >
> >
> >
> > _______________________________________________
> > sakai-dev mailing list
> > sakai-dev at collab.sakaiproject.org
> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >
> > TO UNSUBSCRIBE: send email to
> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> > "unsubscribe"
> >
> [1.2  <text/html; ISO-8859-1 (quoted-printable)>]
> 
> [2  <text/plain; us-ascii (7bit)>]
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> 
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"