[Building Sakai] Forums, Blog Wow and Wiki uselessly slow

Fri Mar 26 07:16:39 PDT 2010

I forgot to mention that we're planning on presenting a detailed 
explanation of what we found at the Denver conference this year, and would 
welcome feedback about what other community members' architectures are.
---------------------------------------------------------------------------
Matt Mize, Systems Administrator
 - Pay no attention to the man in the back office....
Matt.Mize at notes.udayton.edu
(937) 229-1024

UDit Department, University of Dayton
300 College Park, Dayton, OH, 45469-1302

mizematr at notes.udayton.edu 
Sent by: sakai-dev-bounces at collab.sakaiproject.org
03/26/10 10:12 AM

To
"Berg, Alan" <A.M.Berg at uva.nl>
cc
sakai-dev-bounces at collab.sakaiproject.org, 
sakai-dev at collab.sakaiproject.org
Subject
Re: [Building Sakai] Forums, Blog Wow and Wiki uselessly slow

The theory that we proposed to explain the behavior we are seeing was that 
because each Java "task" is being handled by, typically, one thread, our 
users were seeing slower response times than they needed. 

For example, loading a Question Pool took a little over 14 seconds on the 
T-Series, but about 7.5 on the x86 architecture.  Because Java is spawning 
these "long-running" threads it made sense to us that the T-Series wasn't 
right for this application.  We decided that when/if we get to the point 
where we need to handle more threads, we would add Tomcat servers in 
parallel on separate x86 architectures. 
---------------------------------------------------------------------------
Matt Mize, Systems Administrator
- Pay no attention to the man in the back office....
Matt.Mize at notes.udayton.edu
(937) 229-1024

UDit Department, University of Dayton
300 College Park, Dayton, OH, 45469-1302 

"Berg, Alan" <A.M.Berg at uva.nl> 
Sent by: sakai-dev-bounces at collab.sakaiproject.org 
03/26/10 05:24 AM 

To
"Gerwood Stewart" <gstewar8 at une.edu.au>, 
<sakai-dev at collab.sakaiproject.org> 
cc

Subject
Re: [Building Sakai] Forums, Blog Wow and Wiki uselessly slow

Hi all,

I am suspicious of XX:MaxPermSize=256m. If we do that on qa1-nl, then 
sometimes we run out of perm space. How about MaxPermSize=512m
Further we find that our Sun/Solaris T2000 servers are very strong for 
Java applications because the CPU's are optimised for integer work and 
lots of threads. However, we are also moving  services slowly to Vmware 
for consolidation reasons.

Our uPortal environment is on VMware, the minor gotcha is that we run 
Linux on top and that each thread costs 2MB extra of native memory.

Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg

-----Original Message-----
From: sakai-dev-bounces at collab.sakaiproject.org on behalf of Gerwood 
Stewart
Sent: Thu 3/25/2010 23:05
To: sakai-dev at collab.sakaiproject.org
Subject: Re: [Building Sakai] Forums, Blog Wow and Wiki uselessly slow

Thanks for the reply Matt

We already running on x86. Though you did remind me that we are running on 
VMWare ESX.

We are running Oracle 10g RAC and the DBAs believe that it's not 
struggling. They have identified slow queries that could contribute and 
we've added indexes where possible, but it still seems somewhat illogical 
to be running that slow in those tools, general navigation is quite quick 
as well.

These are our Java opts (that I think apply), this is per node:
-ms3072m -mx3072m -XX:NewSize=128m -XX:MaxNewSize=128m -XX:SurvivorRatio=4 
-XX:TargetSurvivorRatio=90 -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode 
-XX:+CMSIncrementalPacing -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC 
-XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseTLAB 
-XX:+CMSClassUnloadingEnabled -XX:+CMSClassUnloadingEnabled 
-XX:CodeCacheMinimumFreeSpace=2M -XX:ReservedCodeCacheSize=64M 
-XX:MaxTenuringThreshold=10

Gerwood

From: mizematr at notes.udayton.edu [mailto:mizematr at notes.udayton.edu]
Sent: Friday, 26 March 2010 12:00 AM
To: Stephen Marquard
Cc: Gerwood Stewart; sakai-dev at collab.sakaiproject.org; 
sakai-dev-bounces at collab.sakaiproject.org
Subject: Re: [Building Sakai] Forums, Blog Wow and Wiki uselessly slow

Gerwood,

We recently overcame a slowness across the board issue that was related to 
the hardware.  Our application server was running on a Sun UltraSparc 
T-Series architecture, which must not be compatible with Java threads.  We 
ran some benchmarks on various architectures and they showed using x86 was 
the best way to go.  We made the switch about three weeks ago and saw a 
40-60% performance increase across the board.
---------------------------------------------------------------------------
Matt Mize, Systems Administrator
- Pay no attention to the man in the back office....
Matt.Mize at notes.udayton.edu
(937) 229-1024

UDit Department, University of Dayton
300 College Park, Dayton, OH, 45469-1302

"Stephen Marquard" <stephen.marquard at uct.ac.za>
Sent by: sakai-dev-bounces at collab.sakaiproject.org

03/25/10 01:44 AM

To

"sakai-dev at collab.sakaiproject.org" <sakai-dev at collab.sakaiproject.org>, 
"Gerwood Stewart" <gstewar8 at une.edu.au>

cc

Subject

Re: [Building Sakai] Forums, Blog Wow and Wiki uselessly slow

Hi Gerwood,

That sounds odd. Forums has some slowdown with large numbers of posts in a 
topic, but we've never seen any speed issues at all with blogwow or wiki.

I would look at database performance firstly (perhaps missing indexes?) 
and secondly at your JVM memory options (available memory, GC times, 
etc.).

Regards
Stephen

>>> Gerwood Stewart <gstewar8 at une.edu.au> 3/25/2010 6:01 AM >>>
We are having a lot of issues with the speed (and thus usability) of the 
Forums, Blog Wow and Wiki tools.

It's bad enough that I don't believe most institutions using Sakai would 
be seeing similar results.

Does anyone know of what issues we might be encountering?

We are running on RHE boxes with an Oracle 10g backend.

Help would be greatly appreciated.

Thanks
Gerwood Stewart

______________________________________________________________________________________________

UNIVERSITY OF CAPE TOWN

This e-mail is subject to the UCT ICT policies and e-mail disclaimer 
published on our website at 
http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from 
+27 21 650 4500. This e-mail is intended only for the person(s) to whom it 
is addressed. If the e-mail has reached you in error, please notify the 
author. If you are not the intended recipient of the e-mail you may not 
use, disclose, copy, redirect or print the content. If this e-mail is not 
related to the business of UCT it is sent by the sender in the sender's 
individual capacity.

_____________________________________________________________________________________________________

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org
<http://www.uct.ac.za/about/policies/emaildisclaimer/>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to 
sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of 
"unsubscribe"
<http://collab.sakaiproject.org/mailman/listinfo/sakai-dev>
_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to 
sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of 
"unsubscribe" _______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to 
sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of 
"unsubscribe"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20100326/c7e6a1c8/attachment.html 