[Building Sakai] Load test of Sakai

[Peter Crowther]

2009/8/20 Daniel Lind <Daniel.Lind at umdac.umu.se>:
> “Think time”: 30s (deviation 10s)

OK, so this isn't just a load of users hammering at T&Q non-stop.  Great.

> The load on the machine is approximately  around 7 at most. What happens is
> that, when too many test users are taking the test, Sakai throws many
> exception like the one below:
>
> org.springframework.orm.hibernate3.HibernateOptimisticLockingFailureException:
> Batch update returned unexpected row count from update [4]; actual row
> count: 0; expected: 1; nested exception is
> org.hibernate.StaleStateException: Batch update returned unexpected row
> count from update [4]; actual row count: 0; expected: 1
>
> Caused by: org.hibernate.StaleStateException: Batch update returned
> unexpected row count from update [4]; actual row count: 0; expected: 1
[...]
>         at $Proxy61.saveOrUpdateAssessmentGrading(Unknown Source)
>
>         at
> org.sakaiproject.tool.assessment.services.GradingService.saveOrUpdateAssessmentGrading(GradingService.java:486)
[...]

> Is this about the limit for this setup, or do you think we could expect many
> more concurrent users?

Firstly: to my untutored eye that looks like a race condition in
GradingService.saveOrUpdateAssessmentGrading - a bug, rather than a
capacity problem.

You say that Tomcat's CPU use is high, and that seems to be the
limiting factor.  I suspect that, in order to chase this further,
someone would need to profile the code and find out what was taking
the time.  With a 30 second request interval, your figures show
there's a capacity limit at about 15 requests per second.  With 4
cores, that's about 0.25 CPU-second per request, which feels like a
lot of CPU per request.

- Peter