[Building Sakai] How does Sakai respond to an overloaded database?

[R.P. Aditya]

On 2009-12-09, Joshua Swink <joshua.swink at gmail.com> wrote:
> We had a near-outage recently when the database server got heavily
> loaded. We had an unprecedented number of users on Sakai. Sakai was
> behaving like this:
>
> - very slow page loads (up to 10 minutes)
> - system load factor very high (10-15)
> - database connection pool being exhausted (one tomcat did exhaust it
> and needed to be restarted; the others survived)
>
> Obviously we need to add better hardware.
>
> What I want to know is whether the application's behavior can be
> attributed to the user load alone, or if the database's increased
> response time could be responsible. One hypothesis is that the
> application servers would have been fine if the database had been
> responding in a timely fashion. Another is that the usage would have
> led to Sakai bogging down even if the db had been fine.
>
> Any thoughts?

we've definitely seen this happen and typically the db connection pool
fills up and the appserver is unresponsive to most Sakai requests (which
need the db, which is most anything to do something useful) -- depending
on the problem, adding more connection pool connections might help but
probably just buys you a few seconds before those extra connections are
eaten up, but you also increase load on the db handling extra
connections so unless you know that the load came from elsewhere on the
db server, you shouldn't exacerbate the issue from Sakai...

Adi