[Building Sakai] How does Sakai respond to an overloaded database?

[Joshua Swink]

We had a near-outage recently when the database server got heavily
loaded. We had an unprecedented number of users on Sakai. Sakai was
behaving like this:

- very slow page loads (up to 10 minutes)
- system load factor very high (10-15)
- database connection pool being exhausted (one tomcat did exhaust it
and needed to be restarted; the others survived)

Obviously we need to add better hardware.

What I want to know is whether the application's behavior can be
attributed to the user load alone, or if the database's increased
response time could be responsible. One hypothesis is that the
application servers would have been fine if the database had been
responding in a timely fashion. Another is that the usage would have
led to Sakai bogging down even if the db had been fine.

Any thoughts?

Josh