[Deploying Sakai] New (old) scaleability issue - large folders

[Stephen Marquard]

Hi all,

We uncovered a scaleability issue recently that hit us in unusual circumstances.

A course site had a folder with 2700 items (more or less) in it. That all worked fine, until we had a class test with 400+ students, who all accessed the folder more or less at the same time.

It turns out the number of db queries required to list the items in a folder scales linearly with the number of items (around 5 x n). So viewing the folder with 2700 items in it generated something like 16000 queries. Needless to say that's not going for performance, especially when lots of people do that at the same time.

You can see if you have any "at risk" folders with a query similar to this:

select IN_COLLECTION, COUNT(IN_COLLECTION) As ColCount FROM CONTENT_RESOURCE GROUP BY IN_COLLECTION HAVING COUNT(IN_COLLECTION) > 100 order by ColCount DESC; 

Cheers
Stephen