[Building Sakai] Resources keeling over with 1000+ files in a folder

[Warwick Chapman]

Except we've got no load balancers (yet) ...

-- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com


On Tue, Dec 7, 2010 at 3:00 PM, Stephen Marquard <stephen.marquard at uct.ac.za
> wrote:

> It could be a timeout value in the loadbalancer configuration, which is
> the longest time it will wait to get a reply back from the tomcat
> worker.
>
> Try accessing the tomcat directly at the http/s port (i.e. not via the
> loadbalancer) and see if that behaves differently.
>
> When we ran into it, it was slow but usable when one user accessed a
> folder like that (1000+ files) but caused general slowness when we had a
> whole class of users trying to do that at the same time.
>
> Regards
> Stephen
>
> Stephen Marquard, Learning Technologies Co-ordinator
> Centre for Educational Technology, University of Cape Town
> http://www.cet.uct.ac.za
> Email/IM/XMPP: stephen.marquard at uct.ac.za
> Phone: +27-21-650-5037 Cell: +27-83-500-5290
>
>
>
> >>> Warwick Chapman <warwickchapman at gmail.com> 12/7/2010 2:39 PM >>>
> So what causes the "Internal Server Error"?  Is there not a timeout
> value I
> can adjust somewhere?
>
> -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>
>
>
> On Tue, Dec 7, 2010 at 2:20 PM, Stephen Marquard
> <stephen.marquard at uct.ac.za
> > wrote:
>
> > Hi Warwick,
> >
> > We ran into this issue as well. It produces very high query volumes
> to
> > the database.
> >
> > The best workaround at this stage is to reorganize the files into a
> > deeper hierarchy so you don't get so many in one folder.
> >
> > Regards
> > Stephen
> >
> >
> > --
> > Stephen Marquard, Learning Technologies Co-ordinator
> > Centre for Educational Technology, University of Cape Town
> > http://www.cet.uct.ac.za
> > Email / IM (Jabber/XMPP): stephen.marquard at uct.ac.za
> > Phone: +27-21-650-5037 Cell: +27-83-500-5290
> >
> >
> > >>> Warwick Chapman <warwickchapman at gmail.com> 12/7/2010 1:50 PM >>>
> > Hi All
> >
> > We are using Umthombo resources to store several thousands PDFs
> > relating to
> > our candidate nomination process.  The forms are sorted into folders
> > for the
> > province each nominee comes from.  In South Africa, we have a great
> > many
> > more nominees in one province than in any other and this has
> resulted
> > in
> > that folder getting more than 1000+ PDFs alone.
> >
> > Listing that folder has become a pain.  It loads very slowly as
> > frequently
> > returns an HTTP500.  I cannot find anything obvious in the
> > catalina.out
> >
> > Any thoughts?  Can I change Resources to list a few items per page?
> >
> > -- Warwick Bruce Chapman | +27 83 7797 094 |
> http://warwickchapman.com
> >
> >
> >
> >
> >
> > ###
> > UNIVERSITY OF CAPE TOWN
> >
> > This e-mail is subject to the UCT ICT policies and e-mail disclaimer
> > published on our website at
> > http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable
> from
> > +27 21 650 9111. This e-mail is intended only for the person(s) to
> whom
> > it is addressed. If the e-mail has reached you in error, please
> notify
> > the author. If you are not the intended recipient of the e-mail you
> may
> > not use, disclose, copy, redirect or print the content. If this
> e-mail
> > is not related to the business of UCT it is sent by the sender in
> the
> > sender's individual capacity.
> >
> > ###
> >
> >
>
>
>
>
> ###
> UNIVERSITY OF CAPE TOWN
>
> This e-mail is subject to the UCT ICT policies and e-mail disclaimer
> published on our website at
> http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from
> +27 21 650 9111. This e-mail is intended only for the person(s) to whom
> it is addressed. If the e-mail has reached you in error, please notify
> the author. If you are not the intended recipient of the e-mail you may
> not use, disclose, copy, redirect or print the content. If this e-mail
> is not related to the business of UCT it is sent by the sender in the
> sender's individual capacity.
>
> ###
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20101208/32675e22/attachment.html