[sakai2-tcc] A popular feature request - sak-800

[Matthew Jones]

Another problem with a task like this and many others already in place
(duplicating a site, sending emails, processing xml) is that Sakai only has
one processing model: everything is done in the same application thread.
Some tasks known to be long running really should have a way to be queued up
and sent to a (ideally) out of cluster machine to be processed in the
background in a queue as time permits. The user would then be notified when
their job is completed. This would be for any task which will take longer
than a few seconds to complete, and the completion no other part of the
workflow is dependent on the completion of the task.

There is a strong need for a sakai queue service (or possibly even a quartz
job) where tools can register a simple class that takes a few defined
parameters which will execute and perform some task that is known to take
more time than a user is expected to wait for. Some of these have been
written for specific purposes like the eval email queue (
https://jira.sakaiproject.org/browse/EVALSYS-69), but nothing generic or
easy to register against.

As one immediate example, we had an issue recently at Umich with xmlencoder
where it crashed the server when exporting. (
https://jira.sakaiproject.org/browse/SAK-19862) It would have been nice to
have only had a out of cluster job processing machine that crashed. Also we
often in the past have had a "X minute" bug where if sites are not done
copying in a reasonable amount of time, or if the user hits abort, they are
corrupted. Even site creation from a template can sometimes run into this
problem depending on load level and app server speed.

-Matthew

On Wed, Feb 2, 2011 at 6:41 AM, Matthew Buckett <
matthew.buckett at oucs.ox.ac.uk> wrote:

> On 2 February 2011 11:30, Steve Swinsburg <steve.swinsburg at gmail.com>
> wrote:
> > Hi all,
> > AFAIK there are people running this in production. It would be good to
> get
> > some metrics from these people. Maybe reach out to the wider community to
> > see who else is running it. I think Oxford and Columbia perhaps.
>
> In short, it works, but as a programmer I don't like it.
>
> We needed to deploy it as ZIP support was needed for migration of
> material from our old LMS to Sakai and didn't have time to fix the
> error. It doesn't handle errors very well (eg over quota) and I don't
> like the model of uploading the ZIP into resources and then
> uncompressing it, as it means that if you are close to your quota
> limit you can't use the zip functionality. But having said that 95% of
> the time it works fine.
>
> We weren't too worries about the performance hit as we already allow
> WebDAV which has the potential to allow creation of 1000s of files
> very easily.
>
> We're currently running it against 2.6.x.
>
> --
>   Matthew Buckett
>   VLE Developer, LTG, Oxford University Computing Services
> _______________________________________________
> sakai2-tcc mailing list
> sakai2-tcc at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20110202/fff210a2/attachment.html