[Building Sakai] [maint] alternative to webdav in sakai
Adrian Fish
a.fish at lancaster.ac.uk
Fri Mar 11 03:45:29 PST 2011
I have a particular use case for the unzipping functionality. We're
running a tool here called LessonBuilder (not the Rutgers one, our own)
that allows you to upload zip files from flash authoring products, such
as Camtasia and Articulate, and the Wimba Create product. When you
upload them they are expanded into a directory inside the webapp and a
link is created in the lesson builder content pointing to the index file
of the expanded content.
It would make a lot more sense to just unzip the content directly in the
resources tool and link to it from there. I looked at SAK-800 myself at
the time of writing the tool and got put off by the discussions around
it. It would be good to revisit it I reckon as it would make a generally
useful piece of functionality.
Cheers,
Adrian.
On 10/03/2011 17:21, Adam Marshall wrote:
>
> I think an unzip (with no zip) would still be very useful ineed.
>
> adam
>
> *From:*sakai-dev-bounces at collab.sakaiproject.org
> [mailto:sakai-dev-bounces at collab.sakaiproject.org] *On Behalf Of
> *Nicola Monat-Jacobs
> *Sent:* 10 March 2011 16:54
> *To:* John Bush
> *Cc:* <mt at collab.sakaiproject.org>; sakai-dev at collab.sakaiproject.org
> *Subject:* Re: [Building Sakai] [maint] alternative to webdav in sakai
>
> As far as I can tell from reading the JIRAs, the issues with SAK-800
> were on the zipping side, rather than the unzipping side, right?
> Unless there was an issue with the 'upload and unzip a zip into
> resources' functionality that I'm not catching in the various JIRAs,
> what about just implemented that part of the functionality? It seems a
> shame to reject a feature because it's partner misbehaves.
>
> Bulk uploading was the original use case that you were looking at,
> right John?
>
> - Nicola
>
> On Mar 9, 2011, at 3:51 PM, John Bush wrote:
>
>
>
> I spent a little time looking for an alternative today that doesn't
> involve webdav or SAK-800 (you guys aren't making me feel fuzzy about
> that idea). I can't find a flash widget that will let you attach
> folders, and I think that is key, for people who are uploading say a
> bunch of html content that is all linked together.
>
> I've settled on jupload which is an applet:
> http://jupload.sourceforge.net/
>
> It does drag and drop, supports attaching folders, and can do a http
> PUT, which I'm not sure will be valuable or not.
>
> I'm going to spend some time prototyping something that would say plug
> this into the upload page in resources. We'll see where this goes...
>
> On Wed, Mar 9, 2011 at 2:27 PM, Matthew Jones <jonespm at umich.edu
> <mailto:jonespm at umich.edu>> wrote:
>
> I agree with these comments, I don't think you're being paranoid.
> There were too many easy to reproduce edge cases that really could
> start up a nearly denial of service back with the last patch that was
> not tested. Only the "happy path" of a regular zip file with a few
> files in it was tested. If you loaded up a zip file with thousands
> (hundreds of thousands?) of zero byte files then the performance was
> much different. Also I remember being able to generate some file names
> that generated exceptions that back then were silently logged.
>
> Having this process that has a good chance of taking longer than a few
> second in the request thread is a long standing problem of Sakai.
> Something like this (a task that is no other workflow depends on it's
> completion and is potentially long running) really needs to be
> backgrounded or moved off to some type of job processing server. And
> when it's processed, ideally, the user notified that their (long
> running) job is complete. No mechanism currently exists for this in
> Sakai. :(
>
> It's also unfortunate that tomcat or java has no (seemingly easy?) way
> to set any type of thread limits or timeouts like php (set_time_limit)
> does with execution time. This might at least make administrators feel
> a little safer. I guess you can setup an ExecutorService and run your
> function in that? Still, gets into some pretty complex stuff for a
> "simple" feature.
>
> I agree that this feature would be really awesome for a user but seems
> like a scary addiition from an operations perspective.
>
> On Wed, Mar 9, 2011 at 4:07 PM, Noah Botimer <botimer at umich.edu
> <mailto:botimer at umich.edu>> wrote:
>
> Sure. When the original patch was merged, it came with a number of
> quality and completeness problems.
>
> https://jira.sakaiproject.org/browse/SAK-16036 and
> https://jira.sakaiproject.org/browse/KNL-155 capture the work that was
> done in the closing moments of the 2.6 release to disable the feature.
> The problems were discovered late and the fix was an ugly combination
> of properties and commented code.
>
> (I refer to this pretty broken code lying latent in trunk without a
> quality plan or much notice as being "parachuted in".)
>
> Perhaps the work done for KNL-273 addresses most of the problems, but
> I am not comfortable without seeing a standalone, cohesive write-up of
> the feature and its behavior. Tracking this across a handful of
> SAK/KNL tickets, some emails, and fifty-some JIRA comments just
> doesn't draw a clear picture.
>
> Specific technical questions are around the library used, performance,
> request-thread processing, mime-type settings, content scanning, and
> quota ramifications. We may or may not have a clear implementation
> plan that addresses these. I am especially concerned about the
> processing in the request thread and the impact on the database pool.
>
> Maybe I'm paranoid or maybe these have all been sufficiently
> addressed. But I would strongly urge those interested to do a complete
> and accurate write-up outside of the smattering of JIRA tickets, so we
> can make a conscious community decision.
>
> By the way, many thanks to Bryan and whomever else for working on a
> real user need.
>
> Thanks,
> -Noah
>
>
> On Mar 9, 2011, at 3:16 PM, Sam Ottenhoff wrote:
>
> > I know nothing about the previous issues or the mitigation that was
> > done. SAK-800 comments don't seem to reflect any local catastrophes.
> > Can you give us some background?
> >
> > --Sam
> >
> > On 3/9/2011 3:11 PM, Noah Botimer wrote:
> >> With all due respect to the work done and the spoken need, I think
> there are still significant concerns with this approach. It solves a
> class of problem and introduces another.
> >>
> >> I am not familiar with the extent of the work done recently, but I
> am very leery of SAK-800. I will be disappointed if it gets parachuted
> in again. Even our mitigation was sloppy last time.
> >>
> >> This absolutely requires review and discussion on list and the TCC
> should form an opinion of the best path. This is not simple bug fixing
> or minor feature enhancement, given the technical and policy issues
> arising.
> >>
> >> None of this is to discourage movement forward, but it is a humble
> request that special care be taken in an area that has really stung us
> before.
> >>
> >> Thanks,
> >> -Noah
> >>
> >> On Mar 9, 2011, at 12:25 PM, Bryan Holladay wrote:
> >>
> >>> Thanks David... Anyone who wants it can take it. I re-assigned it
> to the KERNEL TEAM in mid November.
> >>>
> >>> -Bryan
> >>>
> >>> On Mar 9, 2011, at 12:17 PM, DAVID ROLDAN MARTINEZ wrote:
> >>>
> >>>> Bryan,
> >>>>
> >>>> I offer myself to do this (or provide you help if you prefer to
> manage this yourself) and leave hardcoded messages out of kernel so
> that they will be handled at resources tool. But, as I'm not a member
> of the Kernel-team, I would like to get their feedback before to start
> working.
> >>>>
> >>>> Cheers,
> >>>> David
> >>>> ________________________________________
> >>>> De: sakai-dev-bounces at collab.sakaiproject.org
> <mailto:sakai-dev-bounces at collab.sakaiproject.org>
> [sakai-dev-bounces at collab.sakaiproject.org
> <mailto:sakai-dev-bounces at collab.sakaiproject.org>] En nombre de Bryan
> Holladay [holladay at longsight.com <mailto:holladay at longsight.com>]
> >>>> Enviado el: miércoles, 09 de marzo de 2011 17:51
> >>>> Para: Adam Marshall
> >>>> CC: sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>
> >>>> Asunto: Re: [Building Sakai] alternative to webdav in sakai
> >>>>
> >>>> I've fixed this issue in
> https://jira.sakaiproject.org/browse/KNL-273, along with other issues
> with the unzip facility.
> >>>>
> >>>> The only reason it hasn't been put into trunk is a best code
> practices issue I wasn't able to figure out. It's noted in the
> comments and is waiting for someone with a better understanding of the
> kernel code to look at it.
> >>>>
> >>>> Other than that, it's a good patch and works fine and would be
> safe to run.
> >>>>
> >>>> Thanks,
> >>>> Bryan
> >>>>
> >>>>
> >>>> On Mar 9, 2011, at 11:41 AM, Adam Marshall wrote:
> >>>>
> >>>> We use the unzip facility -- it works OK for extraction but is
> not good for creating a zip file. You cannot zip from the root folder!
> >>>>
> >>>> From: sakai-dev-bounces at collab.sakaiproject.org
> <mailto:sakai-dev-bounces at collab.sakaiproject.org><mailto:sakai-dev-bounces at collab.sakaiproject.org
> <mailto:sakai-dev-bounces at collab.sakaiproject.org>>
> [mailto:sakai-dev-bounces at collab.sakaiproject.org
> <mailto:sakai-dev-bounces at collab.sakaiproject.org>] On Behalf Of Sam
> Ottenhoff
> >>>> Sent: 09 March 2011 16:35
> >>>> To: sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org><mailto:sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>>
> >>>> Subject: Re: [Building Sakai] alternative to webdav in sakai
> >>>>
> >>>> https://jira.sakaiproject.org/browse/SAK-800
> >>>>
> >>>> SAK-800 and KNL-273 would allow users to upload a compressed
> archive and have it unrolled in Resources.
> >>>>
> >>>> --Sam
> >>>>
> >>>> On 3/9/2011 11:19 AM, John Bush wrote:
> >>>> Over the years we've had continued problems with webdav. The
> varying behaviors of different clients, the special authentication
> that doesn't play nice with SSO, and numerous other types of issues.
> I'm beginning to think about alternatives that would allow for big
> bulk uploads that are more reliable. Thinking about some type of
> browser plugin, flash or otherwise. Has anyone else done some
> research into this already?
> >>>>
> >>>> --
> >>>> John Bush
> >>>> 602-490-0470
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>>
> >>>> sakai-dev mailing list
> >>>>
> >>>> sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org><mailto:sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>>
> >>>>
> >>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>>
> >>>>
> >>>>
> >>>> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org><mailto:sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org>> with a
> subject of "unsubscribe"
> >>>>
> >>>> _______________________________________________
> >>>> sakai-dev mailing list
> >>>> sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org><mailto:sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>>
> >>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>>
> >>>> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org><mailto:sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org>> with a
> subject of "unsubscribe"
> >>>>
> >>> _______________________________________________
> >>> sakai-dev mailing list
> >>> sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>
> >>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>
> >>> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject
> of "unsubscribe"
> >>>
> >>>
> >> _______________________________________________
> >> sakai-dev mailing list
> >> sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>
> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>
> >> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject
> of "unsubscribe"
> > _______________________________________________
> > sakai-dev mailing list
> > sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>
> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >
> > TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject
> of "unsubscribe"
> >
> >
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> <mailto:sakai-dev at collab.sakaiproject.org>
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> <mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject
> of "unsubscribe"
>
>
> _______________________________________________
> mt mailing list
> mt at collab.sakaiproject.org <mailto:mt at collab.sakaiproject.org>
> http://collab.sakaiproject.org/mailman/listinfo/mt
>
>
>
>
> --
> John Bush
> 602-490-0470
> _______________________________________________
> mt mailing list
> mt at collab.sakaiproject.org <mailto:mt at collab.sakaiproject.org>
> http://collab.sakaiproject.org/mailman/listinfo/mt
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
--
==================================
Adrian Fish
Software Engineer
Centre for e-Science
Bowland Tower South C Floor
Lancaster University
Lancaster
LA1 4YW
email: a.fish at lancaster.ac.uk
http://confluence.sakaiproject.org/display/YAFT/Yaft
http://confluence.sakaiproject.org/display/CLOG/Home
http://confluence.sakaiproject.org/display/BBB/Home
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20110311/f1e4594d/attachment.html
More information about the sakai-dev
mailing list