[Building Sakai] problem uploading large files to Sakai: windows bodyPath=\\\\some\\other\\machine

Thu Mar 11 07:33:26 PST 2010

Anybody else using bodyPath=\\\\some\\other\\machine on windows and having
problems with large files via WebDAV?

Here's a bit more data on the issue -- we've tested WebDAV on two Sakai
instances:

We have two instances of Sakai -- on one, we can upload large files via
WebDAV, and on the other we get the error message.

System A (works fine): one machine with Tomcat, MySQL, and bodyPath
(resources stored in filesystem) all on the same machine. Here we can upload
large files via WebDAV, no problem. (This system was actually a copy of
System B below, with modifications to sakai.properties.)

System B (works up to a point then bombs with the error): two machines --
Tomcat on one machine, and MySQL with bodyPath on a second machine. The one
difference between the two sakai.properties files is the MySQL database
connection and the bodyPath setting... both are on a second machine, here.

JDK is 1.5.0_18, on windows server 2003 r2 enterprise x64 edition, sp2. No
difference between the two setups as far as domain goes.

Ideas? Anybody else encounter this?

On Thu, Nov 19, 2009 at 8:33 AM, Dave Ross <dave.ross at gmail.com> wrote:

> What version of windows and again, is this in a domain (and is the
> tomcat service account a domain user with rights to
> \\\\192.168.9.99\\SakaiResources )? The "Insufficient system resources
> exist to complete the requested service" is a Windows specific error
> with large files that we saw go away once we moved everything to a
> domain.
>
> Also, what JDK version?
>
>
>
> On Wed, Nov 18, 2009 at 1:40 PM, will at serensoft.com <will at serensoft.com>
> wrote:
> > Hi Steve --
> >
> > Filesystem for storage, not the database. Using Mysql--yikes!
> >
> > I suspect the JAVA_OPTS needs some TLC. Tomcat is running as a service
> > in 64-bit windows:
> > Initial memory pool: 512
> > Maximum memory pool: 3600
> > -Xms3600m
> > -Xmx3600m
> > -XX:MaxPermSize=512m
> > -XX:+UseAdaptiveSizePolicy
> > -XX:+UseConcMarkSweepGC
> > -XX:+UseParNewGC
> > -Dcatalina.base=C:\Tomcat55
> > -Dcatalina.home=C:\Tomcat55
> > -Djava.endorsed.dirs=C:\Tomcat55\common\endorsed
> > -Djava.io.tmpdir=C:\Tomcat55\temp
> > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> > -Djava.util.logging.config.file=C:\Tomcat55\conf\logging.properties
> >
> > Any obvious glitches there?
> >
> >
> > # from sakai.properties
> > content.upload.max=50
> > bodyPath at org.sakaiproject.content.api.ContentHostingService
> =\\\\192.168.9.99\\SakaiResources
> >
> > We are using MySQL, but if we were putting the binary data in the
> > database, it would certainly kill the project (we timed an 8mb upload
> > at 1m43s in the DB, vs. 3s in the filesystem).
> >
> > In theory, here, the only entry in the database is a reference to
> > where the real file is, in the file system. Right?
> >
> >
> >
> > On Tue, Nov 17, 2009 at 5:24 PM, Steve Swinsburg
> > <steve.swinsburg at gmail.com> wrote:
> >> Hi Will,
> >> As Remi said, to make sure your DB isn't restricted, if using MySQL and
> >> storing content in the DB, set the max_allowed_packet to something big.
> I
> >> blogged about this
> >> here:
> http://steve-on-sakai.blogspot.com/2008/11/sakai-mysql-and-storing-content-in.html
> >> This tidbit should probably be in the install guide as its come up a
> number
> >> of times before.
> >> You'll also need to make sure you have enough resources available to the
> >> JVM. What are your JAVA_OPTS? With a 1.5-2Gb heap (wasn't any more than
> >> that), we've successfully uploaded files around 400Mb to an offsite
> Oracle
> >> cluster.
> >> As for adding content via an InputStream rather than byte[], looks
> >> like http://jira.sakaiproject.org/browse/KNL-325 is the one and the fix
> >> version is in Kernel trunk v1.1.0-beta04 (not 2.6.x):
> >>
> >> cheers,
> >> Steve
> >>
> >>
> >> On Wed, Nov 18, 2009 at 8:53 AM, Branden Visser <branden at uwindsor.ca>
> wrote:
> >>>
> >>> Charles,
> >>>
> >>> Content streaming is a work-in-progress. Some tickets that come to mind
> >>> are:
> >>>
> >>> http://jira.sakaiproject.org/browse/SAK-6306
> >>> http://jira.sakaiproject.org/browse/KNL-325
> >>> http://jira.sakaiproject.org/browse/SAK-16806
> >>>
> >>> Thanks,
> >>> Branden
> >>>
> >>> Charles Hedrick wrote:
> >>> > Why is this writing from a byte array to a file? Why aren't we using
> the
> >>> > facilities to patch the input stream directly into a file output
> stream?
> >>> > We want to use Sakai to support multimedia work, so we want to allow
> >>> > uploads of several hundred MB. We'd probably allow 1 GB if we could.
> >>> > However the resulting huge byte arrays are unfriendly for Java's
> heap.
> >>> >
> >>> >
> >>> > On Nov 17, 2009, at 3:22 PM, will at serensoft.com wrote:
> >>> >
> >>> >> org.sakaiproject.content.impl.DbContentService - failed to write
> file
> >>> >> from byte-array to file
> >>> >
> >>> >
> >>> >
> ------------------------------------------------------------------------
> >>> >
> >>> > _______________________________________________
> >>> > sakai-dev mailing list
> >>> > sakai-dev at collab.sakaiproject.org
> >>> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>> >
> >>> > TO UNSUBSCRIBE: send email to
> >>> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> >>> > "unsubscribe"
> >>> _______________________________________________
> >>> sakai-dev mailing list
> >>> sakai-dev at collab.sakaiproject.org
> >>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>
> >>> TO UNSUBSCRIBE: send email to
> >>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> >>> "unsubscribe"
> >>
> >>
> >> _______________________________________________
> >> sakai-dev mailing list
> >> sakai-dev at collab.sakaiproject.org
> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>
> >> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> >> with a subject of "unsubscribe"
> >>
> >
> >
> >
> > --
> > will trillich
> > "Tis the set of the sails / And not the gales / That tells the way we
> > go." -- Ella Wheeler Wilcox
> > _______________________________________________
> > sakai-dev mailing list
> > sakai-dev at collab.sakaiproject.org
> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >
> > TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
> >
>

-- 
will trillich
"A knowledge of the path cannot be substituted for putting one foot in front
of the other." -- M.C.Richards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20100311/fc96b92a/attachment.html 