[Building Sakai] Unable to upload large file/resource, but still smaller than content.upload.max?

[Dave Ross]

What version of windows and again, is this in a domain (and is the
tomcat service account a domain user with rights to
\\\\192.168.9.99\\SakaiResources )? The "Insufficient system resources
exist to complete the requested service" is a Windows specific error
with large files that we saw go away once we moved everything to a
domain.

Also, what JDK version?



On Wed, Nov 18, 2009 at 1:40 PM, will at serensoft.com <will at serensoft.com> wrote:
> Hi Steve --
>
> Filesystem for storage, not the database. Using Mysql--yikes!
>
> I suspect the JAVA_OPTS needs some TLC. Tomcat is running as a service
> in 64-bit windows:
> Initial memory pool: 512
> Maximum memory pool: 3600
> -Xms3600m
> -Xmx3600m
> -XX:MaxPermSize=512m
> -XX:+UseAdaptiveSizePolicy
> -XX:+UseConcMarkSweepGC
> -XX:+UseParNewGC
> -Dcatalina.base=C:\Tomcat55
> -Dcatalina.home=C:\Tomcat55
> -Djava.endorsed.dirs=C:\Tomcat55\common\endorsed
> -Djava.io.tmpdir=C:\Tomcat55\temp
> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
> -Djava.util.logging.config.file=C:\Tomcat55\conf\logging.properties
>
> Any obvious glitches there?
>
>
> # from sakai.properties
> content.upload.max=50
> bodyPath at org.sakaiproject.content.api.ContentHostingService=\\\\192.168.9.99\\SakaiResources
>
> We are using MySQL, but if we were putting the binary data in the
> database, it would certainly kill the project (we timed an 8mb upload
> at 1m43s in the DB, vs. 3s in the filesystem).
>
> In theory, here, the only entry in the database is a reference to
> where the real file is, in the file system. Right?
>
>
>
> On Tue, Nov 17, 2009 at 5:24 PM, Steve Swinsburg
> <steve.swinsburg at gmail.com> wrote:
>> Hi Will,
>> As Remi said, to make sure your DB isn't restricted, if using MySQL and
>> storing content in the DB, set the max_allowed_packet to something big. I
>> blogged about this
>> here: http://steve-on-sakai.blogspot.com/2008/11/sakai-mysql-and-storing-content-in.html
>> This tidbit should probably be in the install guide as its come up a number
>> of times before.
>> You'll also need to make sure you have enough resources available to the
>> JVM. What are your JAVA_OPTS? With a 1.5-2Gb heap (wasn't any more than
>> that), we've successfully uploaded files around 400Mb to an offsite Oracle
>> cluster.
>> As for adding content via an InputStream rather than byte[], looks
>> like http://jira.sakaiproject.org/browse/KNL-325 is the one and the fix
>> version is in Kernel trunk v1.1.0-beta04 (not 2.6.x):
>>
>> cheers,
>> Steve
>>
>>
>> On Wed, Nov 18, 2009 at 8:53 AM, Branden Visser <branden at uwindsor.ca> wrote:
>>>
>>> Charles,
>>>
>>> Content streaming is a work-in-progress. Some tickets that come to mind
>>> are:
>>>
>>> http://jira.sakaiproject.org/browse/SAK-6306
>>> http://jira.sakaiproject.org/browse/KNL-325
>>> http://jira.sakaiproject.org/browse/SAK-16806
>>>
>>> Thanks,
>>> Branden
>>>
>>> Charles Hedrick wrote:
>>> > Why is this writing from a byte array to a file? Why aren't we using the
>>> > facilities to patch the input stream directly into a file output stream?
>>> > We want to use Sakai to support multimedia work, so we want to allow
>>> > uploads of several hundred MB. We'd probably allow 1 GB if we could.
>>> > However the resulting huge byte arrays are unfriendly for Java's heap.
>>> >
>>> >
>>> > On Nov 17, 2009, at 3:22 PM, will at serensoft.com wrote:
>>> >
>>> >> org.sakaiproject.content.impl.DbContentService - failed to write file
>>> >> from byte-array to file
>>> >
>>> >
>>> > ------------------------------------------------------------------------
>>> >
>>> > _______________________________________________
>>> > sakai-dev mailing list
>>> > sakai-dev at collab.sakaiproject.org
>>> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>> >
>>> > TO UNSUBSCRIBE: send email to
>>> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> > "unsubscribe"
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to
>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> "unsubscribe"
>>
>>
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>>
>
>
>
> --
> will trillich
> "Tis the set of the sails / And not the gales / That tells the way we
> go." -- Ella Wheeler Wilcox
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>