[Building Sakai] Unable to upload large file/resource, but still smaller than content.upload.max?

[will at serensoft.com]

Yes, Tomcat can write smaller files (8mb, 400k, 12k etc) just fine, so
it's not a permissions issue.

It's windows server 2003 r2 enterprise x64 edition, sp2.

No domain, it's just in a workgroup. We'll tinker with that next. Thanks, Dave!


On Thu, Nov 19, 2009 at 8:33 AM, Dave Ross <dave.ross at gmail.com> wrote:
> What version of windows and again, is this in a domain (and is the
> tomcat service account a domain user with rights to
> \\\\192.168.9.99\\SakaiResources )? The "Insufficient system resources
> exist to complete the requested service" is a Windows specific error
> with large files that we saw go away once we moved everything to a
> domain.
>
> Also, what JDK version?
>
>
>
> On Wed, Nov 18, 2009 at 1:40 PM, will at serensoft.com <will at serensoft.com> wrote:
>> Hi Steve --
>>
>> Filesystem for storage, not the database. Using Mysql--yikes!
>>
>> I suspect the JAVA_OPTS needs some TLC. Tomcat is running as a service
>> in 64-bit windows:
>> Initial memory pool: 512
>> Maximum memory pool: 3600
>> -Xms3600m
>> -Xmx3600m
>> -XX:MaxPermSize=512m
>> -XX:+UseAdaptiveSizePolicy
>> -XX:+UseConcMarkSweepGC
>> -XX:+UseParNewGC
>> -Dcatalina.base=C:\Tomcat55
>> -Dcatalina.home=C:\Tomcat55
>> -Djava.endorsed.dirs=C:\Tomcat55\common\endorsed
>> -Djava.io.tmpdir=C:\Tomcat55\temp
>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>> -Djava.util.logging.config.file=C:\Tomcat55\conf\logging.properties
>>
>> Any obvious glitches there?
>>
>>
>> # from sakai.properties
>> content.upload.max=50
>> bodyPath at org.sakaiproject.content.api.ContentHostingService=\\\\192.168.9.99\\SakaiResources
>>
>> We are using MySQL, but if we were putting the binary data in the
>> database, it would certainly kill the project (we timed an 8mb upload
>> at 1m43s in the DB, vs. 3s in the filesystem).
>>
>> In theory, here, the only entry in the database is a reference to
>> where the real file is, in the file system. Right?
>>
>>
>>
>> On Tue, Nov 17, 2009 at 5:24 PM, Steve Swinsburg
>> <steve.swinsburg at gmail.com> wrote:
>>> Hi Will,
>>> As Remi said, to make sure your DB isn't restricted, if using MySQL and
>>> storing content in the DB, set the max_allowed_packet to something big. I
>>> blogged about this
>>> here: http://steve-on-sakai.blogspot.com/2008/11/sakai-mysql-and-storing-content-in.html
>>> This tidbit should probably be in the install guide as its come up a number
>>> of times before.
>>> You'll also need to make sure you have enough resources available to the
>>> JVM. What are your JAVA_OPTS? With a 1.5-2Gb heap (wasn't any more than
>>> that), we've successfully uploaded files around 400Mb to an offsite Oracle
>>> cluster.
>>> As for adding content via an InputStream rather than byte[], looks
>>> like http://jira.sakaiproject.org/browse/KNL-325 is the one and the fix
>>> version is in Kernel trunk v1.1.0-beta04 (not 2.6.x):
>>>
>>> cheers,
>>> Steve
>>>
>>>
>>> On Wed, Nov 18, 2009 at 8:53 AM, Branden Visser <branden at uwindsor.ca> wrote:
>>>>
>>>> Charles,
>>>>
>>>> Content streaming is a work-in-progress. Some tickets that come to mind
>>>> are:
>>>>
>>>> http://jira.sakaiproject.org/browse/SAK-6306
>>>> http://jira.sakaiproject.org/browse/KNL-325
>>>> http://jira.sakaiproject.org/browse/SAK-16806
>>>>
>>>> Thanks,
>>>> Branden
>>>>
>>>> Charles Hedrick wrote:
>>>> > Why is this writing from a byte array to a file? Why aren't we using the
>>>> > facilities to patch the input stream directly into a file output stream?
>>>> > We want to use Sakai to support multimedia work, so we want to allow
>>>> > uploads of several hundred MB. We'd probably allow 1 GB if we could.
>>>> > However the resulting huge byte arrays are unfriendly for Java's heap.
>>>> >
>>>> >
>>>> > On Nov 17, 2009, at 3:22 PM, will at serensoft.com wrote:
>>>> >
>>>> >> org.sakaiproject.content.impl.DbContentService - failed to write file
>>>> >> from byte-array to file
>>>> >
>>>> >
>>>> > ------------------------------------------------------------------------
>>>> >
>>>> > _______________________________________________
>>>> > sakai-dev mailing list
>>>> > sakai-dev at collab.sakaiproject.org
>>>> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>> >
>>>> > TO UNSUBSCRIBE: send email to
>>>> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> > "unsubscribe"
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>
>>>
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>>> with a subject of "unsubscribe"
>>>
>>
>>
>>
>> --
>> will trillich
>> "Tis the set of the sails / And not the gales / That tells the way we
>> go." -- Ella Wheeler Wilcox
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>
>



-- 
will trillich
"Tis the set of the sails / And not the gales / That tells the way we
go." -- Ella Wheeler Wilcox