[Building Sakai] Unable to upload large file/resource, but still smaller than content.upload.max?

[will at serensoft.com]

And the JDK is 1.5.0_18. :)


On Thu, Nov 19, 2009 at 4:15 PM, will at serensoft.com <will at serensoft.com> wrote:
> Yes, Tomcat can write smaller files (8mb, 400k, 12k etc) just fine, so
> it's not a permissions issue.
>
> It's windows server 2003 r2 enterprise x64 edition, sp2.
>
> No domain, it's just in a workgroup. We'll tinker with that next. Thanks, Dave!
>
>
> On Thu, Nov 19, 2009 at 8:33 AM, Dave Ross <dave.ross at gmail.com> wrote:
>> What version of windows and again, is this in a domain (and is the
>> tomcat service account a domain user with rights to
>> \\\\192.168.9.99\\SakaiResources )? The "Insufficient system resources
>> exist to complete the requested service" is a Windows specific error
>> with large files that we saw go away once we moved everything to a
>> domain.
>>
>> Also, what JDK version?
>>
>>
>>
>> On Wed, Nov 18, 2009 at 1:40 PM, will at serensoft.com <will at serensoft.com> wrote:
>>> Hi Steve --
>>>
>>> Filesystem for storage, not the database. Using Mysql--yikes!
>>>
>>> I suspect the JAVA_OPTS needs some TLC. Tomcat is running as a service
>>> in 64-bit windows:
>>> Initial memory pool: 512
>>> Maximum memory pool: 3600
>>> -Xms3600m
>>> -Xmx3600m
>>> -XX:MaxPermSize=512m
>>> -XX:+UseAdaptiveSizePolicy
>>> -XX:+UseConcMarkSweepGC
>>> -XX:+UseParNewGC
>>> -Dcatalina.base=C:\Tomcat55
>>> -Dcatalina.home=C:\Tomcat55
>>> -Djava.endorsed.dirs=C:\Tomcat55\common\endorsed
>>> -Djava.io.tmpdir=C:\Tomcat55\temp
>>> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
>>> -Djava.util.logging.config.file=C:\Tomcat55\conf\logging.properties
>>>
>>> Any obvious glitches there?
>>>
>>>
>>> # from sakai.properties
>>> content.upload.max=50
>>> bodyPath at org.sakaiproject.content.api.ContentHostingService=\\\\192.168.9.99\\SakaiResources
>>>
>>> We are using MySQL, but if we were putting the binary data in the
>>> database, it would certainly kill the project (we timed an 8mb upload
>>> at 1m43s in the DB, vs. 3s in the filesystem).
>>>
>>> In theory, here, the only entry in the database is a reference to
>>> where the real file is, in the file system. Right?
>>>
>>>
>>>
>>> On Tue, Nov 17, 2009 at 5:24 PM, Steve Swinsburg
>>> <steve.swinsburg at gmail.com> wrote:
>>>> Hi Will,
>>>> As Remi said, to make sure your DB isn't restricted, if using MySQL and
>>>> storing content in the DB, set the max_allowed_packet to something big. I
>>>> blogged about this
>>>> here: http://steve-on-sakai.blogspot.com/2008/11/sakai-mysql-and-storing-content-in.html
>>>> This tidbit should probably be in the install guide as its come up a number
>>>> of times before.
>>>> You'll also need to make sure you have enough resources available to the
>>>> JVM. What are your JAVA_OPTS? With a 1.5-2Gb heap (wasn't any more than
>>>> that), we've successfully uploaded files around 400Mb to an offsite Oracle
>>>> cluster.
>>>> As for adding content via an InputStream rather than byte[], looks
>>>> like http://jira.sakaiproject.org/browse/KNL-325 is the one and the fix
>>>> version is in Kernel trunk v1.1.0-beta04 (not 2.6.x):
>>>>
>>>> cheers,
>>>> Steve
>>>>
>>>>
>>>> On Wed, Nov 18, 2009 at 8:53 AM, Branden Visser <branden at uwindsor.ca> wrote:
>>>>>
>>>>> Charles,
>>>>>
>>>>> Content streaming is a work-in-progress. Some tickets that come to mind
>>>>> are:
>>>>>
>>>>> http://jira.sakaiproject.org/browse/SAK-6306
>>>>> http://jira.sakaiproject.org/browse/KNL-325
>>>>> http://jira.sakaiproject.org/browse/SAK-16806
>>>>>
>>>>> Thanks,
>>>>> Branden
>>>>>
>>>>> Charles Hedrick wrote:
>>>>> > Why is this writing from a byte array to a file? Why aren't we using the
>>>>> > facilities to patch the input stream directly into a file output stream?
>>>>> > We want to use Sakai to support multimedia work, so we want to allow
>>>>> > uploads of several hundred MB. We'd probably allow 1 GB if we could.
>>>>> > However the resulting huge byte arrays are unfriendly for Java's heap.
>>>>> >
>>>>> >
>>>>> > On Nov 17, 2009, at 3:22 PM, will at serensoft.com wrote:
>>>>> >
>>>>> >> org.sakaiproject.content.impl.DbContentService - failed to write file
>>>>> >> from byte-array to file
>>>>> >
>>>>> >
>>>>> > ------------------------------------------------------------------------
>>>>> >
>>>>> > _______________________________________________
>>>>> > sakai-dev mailing list
>>>>> > sakai-dev at collab.sakaiproject.org
>>>>> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>> >
>>>>> > TO UNSUBSCRIBE: send email to
>>>>> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> > "unsubscribe"
>>>>> _______________________________________________
>>>>> sakai-dev mailing list
>>>>> sakai-dev at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>>
>>>>> TO UNSUBSCRIBE: send email to
>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> "unsubscribe"
>>>>
>>>>
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>
>>>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>>>> with a subject of "unsubscribe"
>>>>
>>>
>>>
>>>
>>> --
>>> will trillich
>>> "Tis the set of the sails / And not the gales / That tells the way we
>>> go." -- Ella Wheeler Wilcox
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>
>>
>
>
>
> --
> will trillich
> "Tis the set of the sails / And not the gales / That tells the way we
> go." -- Ella Wheeler Wilcox
>



-- 
will trillich
"Tis the set of the sails / And not the gales / That tells the way we
go." -- Ella Wheeler Wilcox