[Building Sakai] Unable to upload large file/resource, but still smaller than content.upload.max?

[will at serensoft.com]

Hi Steve --

Filesystem for storage, not the database. Using Mysql--yikes!

I suspect the JAVA_OPTS needs some TLC. Tomcat is running as a service
in 64-bit windows:
Initial memory pool: 512
Maximum memory pool: 3600
-Xms3600m
-Xmx3600m
-XX:MaxPermSize=512m
-XX:+UseAdaptiveSizePolicy
-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC
-Dcatalina.base=C:\Tomcat55
-Dcatalina.home=C:\Tomcat55
-Djava.endorsed.dirs=C:\Tomcat55\common\endorsed
-Djava.io.tmpdir=C:\Tomcat55\temp
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Djava.util.logging.config.file=C:\Tomcat55\conf\logging.properties

Any obvious glitches there?


# from sakai.properties
content.upload.max=50
bodyPath at org.sakaiproject.content.api.ContentHostingService=\\\\192.168.9.99\\SakaiResources

We are using MySQL, but if we were putting the binary data in the
database, it would certainly kill the project (we timed an 8mb upload
at 1m43s in the DB, vs. 3s in the filesystem).

In theory, here, the only entry in the database is a reference to
where the real file is, in the file system. Right?



On Tue, Nov 17, 2009 at 5:24 PM, Steve Swinsburg
<steve.swinsburg at gmail.com> wrote:
> Hi Will,
> As Remi said, to make sure your DB isn't restricted, if using MySQL and
> storing content in the DB, set the max_allowed_packet to something big. I
> blogged about this
> here: http://steve-on-sakai.blogspot.com/2008/11/sakai-mysql-and-storing-content-in.html
> This tidbit should probably be in the install guide as its come up a number
> of times before.
> You'll also need to make sure you have enough resources available to the
> JVM. What are your JAVA_OPTS? With a 1.5-2Gb heap (wasn't any more than
> that), we've successfully uploaded files around 400Mb to an offsite Oracle
> cluster.
> As for adding content via an InputStream rather than byte[], looks
> like http://jira.sakaiproject.org/browse/KNL-325 is the one and the fix
> version is in Kernel trunk v1.1.0-beta04 (not 2.6.x):
>
> cheers,
> Steve
>
>
> On Wed, Nov 18, 2009 at 8:53 AM, Branden Visser <branden at uwindsor.ca> wrote:
>>
>> Charles,
>>
>> Content streaming is a work-in-progress. Some tickets that come to mind
>> are:
>>
>> http://jira.sakaiproject.org/browse/SAK-6306
>> http://jira.sakaiproject.org/browse/KNL-325
>> http://jira.sakaiproject.org/browse/SAK-16806
>>
>> Thanks,
>> Branden
>>
>> Charles Hedrick wrote:
>> > Why is this writing from a byte array to a file? Why aren't we using the
>> > facilities to patch the input stream directly into a file output stream?
>> > We want to use Sakai to support multimedia work, so we want to allow
>> > uploads of several hundred MB. We'd probably allow 1 GB if we could.
>> > However the resulting huge byte arrays are unfriendly for Java's heap.
>> >
>> >
>> > On Nov 17, 2009, at 3:22 PM, will at serensoft.com wrote:
>> >
>> >> org.sakaiproject.content.impl.DbContentService - failed to write file
>> >> from byte-array to file
>> >
>> >
>> > ------------------------------------------------------------------------
>> >
>> > _______________________________________________
>> > sakai-dev mailing list
>> > sakai-dev at collab.sakaiproject.org
>> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> >
>> > TO UNSUBSCRIBE: send email to
>> > sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>> > "unsubscribe"
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to
>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>> "unsubscribe"
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
> with a subject of "unsubscribe"
>



-- 
will trillich
"Tis the set of the sails / And not the gales / That tells the way we
go." -- Ella Wheeler Wilcox