[Building Sakai] WebDav (large) files OutOfMemoryError

Nicolas Lupien niclupien at gmail.com
Wed Mar 16 16:27:55 PDT 2011 

I increased the RAM on the vps to use your exacts JAVA_OPTS and it works
fine.

You are really helpful and thank you for digging into this with me.

Nicolas Lupien



On Wed, Mar 16, 2011 at 4:48 PM, Matthew Jones <jonespm at umich.edu> wrote:

> Yea, in SAK-16806 in 2.7 there were some fixes to the content tool (the web
> tool that you mention) which streamed file uploads rather than loading the
> entire contents into memory. It's possible that this same fix wasn't applied
> to webdav but I'm not sure as I'm not very familiar with that code. I'm sure
> someone else will reply and comment on this.
>
> I figured you might have been on a memory restricted instance. Even on the
> Amazon EC2 you have to at least be running the small (1.7G) as the micro
> with 613MB is just too small to run Sakai + a database instance without
> seeing random memory problems. I know if you removed a bunch of tools you
> could probably get it to run on the micro, but one of those you'd probably
> have to avoid is webdav.
>
> -Matthew
>
>
> On Wed, Mar 16, 2011 at 4:40 PM, Nicolas Lupien <niclupien at gmail.com>wrote:
>
>> First, thank you for your reply. This is my dev site and it is on a small
>> memory VPS, ill get more RAM and see if this fix the problem.
>>
>> For the info, the file is only 25M, it should not take so much heap space.
>> I'm suspecting this could be a bug.
>>
>> The file is correctly imported if i use the web tool. I can upload much
>> larger files with the web tool without problem. It's only the DAV which
>> seems bound to memory.
>>
>> Nicolas Lupien
>>
>>
>>
>>
>> On Wed, Mar 16, 2011 at 4:33 PM, Matthew Jones <jonespm at umich.edu> wrote:
>>
>>> I wonder though also if this was related to
>>> https://jira.sakaiproject.org/browse/SAK-10822 and more specifically
>>> https://jira.sakaiproject.org/browse/SAK-16806. see that SAK-10822
>>> specifically dealt with downloading but perhaps uploading still had to hold
>>> the buffer for writing and wasn't fixed?
>>>
>>> Increasing the heap would get you around the problem in the short term
>>> though.
>>>
>>>
>>> On Wed, Mar 16, 2011 at 4:28 PM, Matthew Jones <jonespm at umich.edu>wrote:
>>>
>>>> Hi Nicolas,
>>>>
>>>> You need a lot more heap than you have configured, probably at least
>>>> double. The install guide recommends 512/1024. Many installations go even
>>>> bigger than that for production. Sakai can use the extra memory for caching
>>>> and improved performance.
>>>>
>>>> export JAVA_OPTS="-server -Xms512m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=512m -XX:NewSize=192m -XX:MaxNewSize=384m -Djava.awt.headless=true -Dhttp.agent=Sakai -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false -Dsun.lang.ClassLoader.allowArraySyntax=true'
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> "
>>>>
>>>>
>>>> https://confluence.sakaiproject.org/display/DOC/Install+Guide+-+Binary+Install+%282.7%29
>>>>
>>>> On Wed, Mar 16, 2011 at 4:20 PM, Nicolas Lupien <niclupien at gmail.com>wrote:
>>>>
>>>>>  Hi,
>>>>>
>>>>> I am trying to upload small courses videos (~25M) via webdav and i get
>>>>> "Internal servlet exception" from my webdav client.
>>>>>
>>>>> On the server logs i get:
>>>>>
>>>>>  javax.servlet.ServletException: L'ex<C3><A9>cution de la servlet a
>>>>> lanc<C3><A9> une exception
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>>         at
>>>>> org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:646)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>>         at
>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>>>>         at
>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>>>>>         at
>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
>>>>>         at
>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>>>>         at
>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>>>>>         at
>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>>>>>         at
>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>>>>>         at
>>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:879)
>>>>>         at
>>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>>         at
>>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>>         at
>>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>>         at
>>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>>         at java.lang.Thread.run(Thread.java:662)
>>>>> 2011-03-16 20:08:13,397 ERROR http-80-Processor18
>>>>> org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/dav].[dav] -
>>>>> "Servlet.service()" pour la servlet dav a g<C3><A9>n<C3><A9>r<C3><A9> une
>>>>> except
>>>>> ion
>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>         at java.util.Arrays.copyOf(Arrays.java:2786)
>>>>>         at
>>>>> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>>>>>         at
>>>>> org.sakaiproject.content.impl.DbContentService$DbStorage.putResourceBodyDb(DbContentService.java:2191)
>>>>>         at
>>>>> org.sakaiproject.content.impl.DbContentService$DbStorage.commitResource(DbContentService.java:1745)
>>>>>         at
>>>>> org.sakaiproject.content.impl.BaseContentService.commitResourceEdit(BaseContentService.java:5639)
>>>>>         at
>>>>> org.sakaiproject.content.impl.BaseContentService.commitResource(BaseContentService.java:5548)
>>>>>         at org.sakaiproject.dav.DavServlet.doPut(DavServlet.java:2779)
>>>>>         at
>>>>> org.sakaiproject.dav.DavServlet.doDispatch(DavServlet.java:1139)
>>>>>         at
>>>>> org.sakaiproject.dav.DavServlet.service(DavServlet.java:1066)
>>>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>>         at
>>>>> org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:646)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>>>>         at
>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>>         at
>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>>>>         at
>>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>>>>>         at
>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
>>>>>         at
>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>>>>         at
>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>>>>>         at
>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>>>>>         at
>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>>>>>         at
>>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:879)
>>>>>         at
>>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>>         at
>>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>>         at
>>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>>         at
>>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>>         at java.lang.Thread.run(Thread.java:662)
>>>>>
>>>>> --------
>>>>>
>>>>> Information :
>>>>>
>>>>> OS : Ubuntu 10.10 Server
>>>>>
>>>>> JAVA : JDK 1.6.0_22
>>>>>
>>>>> SGBD: MySQL 5.1
>>>>>
>>>>> JAVA_OPTS : "-Xms256m -Xmx512m -XX:PermSize=128m -XX:MaxPermSize=256m
>>>>> -Djava.awt.headless=true -Dhttp.agent=Sakai
>>>>> -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false
>>>>> -Dsun.lang.ClassLoader.allowArraySyntax=true -Duser.language=fr
>>>>> -Duser.region=CA"
>>>>>
>>>>> my.conf :  max_allowed_packet = 100M
>>>>>
>>>>> sakai.properties : content.upload.max=100
>>>>>
>>>>> Nicolas Lupien
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> sakai-dev mailing list
>>>>> sakai-dev at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>>
>>>>> TO UNSUBSCRIBE: send email to
>>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>>> "unsubscribe"
>>>>>
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20110316/8b8fe5d1/attachment.html

More information about the sakai-dev mailing list