[Building Sakai] WebDav (large) files OutOfMemoryError

Matthew Jones jonespm at umich.edu
Wed Mar 16 13:48:46 PDT 2011 

Yea, in SAK-16806 in 2.7 there were some fixes to the content tool (the web
tool that you mention) which streamed file uploads rather than loading the
entire contents into memory. It's possible that this same fix wasn't applied
to webdav but I'm not sure as I'm not very familiar with that code. I'm sure
someone else will reply and comment on this.

I figured you might have been on a memory restricted instance. Even on the
Amazon EC2 you have to at least be running the small (1.7G) as the micro
with 613MB is just too small to run Sakai + a database instance without
seeing random memory problems. I know if you removed a bunch of tools you
could probably get it to run on the micro, but one of those you'd probably
have to avoid is webdav.

-Matthew

On Wed, Mar 16, 2011 at 4:40 PM, Nicolas Lupien <niclupien at gmail.com> wrote:

> First, thank you for your reply. This is my dev site and it is on a small
> memory VPS, ill get more RAM and see if this fix the problem.
>
> For the info, the file is only 25M, it should not take so much heap space.
> I'm suspecting this could be a bug.
>
> The file is correctly imported if i use the web tool. I can upload much
> larger files with the web tool without problem. It's only the DAV which
> seems bound to memory.
>
> Nicolas Lupien
>
>
>
>
> On Wed, Mar 16, 2011 at 4:33 PM, Matthew Jones <jonespm at umich.edu> wrote:
>
>> I wonder though also if this was related to
>> https://jira.sakaiproject.org/browse/SAK-10822 and more specifically
>> https://jira.sakaiproject.org/browse/SAK-16806. see that SAK-10822
>> specifically dealt with downloading but perhaps uploading still had to hold
>> the buffer for writing and wasn't fixed?
>>
>> Increasing the heap would get you around the problem in the short term
>> though.
>>
>>
>> On Wed, Mar 16, 2011 at 4:28 PM, Matthew Jones <jonespm at umich.edu> wrote:
>>
>>> Hi Nicolas,
>>>
>>> You need a lot more heap than you have configured, probably at least
>>> double. The install guide recommends 512/1024. Many installations go even
>>> bigger than that for production. Sakai can use the extra memory for caching
>>> and improved performance.
>>>
>>> export JAVA_OPTS="-server -Xms512m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=512m -XX:NewSize=192m -XX:MaxNewSize=384m -Djava.awt.headless=true -Dhttp.agent=Sakai -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false -Dsun.lang.ClassLoader.allowArraySyntax=true'
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> "
>>>
>>>
>>> https://confluence.sakaiproject.org/display/DOC/Install+Guide+-+Binary+Install+%282.7%29
>>>
>>> On Wed, Mar 16, 2011 at 4:20 PM, Nicolas Lupien <niclupien at gmail.com>wrote:
>>>
>>>>  Hi,
>>>>
>>>> I am trying to upload small courses videos (~25M) via webdav and i get
>>>> "Internal servlet exception" from my webdav client.
>>>>
>>>> On the server logs i get:
>>>>
>>>>  javax.servlet.ServletException: L'ex<C3><A9>cution de la servlet a
>>>> lanc<C3><A9> une exception
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>         at
>>>> org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:646)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>         at
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>>>         at
>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>>>>         at
>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
>>>>         at
>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>>>         at
>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>>>>         at
>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>>>>         at
>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>>>>         at
>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:879)
>>>>         at
>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>         at
>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>         at
>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>         at
>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>> 2011-03-16 20:08:13,397 ERROR http-80-Processor18
>>>> org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/dav].[dav] -
>>>> "Servlet.service()" pour la servlet dav a g<C3><A9>n<C3><A9>r<C3><A9> une
>>>> except
>>>> ion
>>>> java.lang.OutOfMemoryError: Java heap space
>>>>         at java.util.Arrays.copyOf(Arrays.java:2786)
>>>>         at
>>>> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
>>>>         at
>>>> org.sakaiproject.content.impl.DbContentService$DbStorage.putResourceBodyDb(DbContentService.java:2191)
>>>>         at
>>>> org.sakaiproject.content.impl.DbContentService$DbStorage.commitResource(DbContentService.java:1745)
>>>>         at
>>>> org.sakaiproject.content.impl.BaseContentService.commitResourceEdit(BaseContentService.java:5639)
>>>>         at
>>>> org.sakaiproject.content.impl.BaseContentService.commitResource(BaseContentService.java:5548)
>>>>         at org.sakaiproject.dav.DavServlet.doPut(DavServlet.java:2779)
>>>>         at
>>>> org.sakaiproject.dav.DavServlet.doDispatch(DavServlet.java:1139)
>>>>         at org.sakaiproject.dav.DavServlet.service(DavServlet.java:1066)
>>>>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>         at
>>>> org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:646)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>>>         at
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>>>         at
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>>>         at
>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
>>>>         at
>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
>>>>         at
>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>>>         at
>>>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>>>>         at
>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>>>>         at
>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>>>>         at
>>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:879)
>>>>         at
>>>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
>>>>         at
>>>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
>>>>         at
>>>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
>>>>         at
>>>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>>
>>>> --------
>>>>
>>>> Information :
>>>>
>>>> OS : Ubuntu 10.10 Server
>>>>
>>>> JAVA : JDK 1.6.0_22
>>>>
>>>> SGBD: MySQL 5.1
>>>>
>>>> JAVA_OPTS : "-Xms256m -Xmx512m -XX:PermSize=128m -XX:MaxPermSize=256m
>>>> -Djava.awt.headless=true -Dhttp.agent=Sakai
>>>> -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false
>>>> -Dsun.lang.ClassLoader.allowArraySyntax=true -Duser.language=fr
>>>> -Duser.region=CA"
>>>>
>>>> my.conf :  max_allowed_packet = 100M
>>>>
>>>> sakai.properties : content.upload.max=100
>>>>
>>>> Nicolas Lupien
>>>>
>>>>
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20110316/9a06c425/attachment.html

More information about the sakai-dev mailing list