[Building Sakai] Unable to upload large file/resource, but still smaller than content.upload.max?

[will at serensoft.com]

We're running into an interesting problem with uploading via the
Resource (web interface) tool on Sakai 2.6.x. We've got
content.upload.max=50, but we can't upload a file that's 39.7 MB
(41,717,760 bytes). Smaller files (20mb-30mb range) are accepted;
larger files (36mb+) are not.

It takes quite a bit longer for a 40mb file to upload and then be
rejected, than for a 8mb file to upload and be accepted. As if it's
uploading the whole file to a temporary location before trying to save
it permanently.

Here's an example, trying to upload a 41,717,760-byte file (39.7 MB):

"Unable to add item stellarium-0.10.0beta.exe" appears in the
interface, and the logs contain:

2009-11-17 14:30:16,223  WARN http-80-Processor58
org.sakaiproject.content.impl.DbContentService - : failed to write
resource: /group/ff1b69b8-4a18-4b07-8a50-d8df77c55a24/stellarium-0.10.0beta.exe
: java.io.IOException: Insufficient system resources exist to complete
the requested service
2009-11-17 14:30:16,239  WARN http-80-Processor58
org.sakaiproject.content.impl.DbContentService - failed to write file
from byte-array to file
org.sakaiproject.exception.ServerOverloadException id=failed to write
file from byte-array to file
	at org.sakaiproject.content.impl.DbContentService$DbStorage.commitResource(DbContentService.java:1726)
	at org.sakaiproject.content.impl.BaseContentService.commitResourceEdit(BaseContentService.java:5570)
	at org.sakaiproject.content.impl.BaseContentService.commitResource(BaseContentService.java:5533)
	at org.sakaiproject.content.cover.ContentHostingService.commitResource(ContentHostingService.java:522)
	at org.sakaiproject.content.tool.ResourcesAction.createResources(ResourcesAction.java:1384)
	at org.sakaiproject.content.tool.ResourcesAction.finishAction(ResourcesAction.java:7247)
	at org.sakaiproject.content.tool.ResourcesAction.buildMainPanelContext(ResourcesAction.java:4754)
	at sun.reflect.GeneratedMethodAccessor881.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at org.sakaiproject.cheftool.VelocityPortletPaneledAction.toolModeDispatch(VelocityPortletPaneledAction.java:401)
	at org.sakaiproject.cheftool.ToolServlet.doGet(ToolServlet.java:227)
	at org.sakaiproject.cheftool.VelocityPortletPaneledAction.doGet(VelocityPortletPaneledAction.java:1014)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
	at org.sakaiproject.vm.ComponentServlet.service(ComponentServlet.java:56)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:565)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:679)
	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:364)
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
	at org.sakaiproject.tool.impl.ActiveToolComponent$MyActiveTool.forward(ActiveToolComponent.java:475)
	at org.sakaiproject.portal.charon.SkinnableCharonPortal.forwardTool(SkinnableCharonPortal.java:1216)
	at org.sakaiproject.portal.charon.handlers.ToolHandler.doTool(ToolHandler.java:188)
	at org.sakaiproject.portal.charon.handlers.ToolHandler.doGet(ToolHandler.java:93)
	at org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:768)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:616)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875)
	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

It's not that disk space is in question -- there are hundreds of gigs
of free space on the bodyPath filesystem (note that we're using a
bodyPath to get files from/to a different machine as a file server in
case that's relevant) and over a hundred gb free on the Sakai/Tomcat
filesystem.

# from sakai.properties
content.upload.max=50
bodyPath at org.sakaiproject.content.api.ContentHostingService=\\\\192.168.9.99\\SakaiResources

Where should we look for the gremlin, here?

-- 
will trillich
"Tis the set of the sails / And not the gales / That tells the way we
go." -- Ella Wheeler Wilcox