[Deploying Sakai] [Building Sakai] ClientAbortException: java.net.SocketException: Broken pipe
Aaron Zeckoski
azeckoski at unicon.net
Fri Nov 1 08:51:04 PDT 2013
I would say 2g is adequate but I am adding the production list to this
to see if anyone else has any comment here. Generally a single
instance would be more than adequate for 60 users. We certainly are
running with many more users per instance in many cases without issue.
-AZ
On Fri, Nov 1, 2013 at 11:36 AM, Warwick Chapman
<warwickchapman at gmail.com> wrote:
> Ok, please give me your best guess because I don't have any opportunity left
> to tune. On Sunday I've got to ensure this thing doesn't crash because of
> java.lang.OutOfMemoryError: Java heap space as I've been getting now ...
>
> I cannot believe I am working so hard to get Sakai stable for no more than
> 60 concurrent users.
>
> I have 3 OpenVZ on 3 separate pieces of tin.
>
> Two have been increased to 8192 and the third to 6144.
>
> Their JAVA_OPTS are now, respectively:
>
> 1. export JAVA_OPTS='-server -d64 -Xmx4g -XX:MaxPermSize=512m
> -Djava.awt.headless=true -Dhttp.agent=Sakai'
> 2. export JAVA_OPTS='-server -d64 -Xmx4g -XX:MaxPermSize=512m
> -Djava.awt.headless=true -Dhttp.agent=Sakai'
> 3. export JAVA_OPTS='-server -d64 -Xmx3g -XX:MaxPermSize=512m
> -Djava.awt.headless=true -Dhttp.agent=Sakai'
>
> I am worried that keeping them at 4192 and doubling up from the original
> 1024 will not be sufficient. But I will be guided by you.
>
>
> -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>
>
> On Fri, Nov 1, 2013 at 5:28 PM, Aaron Zeckoski <azeckoski at unicon.net> wrote:
>>
>> The "No" was for your "OK" question since you are missing MaxPermSize.
>> You can set it to something higher but if it were me I would not do
>> it. Larger memory space means it takes the JVM longer to do garbage
>> collections. I would suggest you start with something smaller and tune
>> it until things are working well for your users and load. Like I said,
>> there is no single magic or "right" number. There are lots of wrong
>> ones though.
>>
>> -AZ
>>
>>
>> On Fri, Nov 1, 2013 at 11:20 AM, Warwick Chapman
>> <warwickchapman at gmail.com> wrote:
>> > Even if I increase the VM memory and swap to 8G each?
>> >
>> > -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>> >
>> >
>> > On Fri, Nov 1, 2013 at 5:17 PM, Aaron Zeckoski <azeckoski at unicon.net>
>> > wrote:
>> >>
>> >> No, you need MaxPermSize and you will definitely run into using swap
>> >> since your OS uses up memory as well. You better go with 2g and 512M
>> >> for Xmx and MaxPerm for now and you can tune from there.
>> >> -AZ
>> >>
>> >>
>> >> On Fri, Nov 1, 2013 at 11:12 AM, Warwick Chapman
>> >> <warwickchapman at gmail.com> wrote:
>> >> > Aaron, if I set the VMs to 8G MEM and SWAP, will the following be OK:
>> >> >
>> >> > export JAVA_OPTS='-server -d64 -Xmx4g -Djava.awt.headless=true
>> >> > -Dhttp.agent=Sakai'
>> >> >
>> >> > -- Warwick Bruce Chapman | +27 83 7797 094 |
>> >> > http://warwickchapman.com
>> >> >
>> >> >
>> >> > On Fri, Nov 1, 2013 at 5:09 PM, Aaron Zeckoski <azeckoski at unicon.net>
>> >> > wrote:
>> >> >>
>> >> >> There is no magic number but you would definitely not want to exceed
>> >> >> 3G for Xmx because if you allow the JVM to start using swap then it
>> >> >> will die very quickly. Garbage collecting swap space causes the JVM
>> >> >> to
>> >> >> basically become unresponsive. 2G is probably safe assuming you
>> >> >> don't
>> >> >> have a ton of other things running on there. The key is to not set
>> >> >> Xmx
>> >> >> any higher than the amount of free ram you have (minus the MaxPerm).
>> >> >>
>> >> >> -AZ
>> >> >>
>> >> >> On Fri, Nov 1, 2013 at 11:03 AM, Warwick Chapman
>> >> >> <warwickchapman at gmail.com> wrote:
>> >> >> > Aaron, if my VM had 4096M memory and 4096 swap, what should Xmx
>> >> >> > be?
>> >> >> >
>> >> >> > -- Warwick Bruce Chapman | +27 83 7797 094 |
>> >> >> > http://warwickchapman.com
>> >> >> >
>> >> >> >
>> >> >> > On Fri, Nov 1, 2013 at 1:30 PM, Aaron Zeckoski
>> >> >> > <azeckoski at unicon.net>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Looks like MaxPermSize is still in there and you will need to
>> >> >> >> define
>> >> >> >> Xmx or the JVM will use the default for that which is much too
>> >> >> >> small.
>> >> >> >>
>> >> >> >> Also, be aware that the GC printing will slow things down a bit
>> >> >> >> so
>> >> >> >> you
>> >> >> >> don't want to run live with that on. If you need GC stats then
>> >> >> >> use a
>> >> >> >> profiler or jstat to grab a quick snapshot of the memory
>> >> >> >> situation
>> >> >> >> every once in awhile.
>> >> >> >>
>> >> >> >> -AZ
>> >> >> >>
>> >> >> >>
>> >> >> >> On Fri, Nov 1, 2013 at 7:21 AM, Warwick Chapman
>> >> >> >> <warwickchapman at gmail.com> wrote:
>> >> >> >> > Ok, so nothing serious then.
>> >> >> >> >
>> >> >> >> > On another, note, as per my previous thread this morning, you
>> >> >> >> > previously
>> >> >> >> > suggested a less specific set of JAVA_OPTS in order to allow
>> >> >> >> > the
>> >> >> >> > JVM
>> >> >> >> > to
>> >> >> >> > autotune (ergonomics). I have taken all but
>> >> >> >> > -XX:MaxPermSize=512m
>> >> >> >> > out
>> >> >> >> > and am
>> >> >> >> > giving it a bash. Is this ok?
>> >> >> >> >
>> >> >> >> > JAVA_OPTS='-server -d64 -XX:MaxPermSize=512m
>> >> >> >> > -Djava.awt.headless=true
>> >> >> >> > -Dhttp.agent=Sakai -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
>> >> >> >> > -verbose:gc'
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > -- Warwick Bruce Chapman | +27 83 7797 094 |
>> >> >> >> > http://warwickchapman.com
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Fri, Nov 1, 2013 at 1:06 PM, Aaron Zeckoski
>> >> >> >> > <azeckoski at unicon.net>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Warwick,
>> >> >> >> >> This is typically what that error indicates:
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> http://stackoverflow.com/questions/839314/clientabortexception-java-net-socketexception-connection-reset-by-peer-socket
>> >> >> >> >>
>> >> >> >> >> -AZ
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Fri, Nov 1, 2013 at 6:55 AM, Warwick Chapman
>> >> >> >> >> <warwickchapman at gmail.com> wrote:
>> >> >> >> >> > Hi All
>> >> >> >> >> >
>> >> >> >> >> > Any idea what the following error is all about?
>> >> >> >> >> >
>> >> >> >> >> > 2013-11-01 12:51:49,585 WARN http-bio-8080-exec-17
>> >> >> >> >> > org.sakaiproject.util.RequestFilter -
>> >> >> >> >> > ClientAbortException: java.net.SocketException: Broken pipe
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:413)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:480)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> > org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:366)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:438)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:426)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:91)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.servlets.DefaultServlet.copy(DefaultServlet.java:1861)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.servlets.DefaultServlet.serveResource(DefaultServlet.java:961)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.servlets.DefaultServlet.doGet(DefaultServlet.java:411)
>> >> >> >> >> > at
>> >> >> >> >> > javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
>> >> >> >> >> > at
>> >> >> >> >> > javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:695)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> >> >> >> >> > at java.lang.Thread.run(Thread.java:662)
>> >> >> >> >> > Caused by: java.net.SocketException: Broken pipe
>> >> >> >> >> > at java.net.SocketOutputStream.socketWrite0(Native
>> >> >> >> >> > Method)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> > java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:215)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> > org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:371)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:240)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:84)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:192)
>> >> >> >> >> > at
>> >> >> >> >> > org.apache.coyote.Response.doWrite(Response.java:517)
>> >> >> >> >> > at
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:408)
>> >> >> >> >> > ... 29 more
>> >> >> >> >> >
>> >> >> >> >> > -- Warwick Bruce Chapman | +27 83 7797 094 |
>> >> >> >> >> > http://warwickchapman.com
>> >> >> >> >> >
>> >> >> >> >> > _______________________________________________
>> >> >> >> >> > sakai-dev mailing list
>> >> >> >> >> > sakai-dev at collab.sakaiproject.org
>> >> >> >> >> > http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> >> >> >> >> >
>> >> >> >> >> > TO UNSUBSCRIBE: send email to
>> >> >> >> >> > sakai-dev-unsubscribe at collab.sakaiproject.org
>> >> >> >> >> > with a subject of "unsubscribe"
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Aaron Zeckoski - Software Architect -
>> >> >> >> >> http://tinyurl.com/azprofile
>> >> >> >> >
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Aaron Zeckoski - Software Architect -
>> >> >> >> http://tinyurl.com/azprofile
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>> >
>> >
>>
>>
>>
>> --
>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>
>
--
Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
More information about the production
mailing list