[Building Sakai] [Deploying Sakai] ClientAbortException: java.net.SocketException: Broken pipe

[Noah Botimer]

It sounds like you were running at 1GB heap, which is quite likely too low for an instance with lots of modules/webapps deployed. Increasing Xmx should do the trick, but in case you still have problems...

You can flip the switch to dump heap on crash (and once it falls over), you could analyze the dump:

http://stackoverflow.com/questions/542979/using-heapdumponoutofmemoryerror-parameter-for-heap-dump-for-jboss

I've had good success with YourKit for examining heaps, but jhat and VisualVM can do it too. You would be looking for a particular class or set of classes that are disproportionately allocated (say 50% or more of total), indicating a leak (often a reference from an item to a container/observer that persists but no longer tracks the child).

If you observe that a JVM (via jstat or other) has a growing heap but is not yet to crashing, you can dump the heap with jmap.

I think it's pretty common these days to run with 4 and 6GB heaps on 64-bit JVMs (with Aaron's proviso of ~25% spare for system).

Thanks,
-Noah

On Nov 1, 2013, at 11:51 AM, Aaron Zeckoski wrote:

> I would say 2g is adequate but I am adding the production list to this
> to see if anyone else has any comment here. Generally a single
> instance would be more than adequate for 60 users. We certainly are
> running with many more users per instance in many cases without issue.
> 
> -AZ
> 
> 
> On Fri, Nov 1, 2013 at 11:36 AM, Warwick Chapman
> <warwickchapman at gmail.com> wrote:
>> Ok, please give me your best guess because I don't have any opportunity left
>> to tune.  On Sunday I've got to ensure this thing doesn't crash because of
>> java.lang.OutOfMemoryError: Java heap space as I've been getting now ...
>> 
>> I cannot believe I am working so hard to get Sakai stable for no more than
>> 60 concurrent users.
>> 
>> I have 3 OpenVZ on 3 separate pieces of tin.
>> 
>> Two have been increased to 8192 and the third to 6144.
>> 
>> Their JAVA_OPTS are now, respectively:
>> 
>> 1. export JAVA_OPTS='-server -d64 -Xmx4g -XX:MaxPermSize=512m
>> -Djava.awt.headless=true -Dhttp.agent=Sakai'
>> 2. export JAVA_OPTS='-server -d64 -Xmx4g -XX:MaxPermSize=512m
>> -Djava.awt.headless=true -Dhttp.agent=Sakai'
>> 3. export JAVA_OPTS='-server -d64 -Xmx3g -XX:MaxPermSize=512m
>> -Djava.awt.headless=true -Dhttp.agent=Sakai'
>> 
>> I am worried that keeping them at 4192 and doubling up from the original
>> 1024 will not be sufficient.  But I will be guided by you.
>> 
>> 
>> -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>> 
>> 
>> On Fri, Nov 1, 2013 at 5:28 PM, Aaron Zeckoski <azeckoski at unicon.net> wrote:
>>> 
>>> The "No" was for your "OK" question since you are missing MaxPermSize.
>>> You can set it to something higher but if it were me I would not do
>>> it. Larger memory space means it takes the JVM longer to do garbage
>>> collections. I would suggest you start with something smaller and tune
>>> it until things are working well for your users and load. Like I said,
>>> there is no single magic or "right" number. There are lots of wrong
>>> ones though.
>>> 
>>> -AZ
>>> 
>>> 
>>> On Fri, Nov 1, 2013 at 11:20 AM, Warwick Chapman
>>> <warwickchapman at gmail.com> wrote:
>>>> Even if I increase the VM memory and swap to 8G each?
>>>> 
>>>> -- Warwick Bruce Chapman | +27 83 7797 094 | http://warwickchapman.com
>>>> 
>>>> 
>>>> On Fri, Nov 1, 2013 at 5:17 PM, Aaron Zeckoski <azeckoski at unicon.net>
>>>> wrote:
>>>>> 
>>>>> No, you need MaxPermSize and you will definitely run into using swap
>>>>> since your OS uses up memory as well. You better go with 2g and 512M
>>>>> for Xmx and MaxPerm for now and you can tune from there.
>>>>> -AZ
>>>>> 
>>>>> 
>>>>> On Fri, Nov 1, 2013 at 11:12 AM, Warwick Chapman
>>>>> <warwickchapman at gmail.com> wrote:
>>>>>> Aaron, if I set the VMs to 8G MEM and SWAP, will the following be OK:
>>>>>> 
>>>>>> export JAVA_OPTS='-server -d64 -Xmx4g -Djava.awt.headless=true
>>>>>> -Dhttp.agent=Sakai'
>>>>>> 
>>>>>> -- Warwick Bruce Chapman | +27 83 7797 094 |
>>>>>> http://warwickchapman.com
>>>>>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20131101/96ce3ed3/attachment.html