[Building Sakai] Sakai Instances 100% CPU Usage

[Beth Kirschner]

There was a leap second this weekend (June 30, 8pm EDT) which we believe caused all our servers to hit 100% CPU usage simultaneously. Has anyone else in the Sakai community seen this? We're running:
	Linux 2.6.32-220.17.1.el6.x86_64 
	java version "1.6.0_22" 

Thanks,
- Beth

Begin forwarded message:
> 
> 
> 
> URL = http://wiki.os-ux.com/java-leap-second-bug-30-june-1-july-2012-fix/
> 
> 
> 
> java leap second bug – 30 June / 1 July 2012 – fix
> If your java applications suddenly started to use 100% CPU, you’ve hit java leap second bug (actually it’s a bug in Linux kernel, just java programs seem to be severely affected).
> 
> First, you may check if you have the following in your dmesg:
> 
> 
> [10703552.860274] Clock: inserting leap second 23:59:60 UTC
> 
> Fortunately the fix is straightforward:
> 
> /etc/init.d/ntpd stop
> date -s "$(date)"
> 
> /etc/init.d/ntpd start
> 
> (the other suggested way around the web would be date `date +"%m%d%H%M%C%y.%S"`, having the same effect).
> 
> You don’t have to restart your java applications (tomcat, solr, wowza, or whatever using java); it should be enough to run the above commands.
> 
> You may want to enable ntp the next day (there were some reports that enabling it makes java misbehave on some installations).
> 
> Apparently this issue has knocked down lots of Linux servers running Debian, Ubuntu, CentOS and the like around the globe!
> 
>  read more
> 
> (via wpkg.org)
> 
> Author: Vucomir Ianculov on July 3, 2012 
> Category: Linux / Unix 
> Tags: date, java, kernel, linux, ntpd, Zimbra
>  
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20120703/59c83cbc/attachment.html