[Building Sakai] admin page loading causes OutOfMemoryError: Java heap space

[Sanghyun Jeon]

Hello,

One of our sakai severs was crashed after complaining critical CPU loading
issue. At the same time I got the bug report below, which was the same as
tomcat log said. Before this server's cpu warning, I got the mysql db high
CPU loading complaint and there were unusual CPU spike. Mysql workbench
showed that several statistics  state and multiple queries associated with
sakai_realm* tables were read/written to the db. as admin, I was about to
check the online tool.
I don't know the root cause of this crash and there were very few users at
the system at that time.
Can anybody shed some light on this issue?

bug-id: 2217f548-d6b2-41c8-8caf-1140e2578934

user: admin (Sakai Administrator)

email:

usage-session: a886b570-7fd8-4a67-9cdf-8d989065b883

stack-trace-digest: ADE287B32BC95FC455386D8ACEEC5458E65126F8

sakai-version: 2.8.x

service-version: v2.8.x

app-server: Sakai2B

user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/34.0.1847.116 Safari/537.36

browser-id: Win-Mozilla

IP: 134.173.69.30

request-path: /portal

time: Apr 11, 2014 09:07:41





stack trace:



java.lang.OutOfMemoryError: Java heap space

    at java.nio.HeapCharBuffer.<init>(HeapCharBuffer.java:39)

    at java.nio.CharBuffer.allocate(CharBuffer.java:312)

    at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:760)

    at java.nio.charset.Charset.decode(Charset.java:771)

    at com.mysql.jdbc.StringUtils.toString(StringUtils.java:1871)

    at com.mysql.jdbc.ResultSetRow.getString(ResultSetRow.java:821)

    at com.mysql.jdbc.BufferRow.getString(BufferRow.java:542)

    at
com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5815)

    at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5692)

    at
org.apache.commons.dbcp.DelegatingResultSet.getString(DelegatingResultSet.java:175)

    at
org.sakaiproject.site.impl.DbSiteService$DbStorage.readSqlResultRecord(DbSiteService.java:2129)

    at
org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:575)

    at
org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:471)

    at
org.sakaiproject.util.BaseDbFlatStorage.getSelectedResources(BaseDbFlatStorage.java:471)

    at
org.sakaiproject.site.impl.DbSiteService$DbStorage.getSites(DbSiteService.java:803)

    at
org.sakaiproject.site.impl.BaseSiteService.getSites(BaseSiteService.java:1623)

    at
org.sakaiproject.portal.service.SiteNeighbourhoodServiceImpl.getAllSites(SiteNeighbourhoodServiceImpl.java:132)

    at
org.sakaiproject.portal.service.SiteNeighbourhoodServiceImpl.getSitesAtNode(SiteNeighbourhoodServiceImpl.java:96)

    at
org.sakaiproject.portal.charon.site.AbstractSiteViewImpl.<init>(AbstractSiteViewImpl.java:95)

    at
org.sakaiproject.portal.charon.site.DefaultSiteViewImpl.<init>(DefaultSiteViewImpl.java:61)

    at
org.sakaiproject.portal.charon.site.PortalSiteHelperImpl.getSitesView(PortalSiteHelperImpl.java:1167)

    at
org.sakaiproject.portal.charon.handlers.SiteHandler.includeTabs(SiteHandler.java:650)

    at
org.sakaiproject.portal.charon.handlers.SiteHandler.includeSiteNav(SiteHandler.java:438)

    at
org.sakaiproject.portal.charon.handlers.SiteHandler.doSite(SiteHandler.java:267)

    at
org.sakaiproject.portal.charon.handlers.SiteHandler.doGet(SiteHandler.java:126)

    at
org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:862)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

    at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)

    at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)

    at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:659)

    at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)





Tool Placement:

No Placement





Request:

:    AuthType:null

:    CharEncoding:UTF-8

:    ContentLength:-1

:    ContentType:null

:    ContextPath:/portal

:    LocalAddress:134.173.69.32

:    LocalName:sakai-tc2.pomona.edu

:    LocalPort:8444

:    Method:GET

:    PathInfo:/

:    Protocol:HTTP/1.1

:    QueryString:null

:    RemoteAddress:134.173.69.30

:    RemoteHost:134.173.69.30

:    RemotePort:33308

:    Requested URL:https://sakai.claremont.edu/portal

:    Scheme:https

:    ServerName:sakai.claremont.edu

:    Headers:

:        Header:host:134.173.69.32:8444

:        Header:cache-control:max-age=0

:
Header:accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8

:        Header:user-agent:Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36

:        Header:accept-encoding:gzip,deflate,sdch

:        Header:accept-language:en-US,en;q=0.8

:        Header:cookie:---censored---

:        Header:x-forwarded-for:134.173.65.106

:        Header:x-forwarded-host:sakai.claremont.edu

:        Header:x-forwarded-server:Sakai.Claremont.Edu

:        Header:connection:Keep-Alive

:    Parameters:

:    Attributes:

:
Attribute:sakai.character.encoding.done:sakai.character.encoding.done

:        Attribute:javax.servlet.request.key_size:168

:        Attribute:sakai.filtered:sakai.filtered

:
Attribute:sakai.session:org.sakaiproject.tool.impl.MySession at 26e5415e

:
Attribute:javax.servlet.request.ssl_session:534812961421c3760e3231cecb9a992e5daef31954abcaebfb96bf0fced81db2

:
 Attribute:javax.servlet.request.cipher_suite:SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA

Session:

:    Created:1397232174139

:    LastAccess:1397232461852

:    MaxInactive:10800

:    Attributes:

:        Attribute:portalskin:defaultskin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20140411/a9013b03/attachment.html