[Building Sakai] Production Sakai 2.7.0 deadlocks

Stephen Marquard stephen.marquard at uct.ac.za
Wed Aug 4 04:32:51 PDT 2010


Hi David,

The "GregorSama" message is just a side-effect of some garbage
collection taking place, and is probably unrelated to your problem.

http://thedailywtf.com/Articles/Who_is_Gregor_Samsa_0x3f_.aspx

You might have run into a dbcp issue. Switching to c3p0 might solve it,
or adjusting the number of db connections allowed and/or the Oracle
config. The problem and results seem to be Oracle-specific so I will
defer to all our talented Oracle DBAs out there to give you better
advice :-)

Regards
Stephen 
 
>>> David Barroso <david at asic.udl.cat> 8/4/2010 12:50 PM >>> 
Hi all,

We have a Sakai 2.7.0 instance in production using
Oracle 10g, Java1.6 and Linux.

When a high level of sessions is reached, the
java garbage collector starts working and the
following error is shown in catalina.out

Once the GregorSamsa class is unloaded and the
java.lang.IndexOutOfBoundsException appears,
the servers enters in a deadlock, waiting for
database connections

org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, 
pool error Timeout waiting for idle object
Caused by: java.util.NoSuchElementException: Timeout waiting for idle
object

We have found Oracle and Tomcat bug reports
talking about this, but in very old versions.

Any idea or clue to solve this problem ?

I've been watching the nightly2 oracle instance and
the Unloading class GregorSamsa appears at the end
of the catalina.out log.

http://nightly2.sakaiproject.org/logs/tomcat-nightly-oracle/

Regards,


-------------
our catalina.out

[Unloading class GregorSamsa]
[Unloading class GregorSamsa$2]
[Unloading class GregorSamsa$0]
[Unloading class GregorSamsa]
[Unloading class GregorSamsa$4]
[Unloading class GregorSamsa$0]
[Unloading class GregorSamsa]
[Unloading class GregorSamsa]
[Unloading class GregorSamsa$2]
[Unloading class GregorSamsa$3]
[Unloading class GregorSamsa$4]
[Unloading class GregorSamsa$1]
2010-08-04 12:17:07,075  WARN TP-Processor17 
org.apache.myfaces.application.jsp.JspViewHandlerImpl - no faces
servlet 
mappings found
2010-08-04 12:17:08,841  INFO TP-Processor10 
org.apache.myfaces.shared_impl.config.MyfacesConfig - Tomahawk jar not

available. Autoscrolling, DetectJavascript, AddResourceClass and 
CheckExtensionsFilter are disabled now.
2010-08-04 12:17:08,841  INFO TP-Processor10 
org.apache.myfaces.shared_impl.config.MyfacesConfig - Starting up 
Tomahawk on the MyFaces-JSF-Implementation
2010-08-04 12:17:12,435 ERROR http-8080-Processor61 
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/sakai-site-manage-tool].[sakai.membership]

- Servlet.service() for servlet sakai.membership threw exception
java.lang.ArrayIndexOutOfBoundsException: -31
         at 
sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:436)
         at 
java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2081)
         at 
java.util.GregorianCalendar.computeTime(GregorianCalendar.java:2472)
         at java.util.Calendar.updateTime(Calendar.java:2468)
         at java.util.Calendar.getTimeInMillis(Calendar.java:1087)
         at 
oracle.jdbc.driver.DateTimeCommonAccessor.getTimestamp(DateTimeCommonAccessor.java:256)
         at 
oracle.jdbc.driver.OracleResultSetImpl.getTimestamp(OracleResultSetImpl.java:1644)
         at 
org.apache.commons.dbcp.DelegatingResultSet.getTimestamp(DelegatingResultSet.java:548)
         at 
org.sakaiproject.site.impl.DbSiteService$DbStorage.readSqlResultRecord(DbSiteService.java:2128)
         at 
org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:564)
         at 
org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:464)
         at 
org.sakaiproject.util.BaseDbFlatStorage.getSelectedResources(BaseDbFlatStorage.java:470)
         at 
org.sakaiproject.site.impl.DbSiteService$DbStorage.getSites(DbSiteServicejava:782)
         at 
org.sakaiproject.site.impl.BaseSiteService.getSites(BaseSit
eService.java:1616)
         at 
org.sakaiproject.site.cover.SiteService.getSites(SiteService.java:400)
         at 
org.sakaiproject.site.tool.MembershipAction.sizeResources(MembershipAction.java:85)
         at 
org.sakaiproject.cheftool.PagedResourceActionII.prepPage(PagedResourceActionII.java:270)
         at 
org.sakaiproject.site.tool.MembershipAction.buildMainPanelContext(MembershipAction.java:196)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at 
org.sakaiproject.cheftool.VelocityPortletPaneledAction.toolModeDispatch(VelocityPortletPaneledAction.java:398)
         at 
org.sakaiproject.cheftool.ToolServlet.doGet(ToolServlet.java:236)
         at 
org.sakaiproject.cheftool.VelocityPortletPaneledAction.doGet(VelocityPortletPaneledAction.java:1029)
         at
javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
         at 
org.sakaiproject.vm.ComponentServlet.service(ComponentServlet.java:56)
         at
javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
         at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
         at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
         at 
org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:580)
         at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
         at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
         at 
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcherjava:659)
         at 
org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:457)
         at 
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:359)
         at 
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:311)
         at 
org.sakaiproject.tool.impl.ActiveToolComponent$MyActiveTool.forward(ActiveToolComponent.java:500)
         at 
org.sakaiproject.portal.charon.SkinnableCharonPortal.forwardTool(SkinnableCharonPortal.java:1216)
         at 
org.sakaiproject.portal.charon.handlers.ToolHandler.doTool(ToolHandler.java:204)
         at 
org.sakaiproject.portal.charon.handlers.ToolHandler.doGet(ToolHandler.java:96)
         at 
org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:768)


-- 
David Barroso Iglesias   david at asic.udl.cat
=====================
Universitat de Lleida
Àrea de sistemes d'Informació i Comunicacions
Cap de projectes interns

University of Lleida
Information and Communication Systems Service
Internal project manager

Tlf: +34 973 702040
Fax: +34 973 702130
=====================

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to
sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
"unsubscribe"



 

###
UNIVERSITY OF CAPE TOWN 

This e-mail is subject to the UCT ICT policies and e-mail disclaimer
published on our website at
http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from
+27 21 650 4500. This e-mail is intended only for the person(s) to whom
it is addressed. If the e-mail has reached you in error, please notify
the author. If you are not the intended recipient of the e-mail you may
not use, disclose, copy, redirect or print the content. If this e-mail
is not related to the business of UCT it is sent by the sender in the
sender's individual capacity.

###
 


More information about the sakai-dev mailing list