[Building Sakai] Tomcat not shutting down properly

Mike Jennings mike_jennings at unc.edu
Tue Mar 27 19:37:36 PDT 2012 

Maby the issue is with the Phoenix Shutdown.  When I raise log4j to 
debug I get this in the logs which seems to be related to a James server 
Jira

https://issues.apache.org/jira/browse/JAMES-767

Shutting down Phoenix.
2012-03-27 22:29:12,773  INFO main org.sakaiproject.james.SakaiMailet - 
destroy()
org.apache.excalibur.containerkit.lifecycle.LifecycleException: 
Component named "thread-manager" failed to pass through the Disposing 
stage. (Reason: java.lang.IllegalThreadStateException).
         at 
org.apache.excalibur.containerkit.lifecycle.LifecycleHelper.fail(LifecycleHelper.java:289)
         at 
org.apache.excalibur.containerkit.lifecycle.LifecycleHelper.shutdown(LifecycleHelper.java:223)
         at 
org.apache.avalon.phoenix.components.application.DefaultApplication.shutdown(DefaultApplication.java:518)
         at 
org.apache.avalon.phoenix.components.application.DefaultApplication.doRunPhase(DefaultApplication.java:433)
         at 
org.apache.avalon.phoenix.components.application.DefaultApplication.runPhase(DefaultApplication.java:365)
         at 
org.apache.avalon.phoenix.components.application.DefaultApplication.stop(DefaultApplication.java:192)
         at 
org.apache.avalon.framework.container.ContainerUtil.stop(ContainerUtil.java:292)
         at 
org.apache.avalon.framework.container.ContainerUtil.shutdown(ContainerUtil.java:66)
         at 
org.apache.avalon.phoenix.components.kernel.DefaultKernel.shutdown(DefaultKernel.java:228)
         at 
org.apache.avalon.phoenix.components.kernel.DefaultKernel.removeApplication(DefaultKernel.java:331)
         at 
org.apache.avalon.phoenix.components.deployer.DefaultDeployer.undeploy(DefaultDeployer.java:248)
         at 
org.apache.avalon.phoenix.components.deployer.DefaultDeployer.dispose(DefaultDeployer.java:184)
         at 
org.apache.avalon.framework.container.ContainerUtil.dispose(ContainerUtil.java:306)
         at 
org.apache.avalon.framework.container.ContainerUtil.shutdown(ContainerUtil.java:67)
         at 
org.apache.avalon.phoenix.components.embeddor.DefaultEmbeddor.shutdownComponents(DefaultEmbeddor.java:554)
         at 
org.apache.avalon.phoenix.components.embeddor.DefaultEmbeddor.dispose(DefaultEmbeddor.java:298)
         at 
org.apache.avalon.framework.container.ContainerUtil.dispose(ContainerUtil.java:306)
         at 
org.apache.avalon.framework.container.ContainerUtil.shutdown(ContainerUtil.java:67)
         at 
org.apache.avalon.phoenix.frontends.CLIMain.shutdown(CLIMain.java:286)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at 
org.sakaiproject.james.PhoenixLauncherMain.shutdown(PhoenixLauncherMain.java:151)
         at 
org.sakaiproject.james.JamesServlet.destroy(JamesServlet.java:145)
         at 
org.apache.catalina.core.StandardWrapper.unload(StandardWrapper.java:1360)
         at 
org.apache.catalina.core.StandardWrapper.stop(StandardWrapper.java:1688)
         at 
org.apache.catalina.core.StandardContext.stop(StandardContext.java:4352)
         at 
org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:893)
         at 
org.apache.catalina.startup.HostConfig.undeployApps(HostConfig.java:1245)
         at 
org.apache.catalina.startup.HostConfig.stop(HostConfig.java:1216)
         at 
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:321)
         at 
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:120)
         at 
org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1055)
         at 
org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1067)
         at 
org.apache.catalina.core.StandardEngine.stop(StandardEngine.java:448)
         at 
org.apache.catalina.core.StandardService.stop(StandardService.java:510)
         at 
org.apache.catalina.core.StandardServer.stop(StandardServer.java:734)
         at org.apache.catalina.startup.Catalina.stop(Catalina.java:602)
         at org.apache.catalina.startup.Catalina.start(Catalina.java:577)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433)
Caused by: java.lang.IllegalThreadStateException
         at java.lang.ThreadGroup.destroy(ThreadGroup.java:754)
         at 
org.apache.avalon.excalibur.thread.impl.DefaultThreadPool.dispose(DefaultThreadPool.java:76)
         at 
org.apache.avalon.framework.container.ContainerUtil.dispose(ContainerUtil.java:306)
         at 
org.apache.avalon.cornerstone.blocks.threads.AbstractThreadManager.dispose(AbstractThreadManager.java:69)
         at 
org.apache.avalon.framework.container.ContainerUtil.dispose(ContainerUtil.java:306)
         at 
org.apache.excalibur.containerkit.lifecycle.LifecycleHelper.shutdown(LifecycleHelper.java:209)






==============================================================================
Mike Jennings
Teaching and Learning Developer
University of North Carolina at Chapel Hill

Office: (919) 843-5013
Cell: (919) 698-3746
E-mail: mike_jennings at unc.edu

On 3/27/2012 8:47 PM, Aaron Zeckoski wrote:
> That code you are looking at is part of the ReferenceMap which is
> actually code from part of google's guice project. I didn't want to
> depend on that entire project. That map is supposed to in theory hold
> references to objects weakly in order to keep them from causing issues
> on shutdown or garbage collection.
>
> http://code.google.com/p/reflectutils/source/browse/trunk/src/main/java/org/azeckoski/reflectutils/refmap/ReferenceMap.java
>
> That said, as far as I know, the ReferenceMap is only used in
> entitybroker as a way to track the providers which are registered from
> across classloaders in various webapps and components. This is why it
> can't just be a normal map.
>
> It may very well not be working but at the time (about 2007 or 2008) I
> could not find anything out there that worked better and in my tests
> in Java 1.5 and tomcat 5.5 it did allow things to cleanly shutdown
> that otherwise would hang. Might not be working anymore but the
> finalizer in there is supposed to shutdown after everything else so it
> might simply be showing up because other things haven't shutdown yet.
>
> -AZ
>
>
> On Tue, Mar 27, 2012 at 7:47 PM, Adams, David<da1 at vt.edu>  wrote:
>> We've seen this behavior for years. I find it amusing that Aaron's name is in the culprit class. Nevertheless we just have our shutdown script check to see if tomcat is stopped at intervals and kill it after a reasonable amount of time (we've settled on about 15 seconds). It only has to resort to that about half of the time I'd guess. There don't seem to be any ill effects from this forced shutdown. Most everything has shutdown correctly by then, but we've been running that way for years.
>>
>> On Mar 27, 2012, at 11:03 AM, "Brian Jones"<bjones86 at uwo.ca>  wrote:
>>
>>> Community,
>>>
>>> Recently we here at Western University polled the community for existing
>>> certification tools for Sakai. Rsmart was kind enough to provide their
>>> certification tool as contrib.
>>>
>>> We've tested deploying this tool on our local and dev environments and
>>> noticed something troubling. In a nutshell, after deploying the tool, it
>>> causes Tomcat to be unable to shut down properly. It's always waiting for
>>> some threads that are waiting for something.
>>>
>>> Please find attached two files, one outlining the test plan which verified
>>> the certification tool as the culprit, the second a thread dump. I noticed
>>> in the thread dump there are two suspicious entries:
>>>
>>> . "SakaiPresenceService.Maintenance"
>>> . "org.azeckoski.reflectutils.refmap.Finalizer"
>>>
>>> But I can't find any references to these objects/packages/methods in any the
>>> code of either rsmart-common or the certification tool itself. We're kind of
>>> at a loss as to how to track this issue down, or what to do to resolve it.
>>> If anyone in the community, or at Rsmart, has an ideas we would be extremely
>>> appreciative.
>>>
>>> Thanks in advance,
>>>
>>> Brian Jones
>>> Applications Development
>>> Information Technology Services
>>> Support Services Building, Room 4326
>>> University of Western Ontario
>>> (519) 661-2111 x86969
>>> bjones86 at uwo.ca
>>>
>>> <Test Plan.txt>
>>> <threadDump.log>
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
>

More information about the sakai-dev mailing list