[Deploying Sakai] Profile2 issue [was: Problems with deprecated tools after upgrade]

Steve Swinsburg steve.swinsburg at gmail.com
Mon Aug 16 04:23:06 PDT 2010


Hi Markus,

The issue seems to be this:
profileLogic.createDefaultPreferencesRecord(userUuid)

which will create a record if you don't already have one. But that shouldn't block anything, it's a simple insert by Hibernate. I noticed a few MySQL exceptions in there, what do your MySQL logs say?

cheers,
Steve



On 16/08/2010, at 6:15 PM, Markus Binsteiner wrote:

> Hi Steve.
> 
>> What version Profile2 are you running? And how many users might you have in your system?
> 
> The latest one, 1.3.10. Had the same behaviour with 1.3.9 though. There
> was no earlier version of Profile2 involved, just Profile1. We have a
> few hundred users...
> 
>> 
>> Can you try a thread dump? This happens when one request takes a long time and block another incoming request so it's getting stuck somewhere processing in the backend.
>> 
>> Here's a couple of discussions about it:
>> http://apache-wicket.1842946.n4.nabble.com/After-1-minute-the-Pagemap-null-is-still-locked-td1890832.html
>> http://apache-wicket.1842946.n4.nabble.com/pagemap-locking-td1932905.html
>> 
> Thanks for that. The thread dump is pasted below...
> 
> Cheers,
> Markus
> 
> 
> 2010-08-16 20:07:50,517  INFO TP-Processor5
> org.sakaiproject.profile2.logic.ProfileLogicImpl - Created default
> preferences record for user: 7a41c257-1336-4e1b-809a-e5bec01877f3
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.5.0_22-b03 mixed
> mode):
> 
> "TP-Processor12" daemon prio=1 tid=0x00007fb1f3298dd0 nid=0x2924
> runnable [0x0000000045715000..0x0000000045715ca0]
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
> 	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> 	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
> 	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
> 	- locked <0x00007fb20e9329b8> (a java.io.BufferedInputStream)
> 	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
> 	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
> 	at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
> 	at org.apache.jk.common.ChannelSocket
> $SocketConnection.runIt(ChannelSocket.java:897)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor11" daemon prio=1 tid=0x00007fb1f295d4f0 nid=0x2923 in
> Object.wait() [0x0000000045614000..0x0000000045614d20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e77f850> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e77f850> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor10" daemon prio=1 tid=0x00007fb1ef05cb50 nid=0x2922 in
> Object.wait() [0x0000000045513000..0x0000000045513da0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e77f4a8> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e77f4a8> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor9" daemon prio=1 tid=0x00007fb1f24162a0 nid=0x2921 in
> Object.wait() [0x0000000045412000..0x0000000045412e20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e77f030> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e77f030> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "PageSavingThread-sakai.profile2" daemon prio=1 tid=0x00007fb1e5349c40
> nid=0x2908 sleeping[0x0000000045311000..0x0000000045311aa0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at org.apache.wicket.protocol.http.pagestore.DiskPageStore
> $PageSavingThread.run(DiskPageStore.java:947)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor8" daemon prio=1 tid=0x00007fb1f1a1c490 nid=0x2905 runnable
> [0x0000000045210000..0x0000000045210b20]
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
> 	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> 	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
> 	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
> 	- locked <0x00007fb1f494ed20> (a java.io.BufferedInputStream)
> 	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
> 	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
> 	at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
> 	at org.apache.jk.common.ChannelSocket
> $SocketConnection.runIt(ChannelSocket.java:897)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor7" daemon prio=1 tid=0x00007fb1f1a1b970 nid=0x2904 runnable
> [0x000000004510f000..0x000000004510fba0]
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
> 	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> 	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
> 	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
> 	- locked <0x00007fb20e8eb970> (a java.io.BufferedInputStream)
> 	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
> 	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
> 	at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
> 	at org.apache.jk.common.ChannelSocket
> $SocketConnection.runIt(ChannelSocket.java:897)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor6" daemon prio=1 tid=0x00007fb1f1a1ae50 nid=0x2903 runnable
> [0x000000004500e000..0x000000004500ec20]
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
> 	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> 	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
> 	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
> 	- locked <0x00007fb20e8ee728> (a java.io.BufferedInputStream)
> 	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
> 	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
> 	at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
> 	at org.apache.jk.common.ChannelSocket
> $SocketConnection.runIt(ChannelSocket.java:897)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor5" daemon prio=1 tid=0x00007fb1f27c1400 nid=0x2902 runnable
> [0x0000000044f0a000..0x0000000044f0dca0]
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:129)
> 	at
> com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
> 	at
> com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
> 	at
> com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
> 	- locked <0x00007fb20e9991c0> (a
> com.mysql.jdbc.util.ReadAheadInputStream)
> 	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2499)
> 	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2952)
> 	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2941)
> 	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3489)
> 	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959)
> 	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2113)
> 	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
> 	- locked <0x00007fb20e995008> (a java.lang.Object)
> 	at
> com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2113)
> 	at
> com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2409)
> 	- locked <0x00007fb20e995008> (a java.lang.Object)
> 	at
> com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1974)
> 	at
> com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1451)
> 	- locked <0x00007fb20e995008> (a java.lang.Object)
> 	at
> org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
> 	at
> org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48)
> 	at
> org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:247)
> 	at
> org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266)
> 	at
> org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)
> 	at
> org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
> 	at
> org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
> 	at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1001)
> 	at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:339)
> 	at
> org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
> 	at
> org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:655)
> 	at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
> 	at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
> 	at
> org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
> 	at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
> 	at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> 	at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> 	at $Proxy29.createDefaultPreferencesRecord(Unknown Source)
> 	at
> org.sakaiproject.profile2.tool.pages.MyPreferences.<init>(MyPreferences.java:52)
> 	at org.sakaiproject.profile2.tool.pages.BasePage
> $5.onClick(BasePage.java:126)
> 	at org.apache.wicket.markup.html.link.Link.onLinkClicked(Link.java:214)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:592)
> 	at
> org.apache.wicket.RequestListenerInterface.invoke(RequestListenerInterface.java:183)
> 	at
> org.apache.wicket.request.target.component.listener.ListenerInterfaceRequestTarget.processEvents(ListenerInterfaceRequestTarget.java:73)
> 	at
> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:91)
> 	at
> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1220)
> 	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1297)
> 	at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1399)
> 	at org.apache.wicket.RequestCycle.request(RequestCycle.java:529)
> 	at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:356)
> 	at
> org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:124)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> 	at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:585)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> 	at
> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:659)
> 	at
> org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:457)
> 	at
> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:359)
> 	at
> org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:311)
> 	at org.sakaiproject.tool.impl.ActiveToolComponent
> $MyActiveTool.forward(ActiveToolComponent.java:500)
> 	at
> org.sakaiproject.portal.charon.SkinnableCharonPortal.forwardTool(SkinnableCharonPortal.java:1221)
> 	at
> org.sakaiproject.portal.charon.handlers.ToolHandler.doTool(ToolHandler.java:204)
> 	at
> org.sakaiproject.portal.charon.handlers.ToolHandler.doGet(ToolHandler.java:96)
> 	at
> org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:768)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> 	at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:646)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
> 	at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
> 	at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
> 	at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
> 	at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> 	at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
> 	at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
> 	at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
> 	at
> org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> 	at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
> 	at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775)
> 	at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704)
> 	at org.apache.jk.common.ChannelSocket
> $SocketConnection.runIt(ChannelSocket.java:897)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Monitor" daemon prio=1 tid=0x00007fb1eaaa9bd0 nid=0x28f9 in
> Object.wait() [0x0000000044e0c000..0x0000000044e0cd20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e285d10> (a
> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $MonitorRunnable.run(ThreadPool.java:564)
> 	- locked <0x00007fb20e285d10> (a
> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor4" daemon prio=1 tid=0x00007fb1ebade280 nid=0x28f8 runnable
> [0x0000000044d0b000..0x0000000044d0bda0]
> 	at java.net.PlainSocketImpl.socketAccept(Native Method)
> 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> 	- locked <0x00007fb20e283138> (a java.net.SocksSocketImpl)
> 	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> 	at java.net.ServerSocket.accept(ServerSocket.java:421)
> 	at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:313)
> 	at
> org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:667)
> 	at org.apache.jk.common.ChannelSocket
> $SocketAcceptor.runIt(ChannelSocket.java:878)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor3" daemon prio=1 tid=0x00007fb1eaa60b30 nid=0x28f7 in
> Object.wait() [0x0000000044c0a000..0x0000000044c0ae20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e2854f0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e2854f0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor2" daemon prio=1 tid=0x00007fb1e9ef6460 nid=0x28f6 in
> Object.wait() [0x0000000044b09000..0x0000000044b09aa0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e2850e0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e2850e0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "TP-Processor1" daemon prio=1 tid=0x00007fb1e9e858f0 nid=0x28f5 in
> Object.wait() [0x0000000044a08000..0x0000000044a08b20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e284cd0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e284cd0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Monitor" prio=1 tid=0x00007fb1e9ed1ba0 nid=0x28f4 in
> Object.wait() [0x0000000044907000..0x0000000044907ba0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14deb0> (a
> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $MonitorRunnable.run(ThreadPool.java:564)
> 	- locked <0x00007fb20e14deb0> (a
> org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor25" daemon prio=1 tid=0x00007fb1e9ed0b60 nid=0x28f3
> runnable [0x0000000044806000..0x0000000044806c20]
> 	at java.net.PlainSocketImpl.socketAccept(Native Method)
> 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> 	- locked <0x00007fb1ff4146d8> (a java.net.SocksSocketImpl)
> 	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> 	at java.net.ServerSocket.accept(ServerSocket.java:421)
> 	at
> org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
> 	at
> org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:408)
> 	at
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:689)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor24" daemon prio=1 tid=0x00007fb1e9ebe570 nid=0x28f2
> in Object.wait() [0x0000000044705000..0x0000000044705ca0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14d670> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14d670> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor23" daemon prio=1 tid=0x00007fb1ea022690 nid=0x28f1
> in Object.wait() [0x0000000044604000..0x0000000044604d20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14d250> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14d250> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor22" daemon prio=1 tid=0x00007fb1d83d9de0 nid=0x28f0
> in Object.wait() [0x0000000044503000..0x0000000044503da0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14ce30> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14ce30> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor21" daemon prio=1 tid=0x00007fb1ea021ff0 nid=0x28ef
> in Object.wait() [0x0000000044402000..0x0000000044402e20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14ca10> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14ca10> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor20" daemon prio=1 tid=0x00007fb1ea021040 nid=0x28ee
> in Object.wait() [0x0000000044301000..0x0000000044301aa0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14c5f0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14c5f0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor19" daemon prio=1 tid=0x00007fb1ea845550 nid=0x28ed
> in Object.wait() [0x0000000044200000..0x0000000044200b20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14c1d0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14c1d0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor18" daemon prio=1 tid=0x00007fb1eaa8e620 nid=0x28ec
> in Object.wait() [0x00000000440ff000..0x00000000440ffba0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14bc20> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14bc20> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor17" daemon prio=1 tid=0x00007fb1eaa8d620 nid=0x28eb
> in Object.wait() [0x0000000043ffe000..0x0000000043ffec20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14b800> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14b800> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor16" daemon prio=1 tid=0x00007fb1ea529cd0 nid=0x28ea
> in Object.wait() [0x0000000043efd000..0x0000000043efdca0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14b3e0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14b3e0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor15" daemon prio=1 tid=0x00007fb1ea529a20 nid=0x28e9
> in Object.wait() [0x0000000043dfc000..0x0000000043dfcd20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14afc0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14afc0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor14" daemon prio=1 tid=0x00007fb1e9e57530 nid=0x28e8
> in Object.wait() [0x0000000043cfb000..0x0000000043cfbda0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14aba0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14aba0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor13" daemon prio=1 tid=0x00007fb1e9e56620 nid=0x28e7
> in Object.wait() [0x0000000043bfa000..0x0000000043bfae20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14a780> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14a780> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor12" daemon prio=1 tid=0x00007fb1e9e84730 nid=0x28e6
> in Object.wait() [0x0000000043af9000..0x0000000043af9aa0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e14a360> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e14a360> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor11" daemon prio=1 tid=0x00007fb1eb3061b0 nid=0x28e5
> in Object.wait() [0x00000000439f8000..0x00000000439f8b20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e149f40> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e149f40> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor10" daemon prio=1 tid=0x00007fb1eac56350 nid=0x28e4
> in Object.wait() [0x00000000438f7000..0x00000000438f7ba0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e149b20> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e149b20> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor9" daemon prio=1 tid=0x00007fb1eb1e69e0 nid=0x28e3
> in Object.wait() [0x00000000437f6000..0x00000000437f6c20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e149708> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e149708> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor8" daemon prio=1 tid=0x00007fb1ea970520 nid=0x28e2
> in Object.wait() [0x00000000436f5000..0x00000000436f5ca0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e1492f0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e1492f0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor7" daemon prio=1 tid=0x00007fb1ea56f630 nid=0x28e1
> in Object.wait() [0x00000000435f4000..0x00000000435f4d20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e148ed8> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e148ed8> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor6" daemon prio=1 tid=0x00007fb1ea6ef4a0 nid=0x28e0
> in Object.wait() [0x00000000434f3000..0x00000000434f3da0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e148ac0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e148ac0> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor5" daemon prio=1 tid=0x00007fb1e9eb03c0 nid=0x28df
> in Object.wait() [0x00000000433f2000..0x00000000433f2e20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e1486a8> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e1486a8> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor4" daemon prio=1 tid=0x00007fb1ebe56c50 nid=0x28de
> in Object.wait() [0x00000000432f1000..0x00000000432f1aa0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e148290> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e148290> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor3" daemon prio=1 tid=0x00007fb1ebdd88a0 nid=0x28dd
> in Object.wait() [0x00000000431f0000..0x00000000431f0b20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e147e78> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e147e78> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor2" daemon prio=1 tid=0x00007fb1ea937f10 nid=0x28dc
> in Object.wait() [0x00000000430ef000..0x00000000430efba0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e147a60> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e147a60> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "http-8080-Processor1" daemon prio=1 tid=0x00007fb1eaa978c0 nid=0x28db
> in Object.wait() [0x0000000042fee000..0x0000000042feec20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20e147648> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Object.wait(Object.java:474)
> 	at org.apache.tomcat.util.threads.ThreadPool
> $ControlRunnable.run(ThreadPool.java:661)
> 	- locked <0x00007fb20e147648> (a
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=1
> tid=0x00007fb1d8683290 nid=0x28da waiting on condition
> [0x0000000042eed000..0x0000000042eedca0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at org.apache.catalina.core.ContainerBase
> $ContainerBackgroundProcessor.run(ContainerBase.java:1548)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "Timer-3" daemon prio=1 tid=0x00007fb1e733fc60 nid=0x28cc in
> Object.wait() [0x0000000042dec000..0x0000000042decaa0]
> 	at java.lang.Object.wait(Native Method)
> 	at java.util.TimerThread.mainLoop(Timer.java:509)
> 	- locked <0x00007fb205436a38> (a java.util.TaskQueue)
> 	at java.util.TimerThread.run(Timer.java:462)
> 
> "Store uk.ac.cam.caret.sakai.rwiki.service.api.radeox.RenderCache Spool
> Thread" daemon prio=1 tid=0x00000000401a8690 nid=0x28c9
> sleeping[0x0000000042ceb000..0x0000000042cebb20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store
> org.sakaiproject.api.privacy.PrivacyManager.PrivacyQueryCache.queryGetPrivacy Spool Thread" daemon prio=1 tid=0x00007fb1f3a99090 nid=0x28c8 sleeping[0x0000000042bea000..0x0000000042beaba0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "SakaiPresenceService.Maintenance" daemon prio=1 tid=0x00007fb1f2b745f0
> nid=0x28c7 waiting on condition [0x0000000042ae9000..0x0000000042ae9c20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at org.sakaiproject.presence.impl.BasePresenceService
> $Maintenance.run(BasePresenceService.java:791)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "Store org.sakaiproject.user.api.AuthenticationManager Spool Thread"
> daemon prio=1 tid=0x00007fb1f33c7980 nid=0x28c4
> sleeping[0x00000000429e8000..0x00000000429e8ca0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Timer-2" daemon prio=1 tid=0x00007fb1f2732720 nid=0x28c3 in
> Object.wait() [0x00000000428e7000..0x00000000428e7d20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20286c450> (a java.util.TaskQueue)
> 	at java.util.TimerThread.mainLoop(Timer.java:509)
> 	- locked <0x00007fb20286c450> (a java.util.TaskQueue)
> 	at java.util.TimerThread.run(Timer.java:462)
> 
> "Store
> org.sakaiproject.citation.api.SearchManager.metasearchSessionManagerCache Spool Thread" daemon prio=1 tid=0x00007fb1f3be40d0 nid=0x28c2 sleeping[0x00000000427e6000..0x00000000427e6da0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.citation.api.SearchManager.sessionContextCache
> Spool Thread" daemon prio=1 tid=0x00007fb1e5c9dac0 nid=0x28c1
> sleeping[0x00000000426e5000..0x00000000426e5e20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.theospi.portfolio.wizard.mgt.impl.WizardManagerImpl.cache
> Spool Thread" daemon prio=1 tid=0x00007fb1f1b13fb0 nid=0x28c0
> sleeping[0x00000000425e4000..0x00000000425e4aa0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "SakaiClusterService.Maintenance" daemon prio=1 tid=0x00007fb1ea0bba70
> nid=0x28bf waiting on condition [0x00000000424e3000..0x00000000424e3b20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at org.sakaiproject.cluster.impl.SakaiClusterService
> $Maintenance.run(SakaiClusterService.java:469)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "IndexManager" daemon prio=1 tid=0x00007fb1e9d26950 nid=0x28be in
> Object.wait() [0x00000000423e2000..0x00000000423e2ba0]
> 	at java.lang.Object.wait(Native Method)
> 	at java.util.TimerThread.mainLoop(Timer.java:509)
> 	- locked <0x00007fb20257a780> (a java.util.TaskQueue)
> 	at java.util.TimerThread.run(Timer.java:462)
> 
> "org.sakaiproject.sitestats.impl.StatsUpdateManagerImpl" prio=1
> tid=0x00007fb1ea132790 nid=0x28bd in Object.wait()
> [0x00000000422e1000..0x00000000422e1c20]
> 	at java.lang.Object.wait(Native Method)
> 	at
> org.sakaiproject.sitestats.impl.StatsUpdateManagerImpl.run(StatsUpdateManagerImpl.java:485)
> 	- locked <0x00007fb20257a878> (a java.lang.Object)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "org.azeckoski.reflectutils.refmap.Finalizer" daemon prio=1
> tid=0x00007fb1f29da430 nid=0x28ba in Object.wait()
> [0x00000000421e0000..0x00000000421e0ca0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb20220c940> (a java.lang.ref.ReferenceQueue$Lock)
> 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
> 	- locked <0x00007fb20220c940> (a java.lang.ref.ReferenceQueue$Lock)
> 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:136)
> 	at org.azeckoski.reflectutils.refmap.Finalizer.run(Finalizer.java:127)
> 
> "QuartzScheduler_QuartzScheduler-vre_MisfireHandler" prio=1
> tid=0x00007fb1e5ea5ef0 nid=0x28b9 waiting on condition
> [0x00000000420df000..0x00000000420dfd20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at org.quartz.impl.jdbcjobstore.JobStoreSupport
> $MisfireHandler.run(JobStoreSupport.java:3914)
> 
> "QuartzScheduler_QuartzSchedulerThread" prio=1 tid=0x00007fb1f1a63c70
> nid=0x28b8 in Object.wait() [0x0000000041fde000..0x0000000041fdeda0]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb2021e2550> (a java.lang.Object)
> 	at
> org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:433)
> 	- locked <0x00007fb2021e2550> (a java.lang.Object)
> 
> "QuartzScheduler_Worker-5" prio=1 tid=0x00007fb1f1393ab0 nid=0x28b7 in
> Object.wait() [0x0000000041edd000..0x0000000041edde20]
> 	at java.lang.Object.wait(Native Method)
> 	at org.quartz.simpl.SimpleThreadPool
> $WorkerThread.run(SimpleThreadPool.java:528)
> 	- locked <0x00007fb2021e0f38> (a org.quartz.simpl.SimpleThreadPool
> $WorkerThread)
> 
> "QuartzScheduler_Worker-4" prio=1 tid=0x00007fb1f1393410 nid=0x28b6 in
> Object.wait() [0x0000000041ddc000..0x0000000041ddcaa0]
> 	at java.lang.Object.wait(Native Method)
> 	at org.quartz.simpl.SimpleThreadPool
> $WorkerThread.run(SimpleThreadPool.java:528)
> 	- locked <0x00007fb2021e0c98> (a org.quartz.simpl.SimpleThreadPool
> $WorkerThread)
> 
> "QuartzScheduler_Worker-3" prio=1 tid=0x00007fb1e5ea72f0 nid=0x28b5 in
> Object.wait() [0x0000000041cdb000..0x0000000041cdbb20]
> 	at java.lang.Object.wait(Native Method)
> 	at org.quartz.simpl.SimpleThreadPool
> $WorkerThread.run(SimpleThreadPool.java:528)
> 	- locked <0x00007fb2021e09f8> (a org.quartz.simpl.SimpleThreadPool
> $WorkerThread)
> 
> "QuartzScheduler_Worker-2" prio=1 tid=0x00007fb1f18ea210 nid=0x28b4 in
> Object.wait() [0x0000000041bda000..0x0000000041bdaba0]
> 	at java.lang.Object.wait(Native Method)
> 	at org.quartz.simpl.SimpleThreadPool
> $WorkerThread.run(SimpleThreadPool.java:528)
> 	- locked <0x00007fb2021e0758> (a org.quartz.simpl.SimpleThreadPool
> $WorkerThread)
> 
> "QuartzScheduler_Worker-1" prio=1 tid=0x00007fb1f2161e70 nid=0x28b3 in
> Object.wait() [0x0000000041ad9000..0x0000000041ad9c20]
> 	at java.lang.Object.wait(Native Method)
> 	at org.quartz.simpl.SimpleThreadPool
> $WorkerThread.run(SimpleThreadPool.java:528)
> 	- locked <0x00007fb2021e04b8> (a org.quartz.simpl.SimpleThreadPool
> $WorkerThread)
> 
> "Store org.sakaiproject.springframework.orm.hibernate.L2Cache Spool
> Thread" daemon prio=1 tid=0x00007fb1f2f9d690 nid=0x28b0
> sleeping[0x00000000419d8000..0x00000000419d8ca0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.event.api.UsageSessionService.recentUserRefresh
> Spool Thread" daemon prio=1 tid=0x00007fb1e922a030 nid=0x28af
> sleeping[0x00000000418d7000..0x00000000418d7d20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "org.sakaiproject.event.impl.ClusterEventTracking$$EnhancerByCGLIB
> $$33ba687" daemon prio=1 tid=0x00007fb1e9919630 nid=0x28ae waiting on
> condition [0x00000000417d6000..0x00000000417d6da0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> org.sakaiproject.event.impl.ClusterEventTracking.run(ClusterEventTracking.java:592)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "Timer-1" daemon prio=1 tid=0x00007fb1e89be080 nid=0x28ad in
> Object.wait() [0x00000000416d5000..0x00000000416d5e20]
> 	at java.lang.Object.wait(Native Method)
> 	at java.util.TimerThread.mainLoop(Timer.java:509)
> 	- locked <0x00007fb1fff11220> (a java.util.TaskQueue)
> 	at java.util.TimerThread.run(Timer.java:462)
> 
> "Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_USER_PROPERTY Spool
> Thread" daemon prio=1 tid=0x00007fb1e922b7a0 nid=0x28ac
> sleeping[0x00000000415d4000..0x00000000415d4aa0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_GROUP_PROPERTY
> Spool Thread" daemon prio=1 tid=0x00007fb1e922b490 nid=0x28ab
> sleeping[0x00000000414d3000..0x00000000414d3b20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_ALIAS_PROPERTY Spool
> Thread" daemon prio=1 tid=0x00007fb1e922ac10 nid=0x28aa
> sleeping[0x00000000413d2000..0x00000000413d2ba0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_REALM_PROPERTY Spool
> Thread" daemon prio=1 tid=0x00007fb1e922a450 nid=0x28a9
> sleeping[0x00000000412d1000..0x00000000412d1c20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PAGE_PROPERTY
> Spool Thread" daemon prio=1 tid=0x00007fb1e9c572d0 nid=0x28a8
> sleeping[0x00000000411d0000..0x00000000411d0ca0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PROPERTY Spool
> Thread" daemon prio=1 tid=0x00007fb1e9c55790 nid=0x28a7
> sleeping[0x00000000410cf000..0x00000000410cfd20]
> 	at java.lang.Thread.sleep(Native Method)
> 	at
> net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
> 	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
> 	at net.sf.ehcache.store.DiskStore
> $SpoolAndExpiryThread.run(DiskStore.java:1104)
> 
> "Sakai.SessionComponent.Maintenance" daemon prio=1
> tid=0x00007fb1e9c554c0 nid=0x28a6 waiting on condition
> [0x0000000040fce000..0x0000000040fceda0]
> 	at java.lang.Thread.sleep(Native Method)
> 	at org.sakaiproject.tool.impl.SessionComponent
> $Maintenance.run(SessionComponent.java:542)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "RMI TCP Accept-0" daemon prio=1 tid=0x00007fb1f4274020 nid=0x28a1
> runnable [0x0000000040dcc000..0x0000000040dccaa0]
> 	at java.net.PlainSocketImpl.socketAccept(Native Method)
> 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> 	- locked <0x00007fb1ff2301b0> (a java.net.SocksSocketImpl)
> 	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> 	at java.net.ServerSocket.accept(ServerSocket.java:421)
> 	at sun.management.jmxremote.LocalRMIServerSocketFactory
> $1.accept(LocalRMIServerSocketFactory.java:31)
> 	at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:340)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "RMI TCP Accept-1234" daemon prio=1 tid=0x00007fb1f4272b40 nid=0x28a0
> runnable [0x0000000040ccb000..0x0000000040ccbb20]
> 	at java.net.PlainSocketImpl.socketAccept(Native Method)
> 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> 	- locked <0x00007fb1ff230458> (a java.net.SocksSocketImpl)
> 	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> 	at java.net.ServerSocket.accept(ServerSocket.java:421)
> 	at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:340)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "RMI TCP Accept-0" daemon prio=1 tid=0x00007fb1f3fb5c70 nid=0x289f
> runnable [0x0000000040bca000..0x0000000040bcaba0]
> 	at java.net.PlainSocketImpl.socketAccept(Native Method)
> 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> 	- locked <0x00007fb1ff2306f8> (a java.net.SocksSocketImpl)
> 	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> 	at java.net.ServerSocket.accept(ServerSocket.java:421)
> 	at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:340)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> "Timer-0" daemon prio=1 tid=0x00007fb1f3f9c470 nid=0x289e in
> Object.wait() [0x0000000040ac9000..0x0000000040ac9c20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb1ff230990> (a java.util.TaskQueue)
> 	at java.lang.Object.wait(Object.java:474)
> 	at java.util.TimerThread.mainLoop(Timer.java:483)
> 	- locked <0x00007fb1ff230990> (a java.util.TaskQueue)
> 	at java.util.TimerThread.run(Timer.java:462)
> 
> "Low Memory Detector" daemon prio=1 tid=0x00007fb1f41b4b30 nid=0x289d
> runnable [0x0000000000000000..0x0000000000000000]
> 
> "CompilerThread1" daemon prio=1 tid=0x00007fb1f41b2f90 nid=0x289c
> waiting on condition [0x0000000000000000..0x00000000408c6630]
> 
> "CompilerThread0" daemon prio=1 tid=0x00007fb1f41b1a70 nid=0x289b
> waiting on condition [0x0000000000000000..0x00000000407c56b0]
> 
> "AdapterThread" daemon prio=1 tid=0x00007fb1f41b03e0 nid=0x289a waiting
> on condition [0x0000000000000000..0x0000000000000000]
> 
> "Signal Dispatcher" daemon prio=1 tid=0x00007fb1f41af0e0 nid=0x2899
> waiting on condition [0x0000000000000000..0x0000000000000000]
> 
> "Finalizer" daemon prio=1 tid=0x00000000401b4210 nid=0x2898 in
> Object.wait() [0x00000000404c3000..0x00000000404c3b20]
> 	at java.lang.Object.wait(Native Method)
> 	- waiting on <0x00007fb1ff230d10> (a java.lang.ref.ReferenceQueue$Lock)
> 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
> 	- locked <0x00007fb1ff230d10> (a java.lang.ref.ReferenceQueue$Lock)
> 	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:136)
> 	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
> 
> "Reference Handler" daemon prio=1 tid=0x00000000401b39d0 nid=0x2897 in
> Object.wait() [0x00000000403c2000..0x00000000403c2ba0]
> 	at java.lang.Object.wait(Native Method)
> 	at java.lang.Object.wait(Object.java:474)
> 	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> 	- locked <0x00007fb1ff230de0> (a java.lang.ref.Reference$Lock)
> 
> "main" prio=1 tid=0x0000000040116760 nid=0x2894 runnable
> [0x00007fffffff8000..0x00007fffffff9230]
> 	at java.net.PlainSocketImpl.socketAccept(Native Method)
> 	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> 	- locked <0x00007fb20e2cce20> (a java.net.SocksSocketImpl)
> 	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
> 	at java.net.ServerSocket.accept(ServerSocket.java:421)
> 	at
> org.apache.catalina.core.StandardServer.await(StandardServer.java:379)
> 	at org.apache.catalina.startup.Catalina.await(Catalina.java:616)
> 	at org.apache.catalina.startup.Catalina.start(Catalina.java:576)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:592)
> 	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
> 	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433)
> 
> "VM Thread" prio=1 tid=0x00000000401af4f0 nid=0x2896 runnable 
> 
> "VM Periodic Task Thread" prio=1 tid=0x00007fb1f4275030 nid=0x28a2
> waiting on condition 
> 
> 2010-08-16 20:08:34,458  INFO IndexManager
> org.sakaiproject.search.journal.impl.MergeUpdateOperation - Local Merge
> Operation 
> 
> 2010-08-16 20:09:01,655  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.PRETTY_HTML' found, using default
> value true
> 2010-08-16 20:09:01,655  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.ALLOW_JAVASCRIPT' found, using
> default value true
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.READONLY_AS_DISABLED_FOR_SELECTS'
> found, using default value true
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.DETECT_JAVASCRIPT' found, using
> default value false
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.AUTO_SCROLL' found, using default
> value false
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.ADD_RESOURCE_CLASS' found, using
> default value org.apache.myfaces.renderkit.html.util.DefaultAddResource
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.RESOURCE_VIRTUAL_PATH' found, using
> default value /faces/myFacesExtensionResource
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
> init parameter 'org.apache.myfaces.CHECK_EXTENSIONS_FILTER' found, using
> default value true
> 2010-08-16 20:09:01,656  INFO TP-Processor6
> org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - Starting up
> Tomahawk on the RI-JSF-Implementation.
> 
> 
> 
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
> 
> TO UNSUBSCRIBE: send email to production-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20100816/b0642264/attachment-0001.html 


More information about the production mailing list