[Deploying Sakai] Profile2 issue [was: Problems with deprecated tools after upgrade]

Mon Aug 16 01:15:22 PDT 2010

Hi Steve.

> What version Profile2 are you running? And how many users might you have in your system?

The latest one, 1.3.10. Had the same behaviour with 1.3.9 though. There
was no earlier version of Profile2 involved, just Profile1. We have a
few hundred users...

> 
> Can you try a thread dump? This happens when one request takes a long time and block another incoming request so it's getting stuck somewhere processing in the backend.
> 
> Here's a couple of discussions about it:
> http://apache-wicket.1842946.n4.nabble.com/After-1-minute-the-Pagemap-null-is-still-locked-td1890832.html
> http://apache-wicket.1842946.n4.nabble.com/pagemap-locking-td1932905.html
> 
Thanks for that. The thread dump is pasted below...

Cheers,
Markus

2010-08-16 20:07:50,517  INFO TP-Processor5
org.sakaiproject.profile2.logic.ProfileLogicImpl - Created default
preferences record for user: 7a41c257-1336-4e1b-809a-e5bec01877f3
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.5.0_22-b03 mixed
mode):

"TP-Processor12" daemon prio=1 tid=0x00007fb1f3298dd0 nid=0x2924
runnable [0x0000000045715000..0x0000000045715ca0]
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
	- locked <0x00007fb20e9329b8> (a java.io.BufferedInputStream)
	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
	at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
	at org.apache.jk.common.ChannelSocket
$SocketConnection.runIt(ChannelSocket.java:897)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor11" daemon prio=1 tid=0x00007fb1f295d4f0 nid=0x2923 in
Object.wait() [0x0000000045614000..0x0000000045614d20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e77f850> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e77f850> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor10" daemon prio=1 tid=0x00007fb1ef05cb50 nid=0x2922 in
Object.wait() [0x0000000045513000..0x0000000045513da0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e77f4a8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e77f4a8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor9" daemon prio=1 tid=0x00007fb1f24162a0 nid=0x2921 in
Object.wait() [0x0000000045412000..0x0000000045412e20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e77f030> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e77f030> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"PageSavingThread-sakai.profile2" daemon prio=1 tid=0x00007fb1e5349c40
nid=0x2908 sleeping[0x0000000045311000..0x0000000045311aa0]
	at java.lang.Thread.sleep(Native Method)
	at org.apache.wicket.protocol.http.pagestore.DiskPageStore
$PageSavingThread.run(DiskPageStore.java:947)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor8" daemon prio=1 tid=0x00007fb1f1a1c490 nid=0x2905 runnable
[0x0000000045210000..0x0000000045210b20]
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
	- locked <0x00007fb1f494ed20> (a java.io.BufferedInputStream)
	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
	at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
	at org.apache.jk.common.ChannelSocket
$SocketConnection.runIt(ChannelSocket.java:897)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor7" daemon prio=1 tid=0x00007fb1f1a1b970 nid=0x2904 runnable
[0x000000004510f000..0x000000004510fba0]
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
	- locked <0x00007fb20e8eb970> (a java.io.BufferedInputStream)
	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
	at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
	at org.apache.jk.common.ChannelSocket
$SocketConnection.runIt(ChannelSocket.java:897)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor6" daemon prio=1 tid=0x00007fb1f1a1ae50 nid=0x2903 runnable
[0x000000004500e000..0x000000004500ec20]
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
	- locked <0x00007fb20e8ee728> (a java.io.BufferedInputStream)
	at org.apache.jk.common.ChannelSocket.read(ChannelSocket.java:627)
	at org.apache.jk.common.ChannelSocket.receive(ChannelSocket.java:565)
	at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:692)
	at org.apache.jk.common.ChannelSocket
$SocketConnection.runIt(ChannelSocket.java:897)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor5" daemon prio=1 tid=0x00007fb1f27c1400 nid=0x2902 runnable
[0x0000000044f0a000..0x0000000044f0dca0]
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at
com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:114)
	at
com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:161)
	at
com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:189)
	- locked <0x00007fb20e9991c0> (a
com.mysql.jdbc.util.ReadAheadInputStream)
	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2499)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2952)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2941)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3489)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2113)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
	- locked <0x00007fb20e995008> (a java.lang.Object)
	at
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2113)
	at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2409)
	- locked <0x00007fb20e995008> (a java.lang.Object)
	at
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1974)
	at
com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1451)
	- locked <0x00007fb20e995008> (a java.lang.Object)
	at
org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:297)
	at
org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48)
	at
org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:247)
	at
org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266)
	at
org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)
	at
org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
	at
org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
	at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1001)
	at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:339)
	at
org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)
	at
org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:655)
	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
	at
org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
	at
org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
	at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
	at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy29.createDefaultPreferencesRecord(Unknown Source)
	at
org.sakaiproject.profile2.tool.pages.MyPreferences.<init>(MyPreferences.java:52)
	at org.sakaiproject.profile2.tool.pages.BasePage
$5.onClick(BasePage.java:126)
	at org.apache.wicket.markup.html.link.Link.onLinkClicked(Link.java:214)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:592)
	at
org.apache.wicket.RequestListenerInterface.invoke(RequestListenerInterface.java:183)
	at
org.apache.wicket.request.target.component.listener.ListenerInterfaceRequestTarget.processEvents(ListenerInterfaceRequestTarget.java:73)
	at
org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:91)
	at
org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1220)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1297)
	at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1399)
	at org.apache.wicket.RequestCycle.request(RequestCycle.java:529)
	at
org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:356)
	at
org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:124)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:585)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at
org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:659)
	at
org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:457)
	at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:359)
	at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:311)
	at org.sakaiproject.tool.impl.ActiveToolComponent
$MyActiveTool.forward(ActiveToolComponent.java:500)
	at
org.sakaiproject.portal.charon.SkinnableCharonPortal.forwardTool(SkinnableCharonPortal.java:1221)
	at
org.sakaiproject.portal.charon.handlers.ToolHandler.doTool(ToolHandler.java:204)
	at
org.sakaiproject.portal.charon.handlers.ToolHandler.doGet(ToolHandler.java:96)
	at
org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:768)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:627)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:646)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
	at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
	at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
	at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
	at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
	at
org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
	at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
	at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775)
	at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704)
	at org.apache.jk.common.ChannelSocket
$SocketConnection.runIt(ChannelSocket.java:897)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"TP-Monitor" daemon prio=1 tid=0x00007fb1eaaa9bd0 nid=0x28f9 in
Object.wait() [0x0000000044e0c000..0x0000000044e0cd20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e285d10> (a
org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
	at org.apache.tomcat.util.threads.ThreadPool
$MonitorRunnable.run(ThreadPool.java:564)
	- locked <0x00007fb20e285d10> (a
org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor4" daemon prio=1 tid=0x00007fb1ebade280 nid=0x28f8 runnable
[0x0000000044d0b000..0x0000000044d0bda0]
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x00007fb20e283138> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:313)
	at
org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:667)
	at org.apache.jk.common.ChannelSocket
$SocketAcceptor.runIt(ChannelSocket.java:878)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor3" daemon prio=1 tid=0x00007fb1eaa60b30 nid=0x28f7 in
Object.wait() [0x0000000044c0a000..0x0000000044c0ae20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e2854f0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e2854f0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor2" daemon prio=1 tid=0x00007fb1e9ef6460 nid=0x28f6 in
Object.wait() [0x0000000044b09000..0x0000000044b09aa0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e2850e0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e2850e0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"TP-Processor1" daemon prio=1 tid=0x00007fb1e9e858f0 nid=0x28f5 in
Object.wait() [0x0000000044a08000..0x0000000044a08b20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e284cd0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e284cd0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Monitor" prio=1 tid=0x00007fb1e9ed1ba0 nid=0x28f4 in
Object.wait() [0x0000000044907000..0x0000000044907ba0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14deb0> (a
org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
	at org.apache.tomcat.util.threads.ThreadPool
$MonitorRunnable.run(ThreadPool.java:564)
	- locked <0x00007fb20e14deb0> (a
org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor25" daemon prio=1 tid=0x00007fb1e9ed0b60 nid=0x28f3
runnable [0x0000000044806000..0x0000000044806c20]
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x00007fb1ff4146d8> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at
org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
	at
org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java:408)
	at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:71)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor24" daemon prio=1 tid=0x00007fb1e9ebe570 nid=0x28f2
in Object.wait() [0x0000000044705000..0x0000000044705ca0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14d670> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14d670> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor23" daemon prio=1 tid=0x00007fb1ea022690 nid=0x28f1
in Object.wait() [0x0000000044604000..0x0000000044604d20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14d250> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14d250> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor22" daemon prio=1 tid=0x00007fb1d83d9de0 nid=0x28f0
in Object.wait() [0x0000000044503000..0x0000000044503da0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14ce30> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14ce30> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor21" daemon prio=1 tid=0x00007fb1ea021ff0 nid=0x28ef
in Object.wait() [0x0000000044402000..0x0000000044402e20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14ca10> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14ca10> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor20" daemon prio=1 tid=0x00007fb1ea021040 nid=0x28ee
in Object.wait() [0x0000000044301000..0x0000000044301aa0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14c5f0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14c5f0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor19" daemon prio=1 tid=0x00007fb1ea845550 nid=0x28ed
in Object.wait() [0x0000000044200000..0x0000000044200b20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14c1d0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14c1d0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor18" daemon prio=1 tid=0x00007fb1eaa8e620 nid=0x28ec
in Object.wait() [0x00000000440ff000..0x00000000440ffba0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14bc20> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14bc20> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor17" daemon prio=1 tid=0x00007fb1eaa8d620 nid=0x28eb
in Object.wait() [0x0000000043ffe000..0x0000000043ffec20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14b800> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14b800> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor16" daemon prio=1 tid=0x00007fb1ea529cd0 nid=0x28ea
in Object.wait() [0x0000000043efd000..0x0000000043efdca0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14b3e0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14b3e0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor15" daemon prio=1 tid=0x00007fb1ea529a20 nid=0x28e9
in Object.wait() [0x0000000043dfc000..0x0000000043dfcd20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14afc0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14afc0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor14" daemon prio=1 tid=0x00007fb1e9e57530 nid=0x28e8
in Object.wait() [0x0000000043cfb000..0x0000000043cfbda0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14aba0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14aba0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor13" daemon prio=1 tid=0x00007fb1e9e56620 nid=0x28e7
in Object.wait() [0x0000000043bfa000..0x0000000043bfae20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14a780> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14a780> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor12" daemon prio=1 tid=0x00007fb1e9e84730 nid=0x28e6
in Object.wait() [0x0000000043af9000..0x0000000043af9aa0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e14a360> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e14a360> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor11" daemon prio=1 tid=0x00007fb1eb3061b0 nid=0x28e5
in Object.wait() [0x00000000439f8000..0x00000000439f8b20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e149f40> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e149f40> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor10" daemon prio=1 tid=0x00007fb1eac56350 nid=0x28e4
in Object.wait() [0x00000000438f7000..0x00000000438f7ba0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e149b20> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e149b20> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor9" daemon prio=1 tid=0x00007fb1eb1e69e0 nid=0x28e3
in Object.wait() [0x00000000437f6000..0x00000000437f6c20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e149708> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e149708> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor8" daemon prio=1 tid=0x00007fb1ea970520 nid=0x28e2
in Object.wait() [0x00000000436f5000..0x00000000436f5ca0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e1492f0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e1492f0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor7" daemon prio=1 tid=0x00007fb1ea56f630 nid=0x28e1
in Object.wait() [0x00000000435f4000..0x00000000435f4d20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e148ed8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e148ed8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor6" daemon prio=1 tid=0x00007fb1ea6ef4a0 nid=0x28e0
in Object.wait() [0x00000000434f3000..0x00000000434f3da0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e148ac0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e148ac0> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor5" daemon prio=1 tid=0x00007fb1e9eb03c0 nid=0x28df
in Object.wait() [0x00000000433f2000..0x00000000433f2e20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e1486a8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e1486a8> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor4" daemon prio=1 tid=0x00007fb1ebe56c50 nid=0x28de
in Object.wait() [0x00000000432f1000..0x00000000432f1aa0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e148290> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e148290> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor3" daemon prio=1 tid=0x00007fb1ebdd88a0 nid=0x28dd
in Object.wait() [0x00000000431f0000..0x00000000431f0b20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e147e78> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e147e78> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor2" daemon prio=1 tid=0x00007fb1ea937f10 nid=0x28dc
in Object.wait() [0x00000000430ef000..0x00000000430efba0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e147a60> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e147a60> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"http-8080-Processor1" daemon prio=1 tid=0x00007fb1eaa978c0 nid=0x28db
in Object.wait() [0x0000000042fee000..0x0000000042feec20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20e147648> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Object.wait(Object.java:474)
	at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
	- locked <0x00007fb20e147648> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
	at java.lang.Thread.run(Thread.java:595)

"ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=1
tid=0x00007fb1d8683290 nid=0x28da waiting on condition
[0x0000000042eed000..0x0000000042eedca0]
	at java.lang.Thread.sleep(Native Method)
	at org.apache.catalina.core.ContainerBase
$ContainerBackgroundProcessor.run(ContainerBase.java:1548)
	at java.lang.Thread.run(Thread.java:595)

"Timer-3" daemon prio=1 tid=0x00007fb1e733fc60 nid=0x28cc in
Object.wait() [0x0000000042dec000..0x0000000042decaa0]
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0x00007fb205436a38> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"Store uk.ac.cam.caret.sakai.rwiki.service.api.radeox.RenderCache Spool
Thread" daemon prio=1 tid=0x00000000401a8690 nid=0x28c9
sleeping[0x0000000042ceb000..0x0000000042cebb20]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store
org.sakaiproject.api.privacy.PrivacyManager.PrivacyQueryCache.queryGetPrivacy Spool Thread" daemon prio=1 tid=0x00007fb1f3a99090 nid=0x28c8 sleeping[0x0000000042bea000..0x0000000042beaba0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"SakaiPresenceService.Maintenance" daemon prio=1 tid=0x00007fb1f2b745f0
nid=0x28c7 waiting on condition [0x0000000042ae9000..0x0000000042ae9c20]
	at java.lang.Thread.sleep(Native Method)
	at org.sakaiproject.presence.impl.BasePresenceService
$Maintenance.run(BasePresenceService.java:791)
	at java.lang.Thread.run(Thread.java:595)

"Store org.sakaiproject.user.api.AuthenticationManager Spool Thread"
daemon prio=1 tid=0x00007fb1f33c7980 nid=0x28c4
sleeping[0x00000000429e8000..0x00000000429e8ca0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Timer-2" daemon prio=1 tid=0x00007fb1f2732720 nid=0x28c3 in
Object.wait() [0x00000000428e7000..0x00000000428e7d20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20286c450> (a java.util.TaskQueue)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0x00007fb20286c450> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"Store
org.sakaiproject.citation.api.SearchManager.metasearchSessionManagerCache Spool Thread" daemon prio=1 tid=0x00007fb1f3be40d0 nid=0x28c2 sleeping[0x00000000427e6000..0x00000000427e6da0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.citation.api.SearchManager.sessionContextCache
Spool Thread" daemon prio=1 tid=0x00007fb1e5c9dac0 nid=0x28c1
sleeping[0x00000000426e5000..0x00000000426e5e20]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.theospi.portfolio.wizard.mgt.impl.WizardManagerImpl.cache
Spool Thread" daemon prio=1 tid=0x00007fb1f1b13fb0 nid=0x28c0
sleeping[0x00000000425e4000..0x00000000425e4aa0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"SakaiClusterService.Maintenance" daemon prio=1 tid=0x00007fb1ea0bba70
nid=0x28bf waiting on condition [0x00000000424e3000..0x00000000424e3b20]
	at java.lang.Thread.sleep(Native Method)
	at org.sakaiproject.cluster.impl.SakaiClusterService
$Maintenance.run(SakaiClusterService.java:469)
	at java.lang.Thread.run(Thread.java:595)

"IndexManager" daemon prio=1 tid=0x00007fb1e9d26950 nid=0x28be in
Object.wait() [0x00000000423e2000..0x00000000423e2ba0]
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0x00007fb20257a780> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"org.sakaiproject.sitestats.impl.StatsUpdateManagerImpl" prio=1
tid=0x00007fb1ea132790 nid=0x28bd in Object.wait()
[0x00000000422e1000..0x00000000422e1c20]
	at java.lang.Object.wait(Native Method)
	at
org.sakaiproject.sitestats.impl.StatsUpdateManagerImpl.run(StatsUpdateManagerImpl.java:485)
	- locked <0x00007fb20257a878> (a java.lang.Object)
	at java.lang.Thread.run(Thread.java:595)

"org.azeckoski.reflectutils.refmap.Finalizer" daemon prio=1
tid=0x00007fb1f29da430 nid=0x28ba in Object.wait()
[0x00000000421e0000..0x00000000421e0ca0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb20220c940> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
	- locked <0x00007fb20220c940> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:136)
	at org.azeckoski.reflectutils.refmap.Finalizer.run(Finalizer.java:127)

"QuartzScheduler_QuartzScheduler-vre_MisfireHandler" prio=1
tid=0x00007fb1e5ea5ef0 nid=0x28b9 waiting on condition
[0x00000000420df000..0x00000000420dfd20]
	at java.lang.Thread.sleep(Native Method)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport
$MisfireHandler.run(JobStoreSupport.java:3914)

"QuartzScheduler_QuartzSchedulerThread" prio=1 tid=0x00007fb1f1a63c70
nid=0x28b8 in Object.wait() [0x0000000041fde000..0x0000000041fdeda0]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb2021e2550> (a java.lang.Object)
	at
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:433)
	- locked <0x00007fb2021e2550> (a java.lang.Object)

"QuartzScheduler_Worker-5" prio=1 tid=0x00007fb1f1393ab0 nid=0x28b7 in
Object.wait() [0x0000000041edd000..0x0000000041edde20]
	at java.lang.Object.wait(Native Method)
	at org.quartz.simpl.SimpleThreadPool
$WorkerThread.run(SimpleThreadPool.java:528)
	- locked <0x00007fb2021e0f38> (a org.quartz.simpl.SimpleThreadPool
$WorkerThread)

"QuartzScheduler_Worker-4" prio=1 tid=0x00007fb1f1393410 nid=0x28b6 in
Object.wait() [0x0000000041ddc000..0x0000000041ddcaa0]
	at java.lang.Object.wait(Native Method)
	at org.quartz.simpl.SimpleThreadPool
$WorkerThread.run(SimpleThreadPool.java:528)
	- locked <0x00007fb2021e0c98> (a org.quartz.simpl.SimpleThreadPool
$WorkerThread)

"QuartzScheduler_Worker-3" prio=1 tid=0x00007fb1e5ea72f0 nid=0x28b5 in
Object.wait() [0x0000000041cdb000..0x0000000041cdbb20]
	at java.lang.Object.wait(Native Method)
	at org.quartz.simpl.SimpleThreadPool
$WorkerThread.run(SimpleThreadPool.java:528)
	- locked <0x00007fb2021e09f8> (a org.quartz.simpl.SimpleThreadPool
$WorkerThread)

"QuartzScheduler_Worker-2" prio=1 tid=0x00007fb1f18ea210 nid=0x28b4 in
Object.wait() [0x0000000041bda000..0x0000000041bdaba0]
	at java.lang.Object.wait(Native Method)
	at org.quartz.simpl.SimpleThreadPool
$WorkerThread.run(SimpleThreadPool.java:528)
	- locked <0x00007fb2021e0758> (a org.quartz.simpl.SimpleThreadPool
$WorkerThread)

"QuartzScheduler_Worker-1" prio=1 tid=0x00007fb1f2161e70 nid=0x28b3 in
Object.wait() [0x0000000041ad9000..0x0000000041ad9c20]
	at java.lang.Object.wait(Native Method)
	at org.quartz.simpl.SimpleThreadPool
$WorkerThread.run(SimpleThreadPool.java:528)
	- locked <0x00007fb2021e04b8> (a org.quartz.simpl.SimpleThreadPool
$WorkerThread)

"Store org.sakaiproject.springframework.orm.hibernate.L2Cache Spool
Thread" daemon prio=1 tid=0x00007fb1f2f9d690 nid=0x28b0
sleeping[0x00000000419d8000..0x00000000419d8ca0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.event.api.UsageSessionService.recentUserRefresh
Spool Thread" daemon prio=1 tid=0x00007fb1e922a030 nid=0x28af
sleeping[0x00000000418d7000..0x00000000418d7d20]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"org.sakaiproject.event.impl.ClusterEventTracking$$EnhancerByCGLIB
$$33ba687" daemon prio=1 tid=0x00007fb1e9919630 nid=0x28ae waiting on
condition [0x00000000417d6000..0x00000000417d6da0]
	at java.lang.Thread.sleep(Native Method)
	at
org.sakaiproject.event.impl.ClusterEventTracking.run(ClusterEventTracking.java:592)
	at java.lang.Thread.run(Thread.java:595)

"Timer-1" daemon prio=1 tid=0x00007fb1e89be080 nid=0x28ad in
Object.wait() [0x00000000416d5000..0x00000000416d5e20]
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0x00007fb1fff11220> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_USER_PROPERTY Spool
Thread" daemon prio=1 tid=0x00007fb1e922b7a0 nid=0x28ac
sleeping[0x00000000415d4000..0x00000000415d4aa0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_GROUP_PROPERTY
Spool Thread" daemon prio=1 tid=0x00007fb1e922b490 nid=0x28ab
sleeping[0x00000000414d3000..0x00000000414d3b20]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_ALIAS_PROPERTY Spool
Thread" daemon prio=1 tid=0x00007fb1e922ac10 nid=0x28aa
sleeping[0x00000000413d2000..0x00000000413d2ba0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_REALM_PROPERTY Spool
Thread" daemon prio=1 tid=0x00007fb1e922a450 nid=0x28a9
sleeping[0x00000000412d1000..0x00000000412d1c20]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PAGE_PROPERTY
Spool Thread" daemon prio=1 tid=0x00007fb1e9c572d0 nid=0x28a8
sleeping[0x00000000411d0000..0x00000000411d0ca0]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Store org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PROPERTY Spool
Thread" daemon prio=1 tid=0x00007fb1e9c55790 nid=0x28a7
sleeping[0x00000000410cf000..0x00000000410cfd20]
	at java.lang.Thread.sleep(Native Method)
	at
net.sf.ehcache.store.DiskStore.spoolAndExpiryThreadMain(DiskStore.java:637)
	at net.sf.ehcache.store.DiskStore.access$900(DiskStore.java:67)
	at net.sf.ehcache.store.DiskStore
$SpoolAndExpiryThread.run(DiskStore.java:1104)

"Sakai.SessionComponent.Maintenance" daemon prio=1
tid=0x00007fb1e9c554c0 nid=0x28a6 waiting on condition
[0x0000000040fce000..0x0000000040fceda0]
	at java.lang.Thread.sleep(Native Method)
	at org.sakaiproject.tool.impl.SessionComponent
$Maintenance.run(SessionComponent.java:542)
	at java.lang.Thread.run(Thread.java:595)

"RMI TCP Accept-0" daemon prio=1 tid=0x00007fb1f4274020 nid=0x28a1
runnable [0x0000000040dcc000..0x0000000040dccaa0]
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x00007fb1ff2301b0> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at sun.management.jmxremote.LocalRMIServerSocketFactory
$1.accept(LocalRMIServerSocketFactory.java:31)
	at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:340)
	at java.lang.Thread.run(Thread.java:595)

"RMI TCP Accept-1234" daemon prio=1 tid=0x00007fb1f4272b40 nid=0x28a0
runnable [0x0000000040ccb000..0x0000000040ccbb20]
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x00007fb1ff230458> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:340)
	at java.lang.Thread.run(Thread.java:595)

"RMI TCP Accept-0" daemon prio=1 tid=0x00007fb1f3fb5c70 nid=0x289f
runnable [0x0000000040bca000..0x0000000040bcaba0]
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x00007fb1ff2306f8> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:340)
	at java.lang.Thread.run(Thread.java:595)

"Timer-0" daemon prio=1 tid=0x00007fb1f3f9c470 nid=0x289e in
Object.wait() [0x0000000040ac9000..0x0000000040ac9c20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb1ff230990> (a java.util.TaskQueue)
	at java.lang.Object.wait(Object.java:474)
	at java.util.TimerThread.mainLoop(Timer.java:483)
	- locked <0x00007fb1ff230990> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"Low Memory Detector" daemon prio=1 tid=0x00007fb1f41b4b30 nid=0x289d
runnable [0x0000000000000000..0x0000000000000000]

"CompilerThread1" daemon prio=1 tid=0x00007fb1f41b2f90 nid=0x289c
waiting on condition [0x0000000000000000..0x00000000408c6630]

"CompilerThread0" daemon prio=1 tid=0x00007fb1f41b1a70 nid=0x289b
waiting on condition [0x0000000000000000..0x00000000407c56b0]

"AdapterThread" daemon prio=1 tid=0x00007fb1f41b03e0 nid=0x289a waiting
on condition [0x0000000000000000..0x0000000000000000]

"Signal Dispatcher" daemon prio=1 tid=0x00007fb1f41af0e0 nid=0x2899
waiting on condition [0x0000000000000000..0x0000000000000000]

"Finalizer" daemon prio=1 tid=0x00000000401b4210 nid=0x2898 in
Object.wait() [0x00000000404c3000..0x00000000404c3b20]
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00007fb1ff230d10> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:120)
	- locked <0x00007fb1ff230d10> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:136)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=1 tid=0x00000000401b39d0 nid=0x2897 in
Object.wait() [0x00000000403c2000..0x00000000403c2ba0]
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:474)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
	- locked <0x00007fb1ff230de0> (a java.lang.ref.Reference$Lock)

"main" prio=1 tid=0x0000000040116760 nid=0x2894 runnable
[0x00007fffffff8000..0x00007fffffff9230]
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
	- locked <0x00007fb20e2cce20> (a java.net.SocksSocketImpl)
	at java.net.ServerSocket.implAccept(ServerSocket.java:450)
	at java.net.ServerSocket.accept(ServerSocket.java:421)
	at
org.apache.catalina.core.StandardServer.await(StandardServer.java:379)
	at org.apache.catalina.startup.Catalina.await(Catalina.java:616)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:576)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:592)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:295)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:433)

"VM Thread" prio=1 tid=0x00000000401af4f0 nid=0x2896 runnable 

"VM Periodic Task Thread" prio=1 tid=0x00007fb1f4275030 nid=0x28a2
waiting on condition 

2010-08-16 20:08:34,458  INFO IndexManager
org.sakaiproject.search.journal.impl.MergeUpdateOperation - Local Merge
Operation 

2010-08-16 20:09:01,655  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.PRETTY_HTML' found, using default
value true
2010-08-16 20:09:01,655  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.ALLOW_JAVASCRIPT' found, using
default value true
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.READONLY_AS_DISABLED_FOR_SELECTS'
found, using default value true
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.DETECT_JAVASCRIPT' found, using
default value false
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.AUTO_SCROLL' found, using default
value false
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.ADD_RESOURCE_CLASS' found, using
default value org.apache.myfaces.renderkit.html.util.DefaultAddResource
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.RESOURCE_VIRTUAL_PATH' found, using
default value /faces/myFacesExtensionResource
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - No context
init parameter 'org.apache.myfaces.CHECK_EXTENSIONS_FILTER' found, using
default value true
2010-08-16 20:09:01,656  INFO TP-Processor6
org.apache.myfaces.shared_tomahawk.config.MyfacesConfig - Starting up
Tomahawk on the RI-JSF-Implementation.