[Building Sakai] Announcements: java.lang.OutOfMemoryError
Yzelle, Sonette
SYzelle at unisa.ac.za
Thu Jul 15 22:53:53 PDT 2010
This message (and attachments) is subject to restrictions and a disclaimer. Please refer to http://www.unisa.ac.za/disclaimer for full details.
Thank you Mustansar & Stephen for your replies.
Stephen, apparently we have had announcements going out to the 15000 students successfully on Sakai 2.6.2.
Can the size of the message have an influence that the JVM runs out of threads, I don't think so?
However the previous announcements that was successful did not have any attachments.
My question is, do you still think it might be the LDAP connection that is causing the problem even if there was successful announcements to 15000 students (without attachments)?
I just want to make sure before we start investigating our LDAP connection as we never experienced any problems with LDAP before.
Regards
Sonette
Sonette Yzelle
Analyst Developer
ICT
UNISA
South Africa
email: syzelle at unisa.ac.za
-----Original Message-----
From: sakai-dev-bounces at collab.sakaiproject.org [mailto:sakai-dev-bounces at collab.sakaiproject.org] On Behalf Of Stephen Marquard
Sent: 15 July 2010 04:32 PM
To: sakai-dev at collab.sakaiproject.org
Subject: Re: [Building Sakai] Announcements: java.lang.OutOfMemoryError
>From the stack trace, I'd say:
- Announcements is iterating through the list of 15000 users
- The UNISA user directory provider is doing user lookups for each via LDAP (at a guess)
- Lots of LDAP request threads are getting generated and not closed quickly enough, so the jvm runs out of threads
I'd suggest triggering an announcement to a large group and do a kill -3 while it's busy to get a thread dump. You can probably increase the memory and/or total number of permitted threads, but the real solution is probably to optimize how the user directory provider works (LDAP connection pooling or something).
Regards
Stephen
>>> Mustansar Mehmood <mustansar at rice.edu> 7/15/2010 4:07 PM >>>
As far as memory leak is concerned I think tomcat 5.5.29 covers a lot of
ground there.
in terms of adding more RAM is concerned it depends on your concuurent
user count and how much RAM you have at the moment
I hope some one from announcement team can pitch to help as they might
know does announce tool works in terms of creating separate threads when
multicasting the announcements. I have CC'ed the project lead on
announcements.
./mustansar
On 07/15/2010 08:36 AM, Mustansar Mehmood wrote:
> What version of Sakai, Java and tomcat are you running ?
> On 07/15/2010 08:22 AM, Yzelle, Sonette wrote:
>> This message (and attachments) is subject to restrictions and a
>> disclaimer. Please refer to http://www.unisa.ac.za/disclaimer for
>> full details.
>> ------------------------------------------------------------------------
>>
>> Good day,
>>
>> When we try to send out an announcement to 15000 students with 3
>> attachments we get the error below?
>>
>> Has anyone else experienced something similar with announcements?
>>
>> We have googled and see that it is VM that is not enough and that we
>> might have a memory leak?
>>
>> When we send announcements to smaller groups (with attachments) of
>> students there is no error.
>>
>> We would appreciate your input on whether we must increase our VM
>> memory or if you think there might be a problem with the way threads
>> are opened and closed?
>>
>> Thanking you in advance.
>>
>> Regards
>>
>> Sonette
>>
>> This information will automatically be included in your bug report.
>>
>> org.sakaiproject.portal.api.PortalHandlerException:
>> org.sakaiproject.tool.api.ToolException
>>
>> at
>> org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:768)
>>
>> caused by: org.sakaiproject.tool.api.ToolException
>>
>> at org.sakaiproject.cheftool.ToolServlet.doGet(ToolServlet.java:227)
>>
>> caused by: java.lang.reflect.InvocationTargetException
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> caused by: java.lang.OutOfMemoryError: unable to create new native thread
>>
>> at java.lang.Thread.start0(Native Method)
>>
>> at java.lang.Thread.start(Thread.java:574)
>>
>> at com.sun.jndi.ldap.Connection.<init>(Connection.java:212)
>>
>> at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:118)
>>
>> at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1578)
>>
>> at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2605)
>>
>> at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:284)
>>
>> at
>> com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
>>
>> at
>> com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
>>
>> at
>> com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
>>
>> at
>> com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
>>
>> at
>> javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
>>
>> at
>> javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:247)
>>
>> at javax.naming.InitialContext.init(InitialContext.java:223)
>>
>> at
>> javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:134)
>>
>> at
>> za.ac.unisa.lms.providers.user.ad.StudentDirectoryProvider.getGCCtx(StudentDirectoryProvider.java:349)
>>
>> at
>> za.ac.unisa.lms.providers.user.ad.ADDirectoryProvider.findUser(ADDirectoryProvider.java:234)
>>
>> at
>> za.ac.unisa.lms.providers.user.ad.ADDirectoryProvider.findUserDn(ADDirectoryProvider.java:299)
>>
>> at
>> za.ac.unisa.lms.providers.user.ad.ADDirectoryProvider.userExists(ADDirectoryProvider.java:461)
>>
>> at
>> za.ac.unisa.lms.providers.user.UNISADirectoryProvider.getDisplayName(UNISADirectoryProvider.java:214)
>>
>> at
>> org.sakaiproject.user.impl.BaseUserDirectoryService$BaseUserEdit.getDisplayName(BaseUserDirectoryService.java:2179)
>>
>> at
>> org.sakaiproject.announcement.tool.AnnouncementAction$AnnouncementWrapper.<init>(AnnouncementAction.java:427)
>>
>> at
>> org.sakaiproject.announcement.tool.AnnouncementAction$AnnouncementWrapper.wrapList(AnnouncementAction.java:676)
>>
>> at
>> org.sakaiproject.announcement.tool.AnnouncementAction$AnnouncementWrapper.access$400(AnnouncementAction.java:355)
>>
>> at
>> org.sakaiproject.announcement.tool.AnnouncementAction.getMessages(AnnouncementAction.java:1470)
>>
>> at
>> org.sakaiproject.announcement.tool.AnnouncementAction.buildMainPanelContext(AnnouncementAction.java:906)
>>
>> at sun.reflect.GeneratedMethodAccessor1014.invoke(null:-1)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:585)
>>
>> at
>> org.sakaiproject.cheftool.VelocityPortletPaneledAction.toolModeDispatch(VelocityPortletPaneledAction.java:401)
>>
>> at org.sakaiproject.cheftool.ToolServlet.doGet(ToolServlet.java:227)
>>
>> at
>> org.sakaiproject.cheftool.VelocityPortletPaneledAction.doGet(VelocityPortletPaneledAction.java:1014)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>>
>> at
>> org.sakaiproject.vm.ComponentServlet.service(ComponentServlet.java:56)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>
>> at
>> org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:565)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:679)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:461)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:364)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
>>
>> at
>> org.sakaiproject.tool.impl.ActiveToolComponent$MyActiveTool.forward(ActiveToolComponent.java:486)
>>
>> at
>> org.sakaiproject.portal.charon.SkinnableCharonPortal.forwardTool(SkinnableCharonPortal.java:1216)
>>
>> at
>> org.sakaiproject.portal.charon.handlers.ToolHandler.doTool(ToolHandler.java:188)
>>
>> at
>> org.sakaiproject.portal.charon.handlers.ToolHandler.doGet(ToolHandler.java:93)
>>
>> at
>> org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:768)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>>
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>
>> at
>> org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:616)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
>>
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
>>
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
>>
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>
>> at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
>>
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
>>
>> at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
>>
>> at
>> org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
>>
>> at
>> org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
>>
>> at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
>>
>> at
>> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
>>
>> at
>> org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
>>
>> at
>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
>>
>> at java.lang.Thread.run(Thread.java:595)
>>
>> user: bfab32cb-2ab9-46c6-80e4-ee7664cbabf4
>>
>> usage-session: a5ae0727-081d-44a4-a1a4-a7faf90c683f
>>
>> time: 12-Jul-2010 08:37:54
>>
>> *Klarissa Engelbrecht*
>>
>> *Module Leader EUP1501*
>>
>> End-user Computing Practical Tuition Team
>>
>> School of Computing
>>
>> College of Science, Engineering and Technology
>>
>> Tel.: +27 12 429 6878/6653
>>
>> Unisa-email-signature
>>
>>
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email tosakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
> --
> Mustansar Mehmood
> Educational System Developer& Integrator
>
> Information Technology
> 6100 Main St. MS 119
> Houston Texas 77005
>
> Phone:(713)348-2523
> Fax :(713)348 6099
> email:mustansar at rice.edu
>
>
>
>
>
> I have yet to see any problem, however complicated, which, when,you looked at it in the right way, did not become still more complicated.
> -- Poul Anderson
>
>
>
>
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
--
Mustansar Mehmood
Educational System Developer& Integrator
Information Technology
6100 Main St. MS 119
Houston Texas 77005
Phone:(713)348-2523
Fax :(713)348 6099
email:mustansar at rice.edu
I have yet to see any problem, however complicated, which, when,you looked at it in the right way, did not become still more complicated.
-- Poul Anderson
###
UNIVERSITY OF CAPE TOWN
This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 4500. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity.
###
_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
More information about the sakai-dev
mailing list