[Building Sakai] Help with Java Mail zip file exception error post 2.9 upgrade

[Ramachandran,Narasi]

We upgraded to 2.9 last week and since then we are seeing the following issue almost every day. Have you seen others report this?
 
Here is the scenario we are seeing with the Java Mail zip file exception.  By default the Linux kernel has a per-process upper limit on the number of files that can be memory mapped. That limit on our systems is 65536. Once tomcat is restarted, the number of memory mapped files looks to be about 1200. During the course of running for a few days, this count steadily increases towards the 65k limit until it is exhausted. Once this is reached, method calls to javax.mail.* begin to fail with a java.lang.OutOfMemory exception.  Furthermore, the JVM does not recover from this scenario.  It continues to respond for several hours but eventually enters into a garbage collection loop that causes all JVM threads to stall indefinitely. Once this occurs, Nagios checks fail and we become alerted to go and restart Tomcat on the affected node.
 
Below is an example of the stack trace that is found in catalina.out:
 
Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard pages failed.
mmap failed for CEN and END part of zip file
Exception in thread "Thread-423" java.lang.OutOfMemoryError
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:214)
        at java.util.zip.ZipFile.<init>(ZipFile.java:144)
        at java.util.jar.JarFile.<init>(JarFile.java:152)
        at java.util.jar.JarFile.<init>(JarFile.java:116)
        at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:2789
)
        at org.apache.catalina.loader.WebappClassLoader.findResources(WebappClassLoader.java
:1328)
        at java.lang.ClassLoader.getResources(ClassLoader.java:1176)
        at javax.mail.Session$5.run(Session.java:1231)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.mail.Session.getResources(Session.java:1225)
        at javax.mail.Session.loadAllResources(Session.java:1137)
        at javax.mail.Session.loadProviders(Session.java:902)
        at javax.mail.Session.<init>(Session.java:219)
        at javax.mail.Session.getInstance(Session.java:258)
        at org.sakaiproject.email.impl.BasicEmailService.sendToUsers(BasicEmailService.java:
767)
        at org.sakaiproject.util.EmailNotification$RunSendToUsers.run(EmailNotification.java
:186)
        at java.lang.Thread.run(Thread.java:722)
 
Any advice with regards to this will eb great.

Thanks, 
Narasi
Univ of Florida