[Building Sakai] Java 7 and Tomcat SSL connections

[Stephen Marquard]

Hi Omer,

We have been running JDK 1.7.0_21 in production since Monday, with Tomcat 7.0.40 and the bcprov-jdk15on-149 JCE as below.

So far the garbage collection performance in 1.7 with the G1 collector is a huge improvement on JDK 1.6. We have only seen one glitch so far, which was that one of the app servers started taking much longer to do the initial SSL handshake (on the direct SSL port in Tomcat), up to around 400ms. Restarting the app server solved the issue.

Our JAVA_OPTS include:

-server -d64 -Xms5000m -Xmx5000m -XX:MaxPermSize=768m -XX:+UseG1GC -XX:MaxGCPauseMillis=3000 -verbose:gc -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Djava.awt.headless=true -Dnetworkaddress.cache.ttl=60 -Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false -Dsun.lang.ClassLoader.allowArraySyntax=true -Dhttp.agent=Sakai -Djava.net.preferIPv6Addresses=false -Djava.net.preferIPv4Stack=true -Djava.util.Arrays.useLegacyMergeSort=true

Some of the settings in there may no longer be necessary. The "java.util.Arrays.useLegacyMergeSort" setting is specifically for Java 7 (see https://jira.sakaiproject.org/browse/KNL-1083)

Regards
Stephen

________________________________
From: sakai-dev-bounces at collab.sakaiproject.org [sakai-dev-bounces at collab.sakaiproject.org] on behalf of Omer A Piperdi [omer at rice.edu]
Sent: 13 June 2013 05:03 PM
To: sakai-dev at collab.sakaiproject.org
Subject: Re: [Building Sakai] Java 7 and Tomcat SSL connections

We are currently running 2.9.x on Java 6, planning to upgrade this summer.. Any idea or suggestion on whether we should stay with latest Java 6 (e.g. JDK 1.6.45) or not? We also use direct Tomcat connection for web services calls.

Thanks
Omer

On 6/10/2013 1:38 PM, Stephen Marquard wrote:
Hi all,

Maybe this will save someone some Googling.

We recently switched our production servers to Oracle JDK 1.7.0_21 using the G1 garbage collector to address some GC problems (with excellent results so far, though early days).

Sadly recent versions of JDK 1.7.0 have a bug in the SSL code which causes intermittent failures for SSL connections using DH ciphers. Gory details:

http://mail.openjdk.java.net/pipermail/security-dev/2013-May/007435.html

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8014618

If your Tomcat is configured to serve SSL connections directly (e.g. on port 443 or 8443), then you may be affected by this bug. The symptoms is that some connections simply fail to connect (the SSL handshake fails). However, if all your connections to Tomcat are via AJP (e.g. port 8009) or HTTP then you won't be affected. We use port 8443 on our app servers for SOAP webservice connections, so as a result had random failures on webservice calls (Tomcat 7.0.40). Our 8443 connector is defined in Tomcat's server.xml as:

 <Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
               maxThreads="150" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />

The workaround (prior to a later 1.7.0 JDK release with a fix) is to use the BouncyCastle JCE provider code as an alternative to the Oracle JCE code. You can do this by downloading bcprov-jdk15on-149.jar from:

http://www.bouncycastle.org/latest_releases.html

and installing as per

http://www.bouncycastle.org/wiki/display/JA1/Provider+Installation

i.e. copy the jar into jre/lib/ext (in your JDK installation, not Sakai installation), and update jre/lib/security/java.security.

In our case we listed the bouncycastle provider in position #2 which seemed to work:

#
# List of providers and their preference orders (see above):
#
security.provider.1=sun.security.provider.Sun
security.provider.2=org.bouncycastle.jce.provider.BouncyCastleProvider
security.provider.3=sun.security.rsa.SunRsaSign
security.provider.4=sun.security.ec.SunEC
security.provider.5=com.sun.net.ssl.internal.ssl.Provider
security.provider.6=com.sun.crypto.provider.SunJCE
security.provider.7=sun.security.jgss.SunProvider
security.provider.8=com.sun.security.sasl.Provider
security.provider.9=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.10=sun.security.smartcardio.SunPCSC

One side-effect is that app servers take longer to initially respond to SSL connections after startup - it seems the bouncycastle code takes a while to generate a keypair (the CPU goes up for a while).

Regards
Stephen

________________________________
UNIVERSITY OF CAPE TOWN

This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www..uct.ac.za/about/policies/emaildisclaimer/<http://www.uct.ac.za/about/policies/emaildisclaimer/> or obtainable from +27 21 650 9111. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity. !DSPAM:2294,51b61d1b137623371864040!


_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"

!DSPAM:2294,51b61d1b137623371864040!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20130613/298afd92/attachment.html