[Building Sakai] log4j + DailyRollingFileAppender problems

[David Adams]

Hey everyone,
Since we recently upgraded to 2.7.1 and added Profile2 to our toolset, we've started having issues where our main logfile, which is managed by DailyRollingFileAppender and which we call sakai.log, will be running along fine until at some point in the day a profile2 log statement will trigger the day's sakai.log to rotate to sakai.log.<yesterday's date>, and there will be a single log statement in sakai.log. Later in the day there might be others, but we're seeing just a dozen or fewer in sakai.log and the rest of the day's log goes on as normal, only in a file dated yesterday, which then when we do our nightly log archiving, overwrites the previous day's log.

This doesn't happen every day, nor does it always happen. The particular log statements that we find in the non-rotated sakai.log file when this happens never appear in the misdated log for that day. But on other days when this doesn't happen those same log statements will appear in sakai.log alongside everything else.

So given that it's inconsistent, I don't think there's anything awry with the profile2 logging calls per se, nor with our configuration. Given that there are multiple log4j jars with various versions both in tomcat/common/lib but also in a few tools' WEB-INF/lib, including profile2, I wondered if it wasn't a matter of two log4js not knowing about each other, one assuming the file needed rotating, the other blindly assuming its file handle was still to the unrotated file.

After doing some research, I think my theory is correct, based on comments in this log4j bug report: https://issues.apache.org/bugzilla/show_bug.cgi?id=45236

In particular: "I believe that this may be caused by several web applications configured
identically even if it is in the same web container.  If you need common
logging for all web applications you need to move log4j.jar out of the
individual web applications and into the area providing common jars for all web
application."

So, that being said, there are a number of tools, at least in 2.7 and before, that pull in their own log4j jars. Is there a particular need for that setup, some reason they can't all use the same common/lib log4j?

Has anyone else run into this problem or addressed it? And does anyone have any thoughts on how we could go about safely getting rid of the various copies? I'm not a developer, so I cop to being pretty vague on exactly how all the jar loading works and how the deployment and compilation steps interact with various libraries and packaging steps, etc.

Thanks for your thoughts, anyone.

-- 
David Adams
Director, Learning Systems Integration and Support
Virginia Tech Learning Technologies