[sakai2-tcc] KNL-652 site caching testing (UMich preliminary testing)

[Anthony Whyte]

It's a blocker against kernel 1.3.0 (sakai-2.8 binds to kernel 1.2).  For sakai-2.8.0 it is being tracked as a known issue and David Horwitz and I have held back from including the fix in kernel 1.2/1.1/1.0 releases while we await the results of UMich testing.

I'd like to hear from the UMich members of the TCC list regarding their assessment of the site caching fix (r88169-70) and the implications for sakai-2.8.0 if it is included/excluded.

Cheers,

Anthony   


On Mar 14, 2011, at 11:21 PM, May, Megan Marie wrote:

> Isn't this already on the blocker list for 2.8 or are you proposing an elevation of the bug?
> 
> Megan
> 
> -----Original Message-----
> From: sakai2-tcc-bounces at collab.sakaiproject.org [mailto:sakai2-tcc-bounces at collab.sakaiproject.org] On Behalf Of Aaron Zeckoski
> Sent: Friday, March 11, 2011 8:55 AM
> To: Anthony Whyte
> Cc: sakai2-tcc Committee
> Subject: Re: [sakai2-tcc] KNL-652 site caching testing (UMich preliminary testing)
> 
> I think that is a really good plan.
> It may be worth backporting to 2.7 as well.
> -AZ
> 
> 
> On Fri, Mar 11, 2011 at 8:38 AM, Anthony Whyte <arwhyte at umich.edu> wrote:
>> I recommend that if Michigan's production testing continues to proves 
>> positive we should include this fix in sakai-2.8.0, even if it means 
>> we delay the release a further 7-10 days in order to work it in.
>> Cheers,
>> Anthony
>> 
>> Begin forwarded message:
>> 
>> From: "David Haines (JIRA)" <bugs-admin at sakaiproject.org>
>> Date: March 11, 2011 3:25:42 PM GMT+02:00
>> To: mt-jira at collab.sakaiproject.org
>> Subject: [mt-jira] [Sakai Jira] Commented: (KNL-652) SIte caching is 
>> broken in Kernel 1.1.9
>> 
>> 
>>    [
>> https://jira.sakaiproject.org/browse/KNL-652?page=com.atlassian.jira.p
>> lugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=122629#c
>> omment-122629
>> ]
>> 
>> David Haines commented on KNL-652:
>> ----------------------------------
>> 
>> Load testing at Michigan using a lightly patched 1.1.9 kernel 
>> indicates that the memory leak is significantly reduced.  This 
>> required applying 4 patches related to caching: KNL-293, KNL-654, 
>> KNL-662, and KNL-664 to fix multiple leaks.  Our tests indicate that a 
>> site cache size of 10000 gives a site cache hit rate of around 90%.  
>> We should have a week or more of production experience with these 
>> changes, applied to a 1.1.10 kernel, as of the week of 4/4/11.
>> 
>> SIte caching is broken in Kernel 1.1.9
>> 
>> --------------------------------------
>> 
>>                Key: KNL-652
>> 
>>                URL: https://jira.sakaiproject.org/browse/KNL-652
>> 
>>            Project: Kernel - K1
>> 
>>         Issue Type: Bug
>> 
>>         Components: Impl
>> 
>>   Affects Versions: 1.1.9
>> 
>>           Reporter: David Haines
>> 
>>           Assignee: David Horwitz
>> 
>>           Priority: Blocker
>> 
>>            Fix For: 1.3.0
>> 
>>        Attachments: Classes in Heap.png, Top Consumers.png
>> 
>> 
>> Site caching is broken in the K1 kernel 1.1.9 release.  Sites objects 
>> themselves are cached but when sites are evicted from the site cache 
>> the secondary, in memory, caches for tools, pages, and groups are not 
>> cleaned up, the references to the objects remain and those objects can 
>> not be garbage collected.  Over time this leads to large numbers of 
>> objects that won't be used but will consume memory.  At Michigan, 
>> after not restarting Sakai for a couple of weeks we ended up with 
>> almost 2GB of memory devoted to the site cache.  That caused long 
>> periods of fruitless garbage collection and a degradation of service.
>> 
>> The problem is avoided in the short run by restarting Sakai instances 
>> or by manually clearing the caches with the Admin memory tool.
>> 
>> A patch to address this is being tested at Michigan.
>> 
>> Note that the only way to control Ehcache cache memory consumption for 
>> a memory only cache is to limit the number of elements in the cache.  
>> KNL-293 should also be applied when the site caching is fixed so that 
>> there is a way to adjust the site cache size to match local requirements.
>> 
>> --
>> This message is automatically generated by JIRA.
>> -
>> For more information on JIRA, see: 
>> http://www.atlassian.com/software/jira
>> 
>> 
>> _______________________________________________
>> mt-jira mailing list
>> mt-jira at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/mt-jira
>> 
>> 
>> 
>> 
>> _______________________________________________
>> sakai2-tcc mailing list
>> sakai2-tcc at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
>> 
>> 
> 
> 
> 
> --
> Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile _______________________________________________
> sakai2-tcc mailing list
> sakai2-tcc at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3829 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20110315/7aed8522/attachment-0001.bin