[sakai2-tcc] KNL-652 site caching testing (UMich preliminary testing)

Tue Mar 15 02:31:13 PDT 2011

>From UCT testing (we have this in production( I would vote +1 on the
merge as I am certain from our testing that it does not cause any harm.

Umich should have info as to whether it fixes the underlying issue.

D

On 03/15/2011 11:32 AM, Anthony Whyte wrote:
> It's a blocker against kernel 1.3.0 (sakai-2.8 binds to kernel 1.2).  For sakai-2.8.0 it is being tracked as a known issue and David Horwitz and I have held back from including the fix in kernel 1.2/1.1/1.0 releases while we await the results of UMich testing.
>
> I'd like to hear from the UMich members of the TCC list regarding their assessment of the site caching fix (r88169-70) and the implications for sakai-2.8.0 if it is included/excluded.
>
> Cheers,
>
> Anthony   
>
>
> On Mar 14, 2011, at 11:21 PM, May, Megan Marie wrote:
>
>   
>> Isn't this already on the blocker list for 2.8 or are you proposing an elevation of the bug?
>>
>> Megan
>>
>> -----Original Message-----
>> From: sakai2-tcc-bounces at collab.sakaiproject.org [mailto:sakai2-tcc-bounces at collab.sakaiproject.org] On Behalf Of Aaron Zeckoski
>> Sent: Friday, March 11, 2011 8:55 AM
>> To: Anthony Whyte
>> Cc: sakai2-tcc Committee
>> Subject: Re: [sakai2-tcc] KNL-652 site caching testing (UMich preliminary testing)
>>
>> I think that is a really good plan.
>> It may be worth backporting to 2.7 as well.
>> -AZ
>>
>>
>> On Fri, Mar 11, 2011 at 8:38 AM, Anthony Whyte <arwhyte at umich.edu> wrote:
>>     
>>> I recommend that if Michigan's production testing continues to proves 
>>> positive we should include this fix in sakai-2.8.0, even if it means 
>>> we delay the release a further 7-10 days in order to work it in.
>>> Cheers,
>>> Anthony
>>>
>>> Begin forwarded message:
>>>
>>> From: "David Haines (JIRA)" <bugs-admin at sakaiproject.org>
>>> Date: March 11, 2011 3:25:42 PM GMT+02:00
>>> To: mt-jira at collab.sakaiproject.org
>>> Subject: [mt-jira] [Sakai Jira] Commented: (KNL-652) SIte caching is 
>>> broken in Kernel 1.1.9
>>>
>>>
>>>    [
>>> https://jira.sakaiproject.org/browse/KNL-652?page=com.atlassian.jira.p
>>> lugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=122629#c
>>> omment-122629
>>> ]
>>>
>>> David Haines commented on KNL-652:
>>> ----------------------------------
>>>
>>> Load testing at Michigan using a lightly patched 1.1.9 kernel 
>>> indicates that the memory leak is significantly reduced.  This 
>>> required applying 4 patches related to caching: KNL-293, KNL-654, 
>>> KNL-662, and KNL-664 to fix multiple leaks.  Our tests indicate that a 
>>> site cache size of 10000 gives a site cache hit rate of around 90%.  
>>> We should have a week or more of production experience with these 
>>> changes, applied to a 1.1.10 kernel, as of the week of 4/4/11.
>>>
>>> SIte caching is broken in Kernel 1.1.9
>>>
>>> --------------------------------------
>>>
>>>                Key: KNL-652
>>>
>>>                URL: https://jira.sakaiproject.org/browse/KNL-652
>>>
>>>            Project: Kernel - K1
>>>
>>>         Issue Type: Bug
>>>
>>>         Components: Impl
>>>
>>>   Affects Versions: 1.1.9
>>>
>>>           Reporter: David Haines
>>>
>>>           Assignee: David Horwitz
>>>
>>>           Priority: Blocker
>>>
>>>            Fix For: 1.3.0
>>>
>>>        Attachments: Classes in Heap.png, Top Consumers.png
>>>
>>>
>>> Site caching is broken in the K1 kernel 1.1.9 release.  Sites objects 
>>> themselves are cached but when sites are evicted from the site cache 
>>> the secondary, in memory, caches for tools, pages, and groups are not 
>>> cleaned up, the references to the objects remain and those objects can 
>>> not be garbage collected.  Over time this leads to large numbers of 
>>> objects that won't be used but will consume memory.  At Michigan, 
>>> after not restarting Sakai for a couple of weeks we ended up with 
>>> almost 2GB of memory devoted to the site cache.  That caused long 
>>> periods of fruitless garbage collection and a degradation of service.
>>>
>>> The problem is avoided in the short run by restarting Sakai instances 
>>> or by manually clearing the caches with the Admin memory tool.
>>>
>>> A patch to address this is being tested at Michigan.
>>>
>>> Note that the only way to control Ehcache cache memory consumption for 
>>> a memory only cache is to limit the number of elements in the cache.  
>>> KNL-293 should also be applied when the site caching is fixed so that 
>>> there is a way to adjust the site cache size to match local requirements.
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> -
>>> For more information on JIRA, see: 
>>> http://www.atlassian.com/software/jira
>>>
>>>
>>> _______________________________________________
>>> mt-jira mailing list
>>> mt-jira at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/mt-jira
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> sakai2-tcc mailing list
>>> sakai2-tcc at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
>>>
>>>
>>>       
>>
>>
>> --
>> Aaron Zeckoski - Software Engineer - http://tinyurl.com/azprofile _______________________________________________
>> sakai2-tcc mailing list
>> sakai2-tcc at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
>>
>>
>>     
>   
>
>
> _______________________________________________
> sakai2-tcc mailing list
> sakai2-tcc at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20110315/7de288ce/attachment.html 