[Deploying Sakai] [Building Sakai] Strange database connection memory issue at U-M

[Sean DeMonner]

> Hi have you got a graph for a longer time period.
>
Not yet; we've restarted after a week to avoid problems during the  
beginning of the term. We've only been running 2.6 for a couple weeks.



> The graph did appear to have a relationship. E.G: your server  
> kungpao on the Tuesday and then exactly two days later alaking. If  
> its periodic then scheduled? Quartz job, something internal to the  
> database, backups, cronjobs, spiders etc?
>
We've wondered that too, but haven't found anything that is running  
at those times. The jumps are also not consistently spaced so we  
haven't seen a pattern.

SMD.







>
> Alan
>
> Alan Berg
>
> Senior Developer / Quality Assurance
> Group Education and Research Services
> Central Computer Services
> University of Amsterdam
>
>
>
> -----Original Message-----
> From: sakai-dev-bounces at collab.sakaiproject.org on behalf of Sean  
> DeMonner
> Sent: Thu 9/17/2009 15:48
> To: production at collab.sakaiproject.org; sakai- 
> dev at collab.sakaiproject.org Developers
> Subject: [Building Sakai] Strange database connection memory issue  
> at U-M
>
> All,
>
> We're seeing some strange behavior on our app servers at U-M. Here
> are some details:
>
> - Database connection memory usage on individual app servers
> increases by about .5 GB seemingly randomly every day or two. This is
> total memory for all 50 db pool connection from a given appserver,
> and confusingly enough there seems to be about an equal amount of
> increase in all 50 connections' memory usage
>
> - The memory has not been released, except upon a restart. It is
> likely it is a cache that when filled up might evict some data, but
> we haven't filled it yet -- we've done preemptive restarts to avoid
> problems at the beginning of the term.
>
> - Analysis of the log files does not reveal any telltale activities
> that might account for the increase in usage
>
> - The behavior is new to 2.6 (was not observed in 2.5)
>
> - We have seen it under light and heavy loads. What is interesting is
> that it is not strictly periodic nor obviously "random" (whatever
> that means)
>
> - It looks like this (each color is an app server):
>
>
> ?
>
>
> So far this doesn't seem like a crisis situation, but we are
> concerned that we have some strange resource gobbling activity that
> we can't explain.
>
> Has anyone seem similar behavior or have any ideas about what might
> be causing the memory jumps?
>
> Thanks,
>
> SMD.
>
>
> ==========================================================
> Sean DeMonner, IT Senior Project Manager, CTools Implementation Group
> Digital Media Commons @ The Duderstadt Center, U-M      (734) 615-9765
> Pub Key 24DD6B09
>
>
>
>
>
>
>
> <unknown.tiff>


SMD.


==========================================================
Sean DeMonner, IT Senior Project Manager, CTools Implementation Group
Digital Media Commons @ The Duderstadt Center, U-M      (734) 615-9765
Pub Key 24DD6B09




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20090917/2e3b3d1b/attachment.html