[Deploying Sakai] 2.7 load
David Horwitz
david.horwitz at uct.ac.za
Fri Sep 3 00:44:52 PDT 2010
Looking at the code what seems to have happened is that Chuck added
caching to BaseDbFlat storage that caches all properties collections
accesses that way. The properties in question are accessed directly from
the Db so are not cached. I'll have a look at adding some caching here -
it should be pretty straight foward.
http://jira.sakaiproject.org/browse/KNL-577
D
On 09/02/2010 08:47 PM, Matthew Jones wrote:
> Well, the SITE_PAGE_PROPERTY should be fixed by the issue KNL-259 that
> Noah/Zhen worked on. It didn't get committed to the main sakai repo
> for inclusing until 2.7.1/2.6.3, I'm betting this will help out a
> little. This was run both at IU and Michigan and resulted both in
> decreased queries and increased caching of these properties.
>
> Not sure about that TOOL_PROPERTY one but it "seems" like that one
> should/could be cached as well. I don't see one specifically set up
> for it though in the bean in db-components.xml or displaying in the
> memory tool.
>
> (Sample cache report)
> org.sakaiproject.db.BaseDbFlatStorage.SAKAI_ALIAS_PROPERTY: count:0
> hits:34 misses:34 hit%:50
> org.sakaiproject.db.BaseDbFlatStorage.SAKAI_REALM_PROPERTY: count:85
> hits:180440 misses:59292 hit%:75
> org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_GROUP_PROPERTY:
> count:0 hits:0 misses:24474 hit%:0
> org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PAGE_PROPERTY:
> count:374 hits:12941 misses:17298 hit%:42
> org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PROPERTY: count:329
> hits:725659 misses:187371 hit%:79
> org.sakaiproject.db.BaseDbFlatStorage.SAKAI_USER_PROPERTY: count:0
> hits:12 misses:12 hit%:50
>
> Stephen mentioned back in this JIRA [1] he also didn't notice this
> SAKAI_SITE_TOOL_PROPERTY show up in the cache list either. Would be
> worth investigating if this is possible, why it wasn't done.
>
> In talking to Chuck S, he recommended that kernel property (and other)
> caching be completely reviewed and improved, but this sounds like a
> big (2.9, 2.10) job.
>
> In Glenn's only blog post (that I know of) [2], he talked about
> improving the caching and performance of all of those SAKAI_REALM
> functions. Those hit us pretty hard too (with realm queries and
> presence driving most of the database traffic). Especially on some
> sites with lots of users and resources in content.
>
> [1] http://jira.sakaiproject.org/browse/SAK-13886?focusedCommentId=65873&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_65873
> <http://jira.sakaiproject.org/browse/SAK-13886?focusedCommentId=65873&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_65873>
>
> [2] http://galvanometer.blogspot.com/
>
> On Thu, Sep 2, 2010 at 8:47 AM, Hedrick Charles <hedrick at rutgers.edu
> <mailto:hedrick at rutgers.edu>> wrote:
>
> The students just got back for the Fall, so this is the first time
> we've run 2.7 under a realistic load. It looks to me like the CPU
> usage on both front ends and database is significantly higher, by
> at least a factor of 2. This is probably not going to kill us, but
> it worries me.
>
> I took a very brief snapshot of database queries during the day
> yesterday. Here's the tables it was querying. The scripts were
> pretty simple-minded, so this isn't perfect. But it should give an
> indication. The top queries seem to be simple lookups on single
> tables, so caching should work very well. I see no evidence that
> this data is actually being cached.
>
>
> 2561 SAKAI_SITE_TOOL_PROPERTY
> 2196 SAKAI_SITE_TOOL
> 1237 SAKAI_REALM_RL_FN,SAKAI_REALM
> 1053 SAKAI_SITE_PAGE_PROPERTY
> 253 SAKAI_SITE_PAGE
> 237
> 181 SAKAI_REALM_ROLE
> 108 SAKAI_PRIVACY_RECORD
> 89 SAKAI_SITE,SAKAI_SITE_USER
> 84 CONTENT_COLLECTION
> 70 SAKAI_SITE_PROPERTY
> 64 SAKAI_SITE
> 63 CONTENT_RESOURCE
> 48 SAKAI_REALM_ROLE_DESC
> 48 SAKAI_REALM_RL_GR
> 48 SAKAI_REALM_RL_FN
> 47 rutgers_user
> 33 SAKAI_ALIAS
> 30 SAKAI_SYLLABUS_ITEM
> 24 SAKAI_SESSION
> 24 SAKAI_REALM_FUNCTION
> 22 SAKAI_SITE_GROUP_PROPERTY
> 22 SAKAI_SITE_GROUP
> 20 ANNOUNCEMENT_CHANNEL
> 19 ANNOUNCEMENT_MESSAGE
> 15 SST_EVENTS
> 15 SAKAI_USER_ID_MAP
> 15 SAKAI_USER
> 13 SAKAI_PRESENCE
> 12 melete_resource
> 12 jforum_posts
> 11 SAKAI_REALM_PROPERTY
> 10 SAKAI_PREFERENCES
> 10 CM_MEMBER_CONTAINER_T
> 8 CALENDAR_CALENDAR
> 7 SAM_ASSESSMENTGRADING_T
> 7 CALENDAR_EVENT
> 6 the
> 6 identifications
> 5 melete_section
> 5 SST_SITEVISITS
> 5 SST_SITEACTIVITY
> 4 rutgers_user_section
> 4 osp_site_tool
> 4 osp_presentation_template
> 4 osp_authz_simple
> 4 magazines
> 4 jforum_import
> 3 steps
> 3 jforum_topics_mark
> 3 content_resource_lock
> 3 CHAT2_CHANNEL
> 2 osp_template_file_ref
> 2 osp_presentation_item_def
> 2 metaobj_form_def
> 2 melete_module
> 2 melete_course_module
> 2 jforum_users
> 2 jforum_sakai_sessions
> 2 a
> 2 SST_RESOURCES
> 2 SAM_PUBLISHEDEVALUATION_T
> 2 SAKAI_SITE_USER
> 2 SAKAI_REALM_PROVIDER
> 2 SAKAI_REALM
> 2 SAKAI_EVENT_DELAY
> 2 CONTENT_TYPE_REGISTRY
> 2 CM_ACADEMIC_SESSION_T
> 2 CHAT2_MESSAGE
> 2 ASSIGNMENT_CONTENT
> 2 ASSIGNMENT_ASSIGNMENT
> 1 osp_presentation_item
> 1 osp_presentation_comment
> 1 osp_presentation
> 1 melete_site_preference
> 1 melete_section_resource
> 1 melete_module_shdates
> 1 jforum_topics
> 1 jforum_sessions
> 1 jforum_sakai_course_initvalues
> 1 jforum_sakai_course_categories
> 1 jforum_privmsgs
> 1 jforum_categories
> 1 SITEASSOC_CONTEXT_ASSOCIATION
> 1 SAM_STUDENTGRADINGSUMMARY_T
> 1 SAM_PUBLISHEDSECTION_T
> 1 SAM_PUBLISHEDFEEDBACK_T
> 1 SAM_PUBLISHEDASSESSMENT_T
> 1 SAKAI_SYLLABUS_DATA
> 1 SAKAI_REALM,
> 1 QRTZ_TRIGGERS
> 1 CM_SEC_CATEGORY_T
>
>
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> <mailto:production at collab.sakaiproject.org>
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to
> production-unsubscribe at collab.sakaiproject.org
> <mailto:production-unsubscribe at collab.sakaiproject.org> with a
> subject of "unsubscribe"
>
>
>
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to production-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20100903/aa20ecca/attachment.html
More information about the production
mailing list