[Deploying Sakai] 2.7 load

Thu Sep 2 11:47:53 PDT 2010

Well, the SITE_PAGE_PROPERTY should be fixed by the issue KNL-259 that
Noah/Zhen worked on. It didn't get committed to the main sakai repo for
inclusing until 2.7.1/2.6.3, I'm betting this will help out a little. This
was run both at IU and Michigan and resulted both in decreased queries and
increased caching of these properties.

Not sure about that TOOL_PROPERTY one but it "seems" like that one
should/could be cached as well. I don't see one specifically set up for it
though in the bean in db-components.xml or displaying in the memory tool.

(Sample cache report)
org.sakaiproject.db.BaseDbFlatStorage.SAKAI_ALIAS_PROPERTY: count:0 hits:34
misses:34 hit%:50
org.sakaiproject.db.BaseDbFlatStorage.SAKAI_REALM_PROPERTY: count:85
hits:180440 misses:59292 hit%:75
org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_GROUP_PROPERTY: count:0
hits:0 misses:24474 hit%:0
org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PAGE_PROPERTY: count:374
hits:12941 misses:17298 hit%:42
org.sakaiproject.db.BaseDbFlatStorage.SAKAI_SITE_PROPERTY: count:329
hits:725659 misses:187371 hit%:79
org.sakaiproject.db.BaseDbFlatStorage.SAKAI_USER_PROPERTY: count:0 hits:12
misses:12 hit%:50

Stephen mentioned back in this JIRA [1] he also didn't notice this
SAKAI_SITE_TOOL_PROPERTY show up in the cache list either. Would be worth
investigating if this is possible, why it wasn't done.

In talking to Chuck S, he recommended that kernel property (and other)
caching be completely reviewed and improved, but this sounds like a big
(2.9, 2.10) job.

In Glenn's only blog post (that I know of) [2], he talked about improving
the caching and performance of all of those SAKAI_REALM functions. Those hit
us pretty hard too (with realm queries and presence driving most of the
database traffic). Especially on some sites with lots of users and resources
in content.

[1]
http://jira.sakaiproject.org/browse/SAK-13886?focusedCommentId=65873&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_65873

[2] http://galvanometer.blogspot.com/

On Thu, Sep 2, 2010 at 8:47 AM, Hedrick Charles <hedrick at rutgers.edu> wrote:

> The students just got back for the Fall, so this is the first time we've
> run 2.7 under a realistic load. It looks to me like the CPU usage on both
> front ends and database is significantly higher, by at least a factor of 2.
> This is probably not going to kill us, but it worries me.
>
> I took a very brief snapshot of database queries during the day yesterday.
> Here's the tables it was querying. The scripts were pretty simple-minded, so
> this isn't perfect. But it should give an indication. The top queries seem
> to be simple lookups on single tables, so caching should work very well. I
> see no evidence that this data is actually being cached.
>
>
>   2561 SAKAI_SITE_TOOL_PROPERTY
>   2196 SAKAI_SITE_TOOL
>   1237 SAKAI_REALM_RL_FN,SAKAI_REALM
>   1053 SAKAI_SITE_PAGE_PROPERTY
>    253 SAKAI_SITE_PAGE
>    237
>    181 SAKAI_REALM_ROLE
>    108 SAKAI_PRIVACY_RECORD
>     89 SAKAI_SITE,SAKAI_SITE_USER
>     84 CONTENT_COLLECTION
>     70 SAKAI_SITE_PROPERTY
>     64 SAKAI_SITE
>     63 CONTENT_RESOURCE
>     48 SAKAI_REALM_ROLE_DESC
>     48 SAKAI_REALM_RL_GR
>     48 SAKAI_REALM_RL_FN
>     47 rutgers_user
>     33 SAKAI_ALIAS
>     30 SAKAI_SYLLABUS_ITEM
>     24 SAKAI_SESSION
>     24 SAKAI_REALM_FUNCTION
>     22 SAKAI_SITE_GROUP_PROPERTY
>     22 SAKAI_SITE_GROUP
>     20 ANNOUNCEMENT_CHANNEL
>     19 ANNOUNCEMENT_MESSAGE
>     15 SST_EVENTS
>     15 SAKAI_USER_ID_MAP
>     15 SAKAI_USER
>     13 SAKAI_PRESENCE
>     12 melete_resource
>     12 jforum_posts
>     11 SAKAI_REALM_PROPERTY
>     10 SAKAI_PREFERENCES
>     10 CM_MEMBER_CONTAINER_T
>      8 CALENDAR_CALENDAR
>      7 SAM_ASSESSMENTGRADING_T
>      7 CALENDAR_EVENT
>      6 the
>      6 identifications
>      5 melete_section
>      5 SST_SITEVISITS
>      5 SST_SITEACTIVITY
>      4 rutgers_user_section
>      4 osp_site_tool
>      4 osp_presentation_template
>      4 osp_authz_simple
>      4 magazines
>      4 jforum_import
>      3 steps
>      3 jforum_topics_mark
>      3 content_resource_lock
>      3 CHAT2_CHANNEL
>      2 osp_template_file_ref
>      2 osp_presentation_item_def
>      2 metaobj_form_def
>      2 melete_module
>      2 melete_course_module
>      2 jforum_users
>      2 jforum_sakai_sessions
>      2 a
>      2 SST_RESOURCES
>      2 SAM_PUBLISHEDEVALUATION_T
>      2 SAKAI_SITE_USER
>      2 SAKAI_REALM_PROVIDER
>      2 SAKAI_REALM
>      2 SAKAI_EVENT_DELAY
>      2 CONTENT_TYPE_REGISTRY
>      2 CM_ACADEMIC_SESSION_T
>      2 CHAT2_MESSAGE
>      2 ASSIGNMENT_CONTENT
>      2 ASSIGNMENT_ASSIGNMENT
>      1 osp_presentation_item
>      1 osp_presentation_comment
>      1 osp_presentation
>      1 melete_site_preference
>      1 melete_section_resource
>      1 melete_module_shdates
>      1 jforum_topics
>      1 jforum_sessions
>      1 jforum_sakai_course_initvalues
>      1 jforum_sakai_course_categories
>      1 jforum_privmsgs
>      1 jforum_categories
>      1 SITEASSOC_CONTEXT_ASSOCIATION
>      1 SAM_STUDENTGRADINGSUMMARY_T
>      1 SAM_PUBLISHEDSECTION_T
>      1 SAM_PUBLISHEDFEEDBACK_T
>      1 SAM_PUBLISHEDASSESSMENT_T
>      1 SAKAI_SYLLABUS_DATA
>      1 SAKAI_REALM,
>      1 QRTZ_TRIGGERS
>      1 CM_SEC_CATEGORY_T
>
>
> _______________________________________________
> production mailing list
> production at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/production
>
> TO UNSUBSCRIBE: send email to
> production-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/production/attachments/20100902/d15ba480/attachment.html 