[samigo-team] Performance testing and Samigo 2.9

Tue May 8 04:59:40 PDT 2012

Hi all,

We have identified one of the performance issues as related to queries
for SAM_GRADINGATTACHMENT_T. In fact these aren't queries as such, but
cases where Hibernate issues a Prepare statement for a query but without
an Execute (for mysql server-side prepared statements), presumably
because it decides it has the object and doesn't need or some other
Hibernate behaviour.

If we switch on mysql logging, we see just over 90,000 Prepare
statements like this:

Prepare   select itemgradin0_.ITEMGRADINGID as ITEMGRA15_1_,
itemgradin0_.ATTACHMENTID as ATTACHME1_1_, itemgradin0_.ATTACHMENTID as
ATTACHME1_198_0_, itemgradin0_.RESOURCEID as RESOURCEID198_0_,
itemgradin0_.FILENAME as FILENAME198_0_, itemgradin0_.MIMETYPE as
MIMETYPE198_0_, itemgradin0_.FILESIZE as FILESIZE198_0_,
itemgradin0_.DESCRIPTION as DESCRIPT7_198_0_, itemgradin0_.LOCATION as
LOCATION198_0_, itemgradin0_.ISLINK as ISLINK198_0_, itemgradin0_.STATUS
as STATUS198_0_, itemgradin0_.CREATEDBY as CREATEDBY198_0_,
itemgradin0_.CREATEDDATE as CREATED12_198_0_,
itemgradin0_.LASTMODIFIEDBY as LASTMOD13_198_0_,
itemgradin0_.LASTMODIFIEDDATE as LASTMOD14_198_0_,
itemgradin0_.ITEMGRADINGID as ITEMGRA15_198_0_ from
SAM_GRADINGATTACHMENT_T itemgradin0_ where itemgradin0_.ITEMGRADINGID=?
order by itemgradin0_.createdDate asc

but if that object is changed to lazy loading like this:

Index:
samigo-hibernate/src/java/org/sakaiproject/tool/assessment/data/dao/grading/GradingData.hbm.xml
===================================================================
---
samigo-hibernate/src/java/org/sakaiproject/tool/assessment/data/dao/grading/GradingData.hbm.xml
    (revision 107894)
+++
samigo-hibernate/src/java/org/sakaiproject/tool/assessment/data/dao/grading/GradingData.hbm.xml
    (working copy)
@@ -28,7 +28,7 @@
     <property name="lastDuration" type="string" length="36"
column="LASTDURATION" not-null="false" />

        <set name="itemGradingAttachmentSet"
table="SAM_GRADINGATTACHMENT_T" cascade="all" order-by="createdDate
asc"
-        inverse="true" lazy="false">
+        inverse="true" lazy="true">
         <cache usage="read-write"/>
         <key column="ITEMGRADINGID"/>
         <one-to-many
class="org.sakaiproject.tool.assessment.data.dao.grading.ItemGradingAttachment"/>

then all those Prepare statements do not get issued, and the
performance is much better (by 400ms+ per request towards the end of the
assessment).

What feature is SAM_GRADINGATTACHMENT_T used for? In our production
database, this only has 3 rows so it's presumably not much used.

Is there any reason not to switch it to lazy=true or functionality that
should be QA'd if we do so?

Regards
Stephen

Stephen Marquard, Acting Director 
Centre for Educational Technology, University of Cape Town
http://www.cet.uct.ac.za
Email/IM/XMPP: stephen.marquard at uct.ac.za 
Phone: +27-21-650-5037 Cell: +27-83-500-5290 
 


>>> "Stephen Marquard" <stephen.marquard at uct.ac.za> 5/6/2012 8:04 PM
>>>
Hi all,

Unfortunately I was mistaken about this issue being resolved in 2.9 (I
was misled by an error in a JMeter test). Both 2.9 and trunk still
show
increasing response times as the test progresses. This is related to
the
action of saving the answer. In a test where no answers are saved
(just
moving on to the next question), the response time is constant.

So we don't have an answer yet and are going to continue looking into
the code. Any suggestions welcome.

Regards
Stephen  
 

Stephen Marquard, Acting Director 
Centre for Educational Technology, University of Cape Town
http://www.cet.uct.ac.za 
Email/IM/XMPP: stephen.marquard at uct.ac.za 
Phone: +27-21-650-5037 Cell: +27-83-500-5290 
 


>>> "Stephen Marquard" <stephen.marquard at uct.ac.za> 5/3/2012 9:22 PM
>>> 
Hi all,

We have been doing some performance and load testing of T&Q for large
assessments (160 and 400 questions) with JMeter, and have identified
one
specific issue which is that in 2.8, the response time per item
submission increases as the test progresses (e.g. submitting answer to
Q1 could take 100ms but for Q160 the response time is 500ms).

The good news is that this behaviour is no longer present in 2.9 and
trunk - the response time is the same no matter where you are in the
test.

So two followup questions:

1. Is anyone aware of a specific JIRA / code change between 2.8 and
2.9
that would have fixed this issue? (As if so, we would consider merging
this to a local 2-8-x branch).

2. How much testing has the 2.9 T&Q release candidate had, and would
you consider it ready to run in production?

Regards
Stephen



Stephen Marquard, Acting Director 
Centre for Educational Technology, University of Cape Town
http://www.cet.uct.ac.za 
Email/IM/XMPP: stephen.marquard at uct.ac.za 
Phone: +27-21-650-5037 Cell: +27-83-500-5290 
 




###

UNIVERSITY OF CAPE TOWN 

This e-mail is subject to the UCT ICT policies and e-mail disclaimer
published on our website at
http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable
from
+27 21 650 9111. This e-mail is intended only for the person(s) to
whom
it is addressed. If the e-mail has reached you in error, please notify
the author. If you are not the intended recipient of the e-mail you
may
not use, disclose, copy, redirect or print the content. If this e-mail
is not related to the business of UCT it is sent by the sender in the
sender's individual capacity.

###
 

_______________________________________________
samigo-team mailing list
samigo-team at collab.sakaiproject.org 
http://collab.sakaiproject.org/mailman/listinfo/samigo-team 





###

UNIVERSITY OF CAPE TOWN 

This e-mail is subject to the UCT ICT policies and e-mail disclaimer
published on our website at
http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from
+27 21 650 9111. This e-mail is intended only for the person(s) to whom
it is addressed. If the e-mail has reached you in error, please notify
the author. If you are not the intended recipient of the e-mail you may
not use, disclose, copy, redirect or print the content. If this e-mail
is not related to the business of UCT it is sent by the sender in the
sender's individual capacity.

###
 

