[samigo-team] SAM-2102 small hibernate patch

[Sam Ottenhoff]

Okay, it seems almost too good to be true so I'm hoping others can
review.... I now see a 20x decrease in queries after a patch I just
attached to SAM-2107.  I am still using the same assessment with 8 parts, 2
questions per part from a pool of 42.  Samigo trunk requires 9886 queries.
 Samigo with SAM-2102 and SAM-2107 requires 434 queries.

The problems
with SAM_PUBLISHEDITEMMETADATA_T and SAM_PUBLISHEDITEMFEEDBACK_T seemed to
be related to helper methods that converted the Java Sets to HashMaps for
convenience.  Unfortunately, the conversion to HashMaps turned into
thousands of unnecessary Hibernate retrievals.  So I just removed the
HashMap helper methods and replaced with simple iterations over the Sets
(the Set is already in memory so no more database queries).

--Sam


On Thu, Mar 21, 2013 at 11:57 AM, Sam Ottenhoff <ottenhoff at longsight.com>wrote:

> The batch size doesn't seem to matter much as long as it is over 10 or so.
>  The basic problem is that Samigo uses lots of one-to-many relationships to
> go from one assessment => many sections => many items  (+
> feedback/metadata) => many answers (+feedback).  So the batch seems to
> allow getting the "many" items all at once instead of individually.   So as
> long as the batch-size is large enough to get all answers for an item or
> all items for a section, it should be fine.  50 seemed safe and reasonable
> to me.
>
> I still can't seem to reduce the queries to SAM_PUBLISHEDITEMMETADATA_T
> and SAM_PUBLISHEDITEMFEEDBACK_T.  Samigo seems to load item for every
> single possible question at both assessment begin and assessment end.  Does
> anyone know how/why they are consulted at the end of the assessment?  That
> might be a quick way to cut query volume by another 50% or so.
>
> --Sam
>
>
>
> On Thu, Mar 21, 2013 at 12:58 AM, Karen Tsao <ktsao at stanford.edu> wrote:
>
>> The patch looks good to me.
>>
>> Just have a quick question. How do you decide the batch size?
>>
>> Thanks,
>> Karen
>>
>> On Wed, Mar 20, 2013 at 1:28 PM, Sam Ottenhoff <ottenhoff at longsight.com>wrote:
>>
>>> On the latest trunk, taking a 16-question random draw (2 questions each
>>> in 8 parts from a 42-question pool) assessment generates about 9886 queries
>>> to the database.
>>>
>>> I uploaded a small patch to the hibernate mapping file that reduced the
>>> number of queries for my one attempt down to 1539 queries.
>>>
>>> Can someone review?
>>>
>>> https://jira.sakaiproject.org/browse/SAM-2102
>>>
>>> --Sam
>>>
>>> _______________________________________________
>>> samigo-team mailing list
>>> samigo-team at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/samigo-team
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/samigo-team/attachments/20130321/c5f73e9c/attachment.html