[samigo-team] SAM-2102 small hibernate patch

[Sam Ottenhoff]

Sorry here is the new JIRA: https://jira.sakaiproject.org/browse/SAM-2107


On Thu, Mar 21, 2013 at 1:50 PM, Sam Ottenhoff <ottenhoff at longsight.com>wrote:

> Okay, it seems almost too good to be true so I'm hoping others can
> review.... I now see a 20x decrease in queries after a patch I just
> attached to SAM-2107.  I am still using the same assessment with 8 parts, 2
> questions per part from a pool of 42.  Samigo trunk requires 9886 queries.
>  Samigo with SAM-2102 and SAM-2107 requires 434 queries.
>
> The problems
> with SAM_PUBLISHEDITEMMETADATA_T and SAM_PUBLISHEDITEMFEEDBACK_T seemed to
> be related to helper methods that converted the Java Sets to HashMaps for
> convenience.  Unfortunately, the conversion to HashMaps turned into
> thousands of unnecessary Hibernate retrievals.  So I just removed the
> HashMap helper methods and replaced with simple iterations over the Sets
> (the Set is already in memory so no more database queries).
>
> --Sam
>
>
>
> On Thu, Mar 21, 2013 at 11:57 AM, Sam Ottenhoff <ottenhoff at longsight.com>wrote:
>
>> The batch size doesn't seem to matter much as long as it is over 10 or
>> so.  The basic problem is that Samigo uses lots of one-to-many
>> relationships to go from one assessment => many sections => many items  (+
>> feedback/metadata) => many answers (+feedback).  So the batch seems to
>> allow getting the "many" items all at once instead of individually.   So as
>> long as the batch-size is large enough to get all answers for an item or
>> all items for a section, it should be fine.  50 seemed safe and reasonable
>> to me.
>>
>> I still can't seem to reduce the queries to SAM_PUBLISHEDITEMMETADATA_T
>> and SAM_PUBLISHEDITEMFEEDBACK_T.  Samigo seems to load item for every
>> single possible question at both assessment begin and assessment end.  Does
>> anyone know how/why they are consulted at the end of the assessment?  That
>> might be a quick way to cut query volume by another 50% or so.
>>
>> --Sam
>>
>>
>>
>> On Thu, Mar 21, 2013 at 12:58 AM, Karen Tsao <ktsao at stanford.edu> wrote:
>>
>>> The patch looks good to me.
>>>
>>> Just have a quick question. How do you decide the batch size?
>>>
>>> Thanks,
>>> Karen
>>>
>>> On Wed, Mar 20, 2013 at 1:28 PM, Sam Ottenhoff <ottenhoff at longsight.com>wrote:
>>>
>>>> On the latest trunk, taking a 16-question random draw (2 questions each
>>>> in 8 parts from a 42-question pool) assessment generates about 9886 queries
>>>> to the database.
>>>>
>>>> I uploaded a small patch to the hibernate mapping file that reduced the
>>>> number of queries for my one attempt down to 1539 queries.
>>>>
>>>> Can someone review?
>>>>
>>>> https://jira.sakaiproject.org/browse/SAM-2102
>>>>
>>>> --Sam
>>>>
>>>> _______________________________________________
>>>> samigo-team mailing list
>>>> samigo-team at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/samigo-team
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/samigo-team/attachments/20130321/11706145/attachment-0001.html