[Building Sakai] Samigo problem with valid submissions getting 0 scores

Thu May 31 04:27:37 PDT 2012

Karen,

This is the result from 1 of the 'affected' students:

select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag
where pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and
ag.agentid = '3e51ec13-2e0e-43e5-a28f-bbd102b08030';

Returns 2 records:
'180', 'Final Exam', '151', '', 'comments', '62', '1', '1', '1', '1',
'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-02-23 05:28:18',
'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-04-07 06:44:34',
'11611', '180', '3e51ec13-2e0e-43e5-a28f-bbd102b08030', '2012-03-27
08:25:27', '0', '1', '0', '0', '0', NULL, NULL, NULL, '1', '2012-03-27
07:04:29', '0', NULL

'180', 'Final Exam', '151', '', 'comments', '62', '1', '1', '1', '1',
'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-02-23 05:28:18',
'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-04-07 06:44:34',
'11731', '180', '3e51ec13-2e0e-43e5-a28f-bbd102b08030', NULL, '0',
'0', NULL, '0', NULL, NULL, NULL, NULL, '0', '2012-03-27 08:29:38',
'0', NULL

 2 ASSESSMENTGRADINGIDs: 11611 and 11731

select * from sam_itemgrading_t where assessmentgradingid = '11611';

Returns 50 records: attached as sam_itemgrading_t_11611.csv (notice
SUBMITTEDDATE, PUBLISHEDANSWERID, AUTOSCORE and OVERRIDESCORE are all
NULL)

select * from sam_itemgrading_t where assessmentgradingid = '11731';

Returns 0 records;

Having two rows of ASSESSMENTGRADINGIDs (one with 50 records
containing 'NULL' for SUBMITTEDDATE and PUBLISHEDANSWERID, and one
with 0 records) happens to 3 of the 5 affected students, the other 2
have only one row of ASSESSMENTGRADINGIDs where the sam_itemgrading_t
has 50 records containing 'NULL' for SUBMITTEDDATE and
PUBLISHEDANSWERID.

Tom

On Wed, May 30, 2012 at 9:07 PM, Karen Tsao <ktsao at stanford.edu> wrote:
> Hi Tom,
>
> Can you do the following queries for me?
>
> For one student:
>
> select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag where
> pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and ag.agentid
> = :user_id
>
> select * from sam_itemgrading_t where assessmentgradingid =
> :assessment_grading_id_above
>
> Please check if you get multiple records from the first query. Or if it's ok
> to send me the results, I can take a look for you.
>
> Thanks,
> Karen
>
>
>
> On Wed, May 30, 2012 at 7:30 AM, Tom Pennings <tom.pennings at gmail.com>
> wrote:
>>
>> Karen,
>>
>> Sorry for taking time to get back to you, we are currently working
>> hard at launching our summer classes.
>>
>> Using the UI (steps performed to verify described below):
>>  * For each of the affected students there was a confirmed, recorded
>> submission with a date, time and auto-scored total score of 0.
>>  * For each of the affected students none of the answers for the
>> questions show what was being selected by the student.
>>  * "Only 1 submission allowed" was set and we were able to confirm
>> that each of these affected students only had one submission.
>>
>> Using the database (query described below):
>>  * For each of the affected students no submitted answers could be
>> found in the database
>>
>> Do you have any other suggestions that can help in trying to debug this
>> problem?
>>
>> Sincerely,
>> Tom
>>
>> Steps to confirm no recorded answer:
>> - Log into Sakai
>> - Click tab of the affected course
>> - Click "Test & Quizes"
>> - Under section "Published Assessments/Final Exam" click "Scores"
>> - Click the name of the user
>> - For each of the 14 parts verify that for each question no answer for
>> the student was submitted
>>
>> Steps to confirm a single submission:
>> - Log into Sakai
>> - Click tab of the affected course
>> - Click "Test & Quizes"
>> - Under section "Published Assessments/Final Exam" click "Scores"
>> - Combo Boxes after "View:" should say "All Sections/Groups" and "All
>> Submissions"
>> - Verify that each affected students only appears once
>>
>> SQL query to look up submissions linked to a particular test by a
>> particular student:
>>  select u.user_id,pt.text as pubtext, pa.text as pubanswer,
>> g.SUBMITTEDDATE as submitteddate
>>  from sakai_deanza_final.sakai_user as u
>>  inner join sakai_deanza_final.sam_assessmentgrading_t as g on
>> (g.AGENTID = u.USER_ID)
>>  inner join sakai_deanza_final.sam_publishedassessment_t as a on
>> (a.ID = g.PUBLISHEDASSESSMENTID)
>>  inner join sakai_deanza_final.sam_itemgrading_t as gi on
>> (gi.ASSESSMENTGRADINGID = g.ASSESSMENTGRADINGID)
>>  inner join sakai_deanza_final.sam_publisheditemtext_t as pt on
>> (pt.ITEMTEXTID = gi.PUBLISHEDITEMTEXTID)
>>  inner join sakai_deanza_final.sam_publishedanswer_t as pa on
>> (pa.ANSWERID = gi.PUBLISHEDANSWERID)
>>  where u.FIRST_NAME = 'Foo' and u.LAST_NAME = 'Bar'
>>  and a.TITLE = 'Final Exam'
>>  and g.SUBMITTEDDATE is not null
>>  and gi.PUBLISHEDANSWERID is not null;
>>
>> On Wed, May 2, 2012 at 6:42 AM, Bryan Holladay <holladay at longsight.com>
>> wrote:
>> > It's interesting that you are having this trouble too... I am working
>> > on an issue just like this.  Its a single page test with a mix of
>> > random multiple choice and fill in the blank.  The instructor noticed
>> > that a few random students had a score of 0 and when she looked at the
>> > submission, the student had quite a few correct multiple choice
>> > answers with a score of 0 (even the check mark was next to it).  She
>> > also found that if you edit the test and regrade and republish that
>> > the score would be fixed but other random students would end up
>> > getting a score of 0.
>> >
>> > I'm at a lost and am trying to figure out how to reproduce this
>> > myself.  I have a site that I can see this error where the instructor
>> > gave me permission to "play around with".  This is on 2.8.x + some
>> > additional features in trunk.
>> >
>> > -Bryan
>> >
>> >
>> > On Wed, May 2, 2012 at 1:58 AM, Karen Tsao <ktsao at stanford.edu> wrote:
>> >> Hi Tom,
>> >>
>> >> I don't have any idea regarding your issue at this point. But can you
>> >> please
>> >> check follow things for me?
>> >>
>> >> For these students who got 0 points, do their answers exist? That is,
>> >> can
>> >> you see their answers from grading pages? Can you also check if there
>> >> are
>> >> multiple submissions for these students, from both UI and database.
>> >>
>> >> Thanks,
>> >> Karen
>> >>
>> >>
>> >>
>> >> On Tue, May 1, 2012 at 7:03 PM, Tom Pennings <tom.pennings at gmail.com>
>> >> wrote:
>> >>>
>> >>> We've been experiencing an issue with Samigo 2.6.2 where, even though
>> >>> all of the students answered all of the questions (some correct, some
>> >>> incorrect) and successfully submitted (with confirmation) the
>> >>> assessment, some of their final scores were recorded as 0.
>> >>>
>> >>> The problem occurred intermittently (10 - 25% of the cases) on two
>> >>> different assessments:
>> >>>
>> >>>  1. 7 parts, each on its own page, a total of 33 questions, all
>> >>> multiple-choice
>> >>>  2. 14 parts, all on one page, a total of 50 questions, all
>> >>> multiple-choice
>> >>>
>> >>> So far we've been unable to find the root cause, or a way to
>> >>> reproduce. What we did find though is that with 'Each Question is on a
>> >>> separate Web page' only individual questions would get submitted as 0,
>> >>> whereas in case of 'The complete Assessment is displayed on one Web
>> >>> page' the whole assessment would evaluate as 0, which seems to suggest
>> >>> an issue during processing of a post though no problems were found in
>> >>> the logs. We've also tested and ruled out session time outs as the
>> >>> reason for the problem.
>> >>>
>> >>> Any suggestions and advice on how to further debug the problem or
>> >>> proposed solutions will be greatly appreciated.
>> >>>
>> >>>
>> >>> Tom
>> >>> _______________________________________________
>> >>> sakai-dev mailing list
>> >>> sakai-dev at collab.sakaiproject.org
>> >>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> >>>
>> >>> TO UNSUBSCRIBE: send email to
>> >>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>> >>> "unsubscribe"
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> sakai-dev mailing list
>> >> sakai-dev at collab.sakaiproject.org
>> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> >>
>> >> TO UNSUBSCRIBE: send email to
>> >> sakai-dev-unsubscribe at collab.sakaiproject.org
>> >> with a subject of "unsubscribe"
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sam_itemgrading_t_11611.csv
Type: text/csv
Size: 6171 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20120531/c280d59c/attachment.bin 