[Building Sakai] Samigo problem with valid submissions getting 0 scores

Tom Pennings tom.pennings at gmail.com
Wed Jun 6 14:02:04 PDT 2012 

Karen,

Unfortunately removing the extra grading records for the 3 affected
students from the database didn't provide us with their submitted
answers (as each of their 50 itemgrading records contained 'NULL' for
SUBMITTEDDATE and PUBLISHEDANSWERID).

None of the students mentioned errors or warnings during or after
submitting the exam, and they were able to successfully 'Submit for
Grading'.

The assessment was NOT timed, though had a 'Available Date' and 'Due Date' set.

We continue to be puzzled by this behavior and currently don't have a
way to reproduce. If there are any further developments we'll be happy
to post them to this thread. In the meantime, if you can think of
anything else we can try to help debug this issue further feel free to
let us know.

Tom

On Thu, May 31, 2012 at 9:20 PM, Karen Tsao <ktsao at stanford.edu> wrote:
> Hi Tom,
>
> I don't know what triggered this issue, but I think you can fix the data by
> removing the extra records. From your example, you can remove the records
> for assessmentgradingid 11731(in both sam_assessmentgrading_t and
> sam_itemgrading_t). I don't know why there is no saved/meaningful records
> for the other two students. But if their record contains no answer in db, I
> guess there is no way to get them back.
>
> Did these students see any errors/warning during the exam? Is this a timed
> assessment? I really want to get this issue fixed, if you happen to find a
> way to reproduce this, please let me know.
>
> Thanks,
> Karen
>
>
> On Thu, May 31, 2012 at 4:27 AM, Tom Pennings <tom.pennings at gmail.com>
> wrote:
>>
>> Karen,
>>
>> This is the result from 1 of the 'affected' students:
>>
>> select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag
>> where pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and
>> ag.agentid = '3e51ec13-2e0e-43e5-a28f-bbd102b08030';
>>
>> Returns 2 records:
>> '180', 'Final Exam', '151', '', 'comments', '62', '1', '1', '1', '1',
>> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-02-23 05:28:18',
>> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-04-07 06:44:34',
>> '11611', '180', '3e51ec13-2e0e-43e5-a28f-bbd102b08030', '2012-03-27
>> 08:25:27', '0', '1', '0', '0', '0', NULL, NULL, NULL, '1', '2012-03-27
>> 07:04:29', '0', NULL
>>
>> '180', 'Final Exam', '151', '', 'comments', '62', '1', '1', '1', '1',
>> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-02-23 05:28:18',
>> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-04-07 06:44:34',
>> '11731', '180', '3e51ec13-2e0e-43e5-a28f-bbd102b08030', NULL, '0',
>> '0', NULL, '0', NULL, NULL, NULL, NULL, '0', '2012-03-27 08:29:38',
>> '0', NULL
>>
>>  2 ASSESSMENTGRADINGIDs: 11611 and 11731
>>
>> select * from sam_itemgrading_t where assessmentgradingid = '11611';
>>
>> Returns 50 records: attached as sam_itemgrading_t_11611.csv (notice
>> SUBMITTEDDATE, PUBLISHEDANSWERID, AUTOSCORE and OVERRIDESCORE are all
>> NULL)
>>
>> select * from sam_itemgrading_t where assessmentgradingid = '11731';
>>
>> Returns 0 records;
>>
>>
>> Having two rows of ASSESSMENTGRADINGIDs (one with 50 records
>> containing 'NULL' for SUBMITTEDDATE and PUBLISHEDANSWERID, and one
>> with 0 records) happens to 3 of the 5 affected students, the other 2
>> have only one row of ASSESSMENTGRADINGIDs where the sam_itemgrading_t
>> has 50 records containing 'NULL' for SUBMITTEDDATE and
>> PUBLISHEDANSWERID.
>>
>> Tom
>>
>> On Wed, May 30, 2012 at 9:07 PM, Karen Tsao <ktsao at stanford.edu> wrote:
>> > Hi Tom,
>> >
>> > Can you do the following queries for me?
>> >
>> > For one student:
>> >
>> > select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag
>> > where
>> > pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and
>> > ag.agentid
>> > = :user_id
>> >
>> > select * from sam_itemgrading_t where assessmentgradingid =
>> > :assessment_grading_id_above
>> >
>> > Please check if you get multiple records from the first query. Or if
>> > it's ok
>> > to send me the results, I can take a look for you.
>> >
>> > Thanks,
>> > Karen
>> >
>> >
>> >
>> > On Wed, May 30, 2012 at 7:30 AM, Tom Pennings <tom.pennings at gmail.com>
>> > wrote:
>> >>
>> >> Karen,
>> >>
>> >> Sorry for taking time to get back to you, we are currently working
>> >> hard at launching our summer classes.
>> >>
>> >> Using the UI (steps performed to verify described below):
>> >>  * For each of the affected students there was a confirmed, recorded
>> >> submission with a date, time and auto-scored total score of 0.
>> >>  * For each of the affected students none of the answers for the
>> >> questions show what was being selected by the student.
>> >>  * "Only 1 submission allowed" was set and we were able to confirm
>> >> that each of these affected students only had one submission.
>> >>
>> >> Using the database (query described below):
>> >>  * For each of the affected students no submitted answers could be
>> >> found in the database
>> >>
>> >> Do you have any other suggestions that can help in trying to debug this
>> >> problem?
>> >>
>> >> Sincerely,
>> >> Tom
>> >>
>> >> Steps to confirm no recorded answer:
>> >> - Log into Sakai
>> >> - Click tab of the affected course
>> >> - Click "Test & Quizes"
>> >> - Under section "Published Assessments/Final Exam" click "Scores"
>> >> - Click the name of the user
>> >> - For each of the 14 parts verify that for each question no answer for
>> >> the student was submitted
>> >>
>> >> Steps to confirm a single submission:
>> >> - Log into Sakai
>> >> - Click tab of the affected course
>> >> - Click "Test & Quizes"
>> >> - Under section "Published Assessments/Final Exam" click "Scores"
>> >> - Combo Boxes after "View:" should say "All Sections/Groups" and "All
>> >> Submissions"
>> >> - Verify that each affected students only appears once
>> >>
>> >> SQL query to look up submissions linked to a particular test by a
>> >> particular student:
>> >>  select u.user_id,pt.text as pubtext, pa.text as pubanswer,
>> >> g.SUBMITTEDDATE as submitteddate
>> >>  from sakai_deanza_final.sakai_user as u
>> >>  inner join sakai_deanza_final.sam_assessmentgrading_t as g on
>> >> (g.AGENTID = u.USER_ID)
>> >>  inner join sakai_deanza_final.sam_publishedassessment_t as a on
>> >> (a.ID = g.PUBLISHEDASSESSMENTID)
>> >>  inner join sakai_deanza_final.sam_itemgrading_t as gi on
>> >> (gi.ASSESSMENTGRADINGID = g.ASSESSMENTGRADINGID)
>> >>  inner join sakai_deanza_final.sam_publisheditemtext_t as pt on
>> >> (pt.ITEMTEXTID = gi.PUBLISHEDITEMTEXTID)
>> >>  inner join sakai_deanza_final.sam_publishedanswer_t as pa on
>> >> (pa.ANSWERID = gi.PUBLISHEDANSWERID)
>> >>  where u.FIRST_NAME = 'Foo' and u.LAST_NAME = 'Bar'
>> >>  and a.TITLE = 'Final Exam'
>> >>  and g.SUBMITTEDDATE is not null
>> >>  and gi.PUBLISHEDANSWERID is not null;
>> >>
>> >> On Wed, May 2, 2012 at 6:42 AM, Bryan Holladay <holladay at longsight.com>
>> >> wrote:
>> >> > It's interesting that you are having this trouble too... I am working
>> >> > on an issue just like this.  Its a single page test with a mix of
>> >> > random multiple choice and fill in the blank.  The instructor noticed
>> >> > that a few random students had a score of 0 and when she looked at
>> >> > the
>> >> > submission, the student had quite a few correct multiple choice
>> >> > answers with a score of 0 (even the check mark was next to it).  She
>> >> > also found that if you edit the test and regrade and republish that
>> >> > the score would be fixed but other random students would end up
>> >> > getting a score of 0.
>> >> >
>> >> > I'm at a lost and am trying to figure out how to reproduce this
>> >> > myself.  I have a site that I can see this error where the instructor
>> >> > gave me permission to "play around with".  This is on 2.8.x + some
>> >> > additional features in trunk.
>> >> >
>> >> > -Bryan
>> >> >
>> >> >
>> >> > On Wed, May 2, 2012 at 1:58 AM, Karen Tsao <ktsao at stanford.edu>
>> >> > wrote:
>> >> >> Hi Tom,
>> >> >>
>> >> >> I don't have any idea regarding your issue at this point. But can
>> >> >> you
>> >> >> please
>> >> >> check follow things for me?
>> >> >>
>> >> >> For these students who got 0 points, do their answers exist? That
>> >> >> is,
>> >> >> can
>> >> >> you see their answers from grading pages? Can you also check if
>> >> >> there
>> >> >> are
>> >> >> multiple submissions for these students, from both UI and database.
>> >> >>
>> >> >> Thanks,
>> >> >> Karen
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Tue, May 1, 2012 at 7:03 PM, Tom Pennings
>> >> >> <tom.pennings at gmail.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> We've been experiencing an issue with Samigo 2.6.2 where, even
>> >> >>> though
>> >> >>> all of the students answered all of the questions (some correct,
>> >> >>> some
>> >> >>> incorrect) and successfully submitted (with confirmation) the
>> >> >>> assessment, some of their final scores were recorded as 0.
>> >> >>>
>> >> >>> The problem occurred intermittently (10 - 25% of the cases) on two
>> >> >>> different assessments:
>> >> >>>
>> >> >>>  1. 7 parts, each on its own page, a total of 33 questions, all
>> >> >>> multiple-choice
>> >> >>>  2. 14 parts, all on one page, a total of 50 questions, all
>> >> >>> multiple-choice
>> >> >>>
>> >> >>> So far we've been unable to find the root cause, or a way to
>> >> >>> reproduce. What we did find though is that with 'Each Question is
>> >> >>> on a
>> >> >>> separate Web page' only individual questions would get submitted as
>> >> >>> 0,
>> >> >>> whereas in case of 'The complete Assessment is displayed on one Web
>> >> >>> page' the whole assessment would evaluate as 0, which seems to
>> >> >>> suggest
>> >> >>> an issue during processing of a post though no problems were found
>> >> >>> in
>> >> >>> the logs. We've also tested and ruled out session time outs as the
>> >> >>> reason for the problem.
>> >> >>>
>> >> >>> Any suggestions and advice on how to further debug the problem or
>> >> >>> proposed solutions will be greatly appreciated.
>> >> >>>
>> >> >>>
>> >> >>> Tom
>> >> >>> _______________________________________________
>> >> >>> sakai-dev mailing list
>> >> >>> sakai-dev at collab.sakaiproject.org
>> >> >>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> >> >>>
>> >> >>> TO UNSUBSCRIBE: send email to
>> >> >>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>> >> >>> "unsubscribe"
>> >> >>
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> sakai-dev mailing list
>> >> >> sakai-dev at collab.sakaiproject.org
>> >> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> >> >>
>> >> >> TO UNSUBSCRIBE: send email to
>> >> >> sakai-dev-unsubscribe at collab.sakaiproject.org
>> >> >> with a subject of "unsubscribe"
>> >
>> >
>
>

More information about the sakai-dev mailing list