[Building Sakai] Samigo problem with valid submissions getting 0 scores

Thu May 31 12:20:39 PDT 2012

Hi Tom,

I don't know what triggered this issue, but I think you can fix the data by
removing the extra records. From your example, you can remove the records
for assessmentgradingid 11731(in both sam_assessmentgrading_t and
sam_itemgrading_t). I don't know why there is no saved/meaningful records
for the other two students. But if their record contains no answer in db, I
guess there is no way to get them back.

Did these students see any errors/warning during the exam? Is this a timed
assessment? I really want to get this issue fixed, if you happen to find a
way to reproduce this, please let me know.

Thanks,
Karen

On Thu, May 31, 2012 at 4:27 AM, Tom Pennings <tom.pennings at gmail.com>wrote:

> Karen,
>
> This is the result from 1 of the 'affected' students:
>
> select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag
> where pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and
> ag.agentid = '3e51ec13-2e0e-43e5-a28f-bbd102b08030';
>
> Returns 2 records:
> '180', 'Final Exam', '151', '', 'comments', '62', '1', '1', '1', '1',
> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-02-23 05:28:18',
> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-04-07 06:44:34',
> '11611', '180', '3e51ec13-2e0e-43e5-a28f-bbd102b08030', '2012-03-27
> 08:25:27', '0', '1', '0', '0', '0', NULL, NULL, NULL, '1', '2012-03-27
> 07:04:29', '0', NULL
>
> '180', 'Final Exam', '151', '', 'comments', '62', '1', '1', '1', '1',
> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-02-23 05:28:18',
> 'c8192151-4016-4fbf-ad69-74da2b0e9093', '2012-04-07 06:44:34',
> '11731', '180', '3e51ec13-2e0e-43e5-a28f-bbd102b08030', NULL, '0',
> '0', NULL, '0', NULL, NULL, NULL, NULL, '0', '2012-03-27 08:29:38',
> '0', NULL
>
>  2 ASSESSMENTGRADINGIDs: 11611 and 11731
>
> select * from sam_itemgrading_t where assessmentgradingid = '11611';
>
> Returns 50 records: attached as sam_itemgrading_t_11611.csv (notice
> SUBMITTEDDATE, PUBLISHEDANSWERID, AUTOSCORE and OVERRIDESCORE are all
> NULL)
>
> select * from sam_itemgrading_t where assessmentgradingid = '11731';
>
> Returns 0 records;
>
>
> Having two rows of ASSESSMENTGRADINGIDs (one with 50 records
> containing 'NULL' for SUBMITTEDDATE and PUBLISHEDANSWERID, and one
> with 0 records) happens to 3 of the 5 affected students, the other 2
> have only one row of ASSESSMENTGRADINGIDs where the sam_itemgrading_t
> has 50 records containing 'NULL' for SUBMITTEDDATE and
> PUBLISHEDANSWERID.
>
> Tom
>
> On Wed, May 30, 2012 at 9:07 PM, Karen Tsao <ktsao at stanford.edu> wrote:
> > Hi Tom,
> >
> > Can you do the following queries for me?
> >
> > For one student:
> >
> > select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag
> where
> > pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and
> ag.agentid
> > = :user_id
> >
> > select * from sam_itemgrading_t where assessmentgradingid =
> > :assessment_grading_id_above
> >
> > Please check if you get multiple records from the first query. Or if
> it's ok
> > to send me the results, I can take a look for you.
> >
> > Thanks,
> > Karen
> >
> >
> >
> > On Wed, May 30, 2012 at 7:30 AM, Tom Pennings <tom.pennings at gmail.com>
> > wrote:
> >>
> >> Karen,
> >>
> >> Sorry for taking time to get back to you, we are currently working
> >> hard at launching our summer classes.
> >>
> >> Using the UI (steps performed to verify described below):
> >>  * For each of the affected students there was a confirmed, recorded
> >> submission with a date, time and auto-scored total score of 0.
> >>  * For each of the affected students none of the answers for the
> >> questions show what was being selected by the student.
> >>  * "Only 1 submission allowed" was set and we were able to confirm
> >> that each of these affected students only had one submission.
> >>
> >> Using the database (query described below):
> >>  * For each of the affected students no submitted answers could be
> >> found in the database
> >>
> >> Do you have any other suggestions that can help in trying to debug this
> >> problem?
> >>
> >> Sincerely,
> >> Tom
> >>
> >> Steps to confirm no recorded answer:
> >> - Log into Sakai
> >> - Click tab of the affected course
> >> - Click "Test & Quizes"
> >> - Under section "Published Assessments/Final Exam" click "Scores"
> >> - Click the name of the user
> >> - For each of the 14 parts verify that for each question no answer for
> >> the student was submitted
> >>
> >> Steps to confirm a single submission:
> >> - Log into Sakai
> >> - Click tab of the affected course
> >> - Click "Test & Quizes"
> >> - Under section "Published Assessments/Final Exam" click "Scores"
> >> - Combo Boxes after "View:" should say "All Sections/Groups" and "All
> >> Submissions"
> >> - Verify that each affected students only appears once
> >>
> >> SQL query to look up submissions linked to a particular test by a
> >> particular student:
> >>  select u.user_id,pt.text as pubtext, pa.text as pubanswer,
> >> g.SUBMITTEDDATE as submitteddate
> >>  from sakai_deanza_final.sakai_user as u
> >>  inner join sakai_deanza_final.sam_assessmentgrading_t as g on
> >> (g.AGENTID = u.USER_ID)
> >>  inner join sakai_deanza_final.sam_publishedassessment_t as a on
> >> (a.ID = g.PUBLISHEDASSESSMENTID)
> >>  inner join sakai_deanza_final.sam_itemgrading_t as gi on
> >> (gi.ASSESSMENTGRADINGID = g.ASSESSMENTGRADINGID)
> >>  inner join sakai_deanza_final.sam_publisheditemtext_t as pt on
> >> (pt.ITEMTEXTID = gi.PUBLISHEDITEMTEXTID)
> >>  inner join sakai_deanza_final.sam_publishedanswer_t as pa on
> >> (pa.ANSWERID = gi.PUBLISHEDANSWERID)
> >>  where u.FIRST_NAME = 'Foo' and u.LAST_NAME = 'Bar'
> >>  and a.TITLE = 'Final Exam'
> >>  and g.SUBMITTEDDATE is not null
> >>  and gi.PUBLISHEDANSWERID is not null;
> >>
> >> On Wed, May 2, 2012 at 6:42 AM, Bryan Holladay <holladay at longsight.com>
> >> wrote:
> >> > It's interesting that you are having this trouble too... I am working
> >> > on an issue just like this.  Its a single page test with a mix of
> >> > random multiple choice and fill in the blank.  The instructor noticed
> >> > that a few random students had a score of 0 and when she looked at the
> >> > submission, the student had quite a few correct multiple choice
> >> > answers with a score of 0 (even the check mark was next to it).  She
> >> > also found that if you edit the test and regrade and republish that
> >> > the score would be fixed but other random students would end up
> >> > getting a score of 0.
> >> >
> >> > I'm at a lost and am trying to figure out how to reproduce this
> >> > myself.  I have a site that I can see this error where the instructor
> >> > gave me permission to "play around with".  This is on 2.8.x + some
> >> > additional features in trunk.
> >> >
> >> > -Bryan
> >> >
> >> >
> >> > On Wed, May 2, 2012 at 1:58 AM, Karen Tsao <ktsao at stanford.edu>
> wrote:
> >> >> Hi Tom,
> >> >>
> >> >> I don't have any idea regarding your issue at this point. But can you
> >> >> please
> >> >> check follow things for me?
> >> >>
> >> >> For these students who got 0 points, do their answers exist? That is,
> >> >> can
> >> >> you see their answers from grading pages? Can you also check if there
> >> >> are
> >> >> multiple submissions for these students, from both UI and database.
> >> >>
> >> >> Thanks,
> >> >> Karen
> >> >>
> >> >>
> >> >>
> >> >> On Tue, May 1, 2012 at 7:03 PM, Tom Pennings <tom.pennings at gmail.com
> >
> >> >> wrote:
> >> >>>
> >> >>> We've been experiencing an issue with Samigo 2.6.2 where, even
> though
> >> >>> all of the students answered all of the questions (some correct,
> some
> >> >>> incorrect) and successfully submitted (with confirmation) the
> >> >>> assessment, some of their final scores were recorded as 0.
> >> >>>
> >> >>> The problem occurred intermittently (10 - 25% of the cases) on two
> >> >>> different assessments:
> >> >>>
> >> >>>  1. 7 parts, each on its own page, a total of 33 questions, all
> >> >>> multiple-choice
> >> >>>  2. 14 parts, all on one page, a total of 50 questions, all
> >> >>> multiple-choice
> >> >>>
> >> >>> So far we've been unable to find the root cause, or a way to
> >> >>> reproduce. What we did find though is that with 'Each Question is
> on a
> >> >>> separate Web page' only individual questions would get submitted as
> 0,
> >> >>> whereas in case of 'The complete Assessment is displayed on one Web
> >> >>> page' the whole assessment would evaluate as 0, which seems to
> suggest
> >> >>> an issue during processing of a post though no problems were found
> in
> >> >>> the logs. We've also tested and ruled out session time outs as the
> >> >>> reason for the problem.
> >> >>>
> >> >>> Any suggestions and advice on how to further debug the problem or
> >> >>> proposed solutions will be greatly appreciated.
> >> >>>
> >> >>>
> >> >>> Tom
> >> >>> _______________________________________________
> >> >>> sakai-dev mailing list
> >> >>> sakai-dev at collab.sakaiproject.org
> >> >>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >> >>>
> >> >>> TO UNSUBSCRIBE: send email to
> >> >>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> >> >>> "unsubscribe"
> >> >>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> sakai-dev mailing list
> >> >> sakai-dev at collab.sakaiproject.org
> >> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >> >>
> >> >> TO UNSUBSCRIBE: send email to
> >> >> sakai-dev-unsubscribe at collab.sakaiproject.org
> >> >> with a subject of "unsubscribe"
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20120531/efcaa6d5/attachment.html 