[Building Sakai] Samigo problem with valid submissions getting 0 scores

[Karen Tsao]

Hi Tom,

Can you do the following queries for me?

For one student:

select * from sam_publishedassessment_t pa, sam_assessmentgrading_t ag
where pa.id = ag.publishedassessmentid and pa.title = 'Final Exam' and
ag.agentid = :user_id

select * from sam_itemgrading_t where assessmentgradingid =
:assessment_grading_id_above

Please check if you get multiple records from the first query. Or if it's
ok to send me the results, I can take a look for you.

Thanks,
Karen


On Wed, May 30, 2012 at 7:30 AM, Tom Pennings <tom.pennings at gmail.com>wrote:

> Karen,
>
> Sorry for taking time to get back to you, we are currently working
> hard at launching our summer classes.
>
> Using the UI (steps performed to verify described below):
>  * For each of the affected students there was a confirmed, recorded
> submission with a date, time and auto-scored total score of 0.
>  * For each of the affected students none of the answers for the
> questions show what was being selected by the student.
>  * "Only 1 submission allowed" was set and we were able to confirm
> that each of these affected students only had one submission.
>
> Using the database (query described below):
>  * For each of the affected students no submitted answers could be
> found in the database
>
> Do you have any other suggestions that can help in trying to debug this
> problem?
>
> Sincerely,
> Tom
>
> Steps to confirm no recorded answer:
> - Log into Sakai
> - Click tab of the affected course
> - Click "Test & Quizes"
> - Under section "Published Assessments/Final Exam" click "Scores"
> - Click the name of the user
> - For each of the 14 parts verify that for each question no answer for
> the student was submitted
>
> Steps to confirm a single submission:
> - Log into Sakai
> - Click tab of the affected course
> - Click "Test & Quizes"
> - Under section "Published Assessments/Final Exam" click "Scores"
> - Combo Boxes after "View:" should say "All Sections/Groups" and "All
> Submissions"
> - Verify that each affected students only appears once
>
> SQL query to look up submissions linked to a particular test by a
> particular student:
>  select u.user_id,pt.text as pubtext, pa.text as pubanswer,
> g.SUBMITTEDDATE as submitteddate
>  from sakai_deanza_final.sakai_user as u
>  inner join sakai_deanza_final.sam_assessmentgrading_t as g on
> (g.AGENTID = u.USER_ID)
>  inner join sakai_deanza_final.sam_publishedassessment_t as a on
> (a.ID = g.PUBLISHEDASSESSMENTID)
>  inner join sakai_deanza_final.sam_itemgrading_t as gi on
> (gi.ASSESSMENTGRADINGID = g.ASSESSMENTGRADINGID)
>  inner join sakai_deanza_final.sam_publisheditemtext_t as pt on
> (pt.ITEMTEXTID = gi.PUBLISHEDITEMTEXTID)
>  inner join sakai_deanza_final.sam_publishedanswer_t as pa on
> (pa.ANSWERID = gi.PUBLISHEDANSWERID)
>  where u.FIRST_NAME = 'Foo' and u.LAST_NAME = 'Bar'
>  and a.TITLE = 'Final Exam'
>  and g.SUBMITTEDDATE is not null
>  and gi.PUBLISHEDANSWERID is not null;
>
> On Wed, May 2, 2012 at 6:42 AM, Bryan Holladay <holladay at longsight.com>
> wrote:
> > It's interesting that you are having this trouble too... I am working
> > on an issue just like this.  Its a single page test with a mix of
> > random multiple choice and fill in the blank.  The instructor noticed
> > that a few random students had a score of 0 and when she looked at the
> > submission, the student had quite a few correct multiple choice
> > answers with a score of 0 (even the check mark was next to it).  She
> > also found that if you edit the test and regrade and republish that
> > the score would be fixed but other random students would end up
> > getting a score of 0.
> >
> > I'm at a lost and am trying to figure out how to reproduce this
> > myself.  I have a site that I can see this error where the instructor
> > gave me permission to "play around with".  This is on 2.8.x + some
> > additional features in trunk.
> >
> > -Bryan
> >
> >
> > On Wed, May 2, 2012 at 1:58 AM, Karen Tsao <ktsao at stanford.edu> wrote:
> >> Hi Tom,
> >>
> >> I don't have any idea regarding your issue at this point. But can you
> please
> >> check follow things for me?
> >>
> >> For these students who got 0 points, do their answers exist? That is,
> can
> >> you see their answers from grading pages? Can you also check if there
> are
> >> multiple submissions for these students, from both UI and database.
> >>
> >> Thanks,
> >> Karen
> >>
> >>
> >>
> >> On Tue, May 1, 2012 at 7:03 PM, Tom Pennings <tom.pennings at gmail.com>
> wrote:
> >>>
> >>> We've been experiencing an issue with Samigo 2.6.2 where, even though
> >>> all of the students answered all of the questions (some correct, some
> >>> incorrect) and successfully submitted (with confirmation) the
> >>> assessment, some of their final scores were recorded as 0.
> >>>
> >>> The problem occurred intermittently (10 - 25% of the cases) on two
> >>> different assessments:
> >>>
> >>>  1. 7 parts, each on its own page, a total of 33 questions, all
> >>> multiple-choice
> >>>  2. 14 parts, all on one page, a total of 50 questions, all
> >>> multiple-choice
> >>>
> >>> So far we've been unable to find the root cause, or a way to
> >>> reproduce. What we did find though is that with 'Each Question is on a
> >>> separate Web page' only individual questions would get submitted as 0,
> >>> whereas in case of 'The complete Assessment is displayed on one Web
> >>> page' the whole assessment would evaluate as 0, which seems to suggest
> >>> an issue during processing of a post though no problems were found in
> >>> the logs. We've also tested and ruled out session time outs as the
> >>> reason for the problem.
> >>>
> >>> Any suggestions and advice on how to further debug the problem or
> >>> proposed solutions will be greatly appreciated.
> >>>
> >>>
> >>> Tom
> >>> _______________________________________________
> >>> sakai-dev mailing list
> >>> sakai-dev at collab.sakaiproject.org
> >>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>>
> >>> TO UNSUBSCRIBE: send email to
> >>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
> >>> "unsubscribe"
> >>
> >>
> >>
> >> _______________________________________________
> >> sakai-dev mailing list
> >> sakai-dev at collab.sakaiproject.org
> >> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> >>
> >> TO UNSUBSCRIBE: send email to
> sakai-dev-unsubscribe at collab.sakaiproject.org
> >> with a subject of "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20120530/0ae3ec23/attachment.html