[Building Sakai] Samigo problem with valid submissions getting 0 scores

[Tom Pennings]

Karen,

Sorry for taking time to get back to you, we are currently working
hard at launching our summer classes.

Using the UI (steps performed to verify described below):
  * For each of the affected students there was a confirmed, recorded
submission with a date, time and auto-scored total score of 0.
  * For each of the affected students none of the answers for the
questions show what was being selected by the student.
  * "Only 1 submission allowed" was set and we were able to confirm
that each of these affected students only had one submission.

Using the database (query described below):
  * For each of the affected students no submitted answers could be
found in the database

Do you have any other suggestions that can help in trying to debug this problem?

Sincerely,
Tom

Steps to confirm no recorded answer:
- Log into Sakai
- Click tab of the affected course
- Click "Test & Quizes"
- Under section "Published Assessments/Final Exam" click "Scores"
- Click the name of the user
- For each of the 14 parts verify that for each question no answer for
the student was submitted

Steps to confirm a single submission:
- Log into Sakai
- Click tab of the affected course
- Click "Test & Quizes"
- Under section "Published Assessments/Final Exam" click "Scores"
- Combo Boxes after "View:" should say "All Sections/Groups" and "All
Submissions"
- Verify that each affected students only appears once

SQL query to look up submissions linked to a particular test by a
particular student:
  select u.user_id,pt.text as pubtext, pa.text as pubanswer,
g.SUBMITTEDDATE as submitteddate
  from sakai_deanza_final.sakai_user as u
  inner join sakai_deanza_final.sam_assessmentgrading_t as g on
(g.AGENTID = u.USER_ID)
  inner join sakai_deanza_final.sam_publishedassessment_t as a on
(a.ID = g.PUBLISHEDASSESSMENTID)
  inner join sakai_deanza_final.sam_itemgrading_t as gi on
(gi.ASSESSMENTGRADINGID = g.ASSESSMENTGRADINGID)
  inner join sakai_deanza_final.sam_publisheditemtext_t as pt on
(pt.ITEMTEXTID = gi.PUBLISHEDITEMTEXTID)
  inner join sakai_deanza_final.sam_publishedanswer_t as pa on
(pa.ANSWERID = gi.PUBLISHEDANSWERID)
  where u.FIRST_NAME = 'Foo' and u.LAST_NAME = 'Bar'
  and a.TITLE = 'Final Exam'
  and g.SUBMITTEDDATE is not null
  and gi.PUBLISHEDANSWERID is not null;

On Wed, May 2, 2012 at 6:42 AM, Bryan Holladay <holladay at longsight.com> wrote:
> It's interesting that you are having this trouble too... I am working
> on an issue just like this.  Its a single page test with a mix of
> random multiple choice and fill in the blank.  The instructor noticed
> that a few random students had a score of 0 and when she looked at the
> submission, the student had quite a few correct multiple choice
> answers with a score of 0 (even the check mark was next to it).  She
> also found that if you edit the test and regrade and republish that
> the score would be fixed but other random students would end up
> getting a score of 0.
>
> I'm at a lost and am trying to figure out how to reproduce this
> myself.  I have a site that I can see this error where the instructor
> gave me permission to "play around with".  This is on 2.8.x + some
> additional features in trunk.
>
> -Bryan
>
>
> On Wed, May 2, 2012 at 1:58 AM, Karen Tsao <ktsao at stanford.edu> wrote:
>> Hi Tom,
>>
>> I don't have any idea regarding your issue at this point. But can you please
>> check follow things for me?
>>
>> For these students who got 0 points, do their answers exist? That is, can
>> you see their answers from grading pages? Can you also check if there are
>> multiple submissions for these students, from both UI and database.
>>
>> Thanks,
>> Karen
>>
>>
>>
>> On Tue, May 1, 2012 at 7:03 PM, Tom Pennings <tom.pennings at gmail.com> wrote:
>>>
>>> We've been experiencing an issue with Samigo 2.6.2 where, even though
>>> all of the students answered all of the questions (some correct, some
>>> incorrect) and successfully submitted (with confirmation) the
>>> assessment, some of their final scores were recorded as 0.
>>>
>>> The problem occurred intermittently (10 - 25% of the cases) on two
>>> different assessments:
>>>
>>>  1. 7 parts, each on its own page, a total of 33 questions, all
>>> multiple-choice
>>>  2. 14 parts, all on one page, a total of 50 questions, all
>>> multiple-choice
>>>
>>> So far we've been unable to find the root cause, or a way to
>>> reproduce. What we did find though is that with 'Each Question is on a
>>> separate Web page' only individual questions would get submitted as 0,
>>> whereas in case of 'The complete Assessment is displayed on one Web
>>> page' the whole assessment would evaluate as 0, which seems to suggest
>>> an issue during processing of a post though no problems were found in
>>> the logs. We've also tested and ruled out session time outs as the
>>> reason for the problem.
>>>
>>> Any suggestions and advice on how to further debug the problem or
>>> proposed solutions will be greatly appreciated.
>>>
>>>
>>> Tom
>>> _______________________________________________
>>> sakai-dev mailing list
>>> sakai-dev at collab.sakaiproject.org
>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>
>>> TO UNSUBSCRIBE: send email to
>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>> "unsubscribe"
>>
>>
>>
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"