[samigo-team] question about Data Discrepancy errors in Samigo

[Ward, Lynn E.]

Agreed.  I've added this to the agenda in confluence.

Lynn
==========================
Lynn Ward, Principal Systems Analyst, Instructional Technology Systems and Services
University Information Technology Services<http://uits.iu.edu/>
Indiana University-Purdue University Indianapolis
Information Technology and Communications Complex (IT 225D)
535 West Michigan Street, Indianapolis, IN 46202
Phone: 317-278-5713  E-mail: leward at iupui.edu<mailto:leward at iupui.edu>

From: jamai <jamai at stanford.edu<mailto:jamai at stanford.edu>>
Date: Tue, 13 Sep 2011 16:30:37 -0700
To: Lynn Ward <leward at iupui.edu<mailto:leward at iupui.edu>>
Cc: Samigo Team <samigo-team at collab.sakaiproject.org<mailto:samigo-team at collab.sakaiproject.org>>
Subject: Re: [samigo-team] question about Data Discrepancy errors in Samigo

Hi Lynn,

Data discrepancy has been generating a lot of attention lately (SAM-1088, SAM-1330) and your message below being the latest contribution.

So far, the conditions that I am aware of that trigger data discrepancy error (DDE) are:
1. Student takes an assessment in multiple windows
2. In a timed assessment, time is up. Back-end auto-submit job submits last saved student work. However, server timer and browser timer may be a few seconds off, which would allow for front-end submission (manually by student or automatically via javascript) to take place after the back-end job has run. DDE
3. In a non-timed assessment, due date has passed. auto-submit job is runs. However, student is able to manually submit after the auto-submit job has run. Student clicks Submit and sees DDE. (Bryan Holladay provided steps to reproduce)
Two windows:
instructor: Set up test
student: Log in as student
student: Take part of the test and stay in the test
instructor) Wait till the due date is past, then run the auto-submit job (via admin)
student: Try to finish the test (remember to not leave the test)
student: Data discrepancy error

SAM-1088 is working on resolving DDE for auto-submission of non-timed assessments
SAM-1330 proposes a solution of turning off the system check for data discrepancies (it's a setting that could be enabled on a per institution basis).
Questions:
-Would turning off this setting across the board be desirable? Are there instances where such a check would be desirable?
-In cases where manual/front-end submission takes place after automatic submission job has run and data discrepancy check is turned off, manual submission would override whatever was submitted by auto submission. Are there any cases where this would not be desirable?

Discussion of SAM-1330 and any other issues related to DDE would be a good agenda item at the next SAMigo conference call. At least, the topic could be raised and those interested in working on it further could form a working group. How does this sound?

Thanks,
Jackie

Ward, Lynn E. wrote:
Hi all,

One of the top three development priorities for several Samigo institutions is reducing the occurrence of data discrepancy errors.  This is a concern for IU as well, as many of our users have expressed both concern and confusion when they get the error--particularly because the possible causes cited in the error message are almost never at play when the message appears.  I am able to force the error to occur in most browser by clicking the browser back button and then attempting to save or click next.  However, rarely when I receive the error is their actually a discrepancy between what I entered in the test and what has been saved.  So, I'd like to know more about the error and what causes the error to occur.

What is the actual underlying condition or trigger that results in the error?

When students get this error, should they be concerned their entire assessment  or just the data they entered since they last saved?

We'd like to update the error message to more accurately reflect the cause and scope of the discrepancy, as well as steps to take to verify that the saved or submitted version is in fact correct.

Lynn



==========================
Lynn Ward, Principal Systems Analyst, Instructional Technology Systems and Services
University Information Technology Services<http://uits.iu.edu/>
Indiana University-Purdue University Indianapolis
Information Technology and Communications Complex (IT 225D)
535 West Michigan Street, Indianapolis, IN 46202
Phone: 317-278-5713  E-mail: leward at iupui.edu<mailto:leward at iupui.edu>

________________________________

_______________________________________________
samigo-team mailing list
samigo-team at collab.sakaiproject.org<mailto:samigo-team at collab.sakaiproject.org>http://collab.sakaiproject.org/mailman/listinfo/samigo-team


--
Jacqueline Mai Krueger
User Experience Specialist
Academic Computing Services - CourseWork
Stanford University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/samigo-team/attachments/20110915/7d3c2438/attachment.html