[WG: Sakai QA] [samigo-team] SAM-973 rested - was Re: [Important] SAM-973 for CLE 2.9.2 ?

Karen Tsao ktsao at stanford.edu
Mon Jul 22 10:50:00 PDT 2013 

This is an existing issue for a long time. So I prefer we take time to test
the fix well before we put it into a release. However, on the other hand, I
don't think the test is complicated. Just that except testing the reported
issue, some overall regression tests need to be done because there are
changes in many files.

Thanks,
Karen


On Fri, Jul 19, 2013 at 6:20 AM, Neal Caidin <neal.caidin at apereo.org> wrote:

> Looping in CLE release team, to help with a determination of this issue
> with respect to 2.9.3 (to keep the merge or back out of 2.9.x until after
> 2.9.3 is released).
>
> Please see below.
>
> On Jul 19, 2013, at 8:10 AM, Neal Caidin <neal.caidin at apereo.org> wrote:
>
> Hi Karen,
>
> Thanks for looking at this one!  This seems like a complicated issue to me.
>
> I was referencing Raúl's scenario, which is that the instructor overwrites
> the score assigned to the student, not the total points available to the
> student.  In his scenario, which seems to me like an edge case but
> demonstrates a point, all the answers are wrong for the Fill in the Blank
> (FIB), but the instructor assigns points to the student anyway, overwriting
> the 0. The behavior was to put green check marks as though all the answers
> are right, and show a distribution of the assigned points across all the
> answers too. So it would stay 10 points for the total (or whatever
> instructor assigned), and all the answers would show correct with green
> checkmark, and with each answer accounting for 2 points each (or assigned
> points/([FIB answer count]). That seems confusing to me, both the green
> checkmarks, which I think the ticket addresses, and the distribution of
> points, which I don't think is addressed. To me that is confusing, but at
> least the student grade is calculated properly. I wouldn't consider this a
> blocker (assuming my analysis is correct in the first place).
>
>
> On a separate but related topic, I am concerned with how difficult it
> appears to have been to merge the fixes into 2.9.x .   Is this a risk to
> the 2.9.3 release? If it is, I wonder if we should back out the changes
> until after 2.9.3 is tagged? I hate to bring up this possibility,
> considering how much work and interest there is in this issue, but 2.9.3
> has been enough of a struggle without adding more to it.  My 2 cents.
>
> Regards,
> Neal
>
>
>
> On Jul 19, 2013, at 12:03 AM, Karen Tsao <ktsao at stanford.edu> wrote:
>
> Hi Neal,
>
> I spent some time to look into the issue you mentioned. Do you mean the
> instructor edits the published assessment and changes the question score
> from 10 to 8? If this is the case, I think it's correct that each answer
> gets 1.6 points and the total score of that question is 4.8. This is the
> same as I saw from current Samigo. Maybe I misunderstand your question?
>
> Thanks,
> Karen
>
>
>
> On Mon, Jun 24, 2013 at 2:25 PM, Karen Tsao <ktsao at stanford.edu> wrote:
>
>> Hi Neal,
>>
>> There are many people comment on this JIRA (and SAM-1994) but I haven't
>> got time to read and digest all of them. So I am afraid to say it's ready
>> to be merged into 2.9.x. I can take a look when I get a chance later. Or
>> maybe other developer can help?
>>
>> Thanks,
>> Karen
>>
>>
>>
>> On Mon, Jun 24, 2013 at 11:55 AM, Neal Caidin <neal.caidin at apereo.org>wrote:
>>
>>> Hi Karen,
>>>
>>> That sounds okay to me. For one thing this does not seem like a blocker
>>> for the 2.9.3 release. And for another I'm still not clear anything needs
>>> fixing or if the existing work that's been done is sufficient and it just
>>> needs to be merged into 2.9.x ?
>>>
>>>
>>> Thanks,
>>> Neal
>>>
>>>
>>>
>>> On Mon, Jun 24, 2013 at 2:45 PM, Karen Tsao <ktsao at stanford.edu> wrote:
>>>
>>>> Hi Neal,
>>>>
>>>> I can try to fix it. But am busy with our local issues. If this is not
>>>> urgent, can I look into it later?
>>>>
>>>> Thanks,
>>>> Karen
>>>>
>>>>
>>>>
>>>> On Mon, Jun 24, 2013 at 8:35 AM, Neal Caidin <neal.caidin at apereo.org>wrote:
>>>>
>>>>> Hi Karen,
>>>>>
>>>>> We retested SAM-973 and as far as I can tell the verification holds up
>>>>> in the GUI. The correct answers have green checkmarks and the incorrect
>>>>> have red x's , even after the score update. And the score update remains
>>>>> fine too (the overall score for the individual question).
>>>>>
>>>>> I think the only issue is how the score for an individual FITB (fill
>>>>> in the blank) question is distributed in the database. Example: if an FITB
>>>>> has 5 items for its answer, the points for the question are 10, and , the
>>>>> student gets three right, then that is 6 points -  (10/5) * 3 .  In the
>>>>> database, 3 of the questions would show two points each. If the instructor
>>>>> overrides the assigned grade to 8 points out of 10, the database stores 1.6
>>>>> points per item over all 5 answers, instead of 2.67 points per item over
>>>>> the 3 items answered correctly. Does it matter how the points are
>>>>> distributed among the items as long as the score for the overall question
>>>>> is correct and the display of correct answers is correct?
>>>>>
>>>>> I assert that if the interface is displaying correctly to the user,
>>>>> then it does no matter how the score is distributed in the database for the
>>>>> items in one question.
>>>>>
>>>>> Am I describing the problem correctly? Would you agree that the
>>>>> storage of the score across items is not a problem with which to be
>>>>> concerned? Are there any APIs that can be affected?
>>>>>
>>>>> Thanks,
>>>>> Neal Caidin
>>>>>
>>>>> Sakai CLE Community Coordinator
>>>>> neal.caidin at apereo.org
>>>>> Skype: nealkdin
>>>>> AIM: ncaidin at aol.com
>>>>>
>>>>>
>>>>> On May 2, 2013, at 2:04 PM, Karen Tsao <ktsao at stanford.edu> wrote:
>>>>>
>>>>> Yeah. Maybe we can get it tested again and record the steps.
>>>>>
>>>>> Thanks,
>>>>> Karen
>>>>>
>>>>>
>>>>> On Thu, May 2, 2013 at 10:21 AM, Neal Caidin <
>>>>> nealcaidin at sakaifoundation.org> wrote:
>>>>>
>>>>>> Hi Karen,
>>>>>>
>>>>>> Maybe I misunderstood then, if it is completely fixed. Perhaps it
>>>>>> needs some more testing. My understanding is that the GUI piece was fixed
>>>>>> but not the part related to grading Fill in the Blank questions.
>>>>>>
>>>>>> Should I ask for QA help to retest and document any steps to
>>>>>> reproduce problems (if any)?
>>>>>>
>>>>>> Thanks,
>>>>>> Neal
>>>>>>
>>>>>> On May 2, 2013, at 1:02 PM, Karen Tsao <ktsao at stanford.edu> wrote:
>>>>>>
>>>>>> Hi Neal,
>>>>>>
>>>>>> Can I know what has not fixed in this JIRA? I thought the issue is
>>>>>> fixed? Maybe I miss something?
>>>>>>
>>>>>> Thanks,
>>>>>> Karen
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, May 2, 2013 at 8:13 AM, Neal Caidin <
>>>>>> nealcaidin at sakaifoundation.org> wrote:
>>>>>>
>>>>>>> The CLE team did not identify this as one of the issues to get
>>>>>>> resolved for RC02 (to be honest, I didn't bring it up based on this thread
>>>>>>> :-p ).
>>>>>>>
>>>>>>> But we still have the original quesiton open about whether this
>>>>>>> should get merged into 2.9.x (after 2.9.2) and a new ticket opened with
>>>>>>> Fill in the Blank grading?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> Neal
>>>>>>>
>>>>>>> On Apr 30, 2013, at 8:50 PM, Anthony Whyte <arwhyte at umich.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>> I see no problem with Neal or the QA team raising potential
>>>>>>> concerns, even between release candidates.  We've got a long standing
>>>>>>> ticket triage process in place on the release team plus two branch managers
>>>>>>> with long experience managing 2.9.x.
>>>>>>>
>>>>>>> Indeed, I find it both refreshing and encouraging that we can
>>>>>>> actually talk of the QA team as an actual entity doing good work.  Like
>>>>>>> Megan though I encourage everyone to consider carefully the cost/benefit to
>>>>>>> the community of delaying a release in order to pull in just one more
>>>>>>> non-blocking fix or a few more like it.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Anth
>>>>>>>
>>>>>>>
>>>>>>> On Apr 30, 2013, at 8:31 PM, Neal Caidin wrote:
>>>>>>>
>>>>>>> Hi Megan,
>>>>>>>
>>>>>>> My thought process is one of gathering information, not making
>>>>>>> decisions. I think there are enough checks and balances in the process that
>>>>>>> hopefully your concerns can be allayed. I see these decisions as ultimately
>>>>>>> to be made by the CLE release team, correct?
>>>>>>>
>>>>>>> You've got a great point and I agree, I would not consider this a
>>>>>>> blocker for 2.9.2. Does that mean we should not consider adding it? There
>>>>>>> are a few more like this, where a bug fix has been made and verified in
>>>>>>> trunk, and it seems like "low hanging fruit" to merge it in and make 2.9.2
>>>>>>> even a little better as we head towards rc02. I think rc02 is almost an
>>>>>>> inevitable need, btw. I can understand if you are worried about scope
>>>>>>> creep.
>>>>>>>
>>>>>>> I would prefer to bring these different issues up, especially as
>>>>>>> they are being brought to my attention by QA testers, and, like I said,
>>>>>>> gather information for input to the CLE release team. We can always choose
>>>>>>> to push the issues to 2.9.3 and then we will have a jump start having
>>>>>>> answered some questions and move some progress on these tickets.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Neal
>>>>>>>
>>>>>>>
>>>>>>> On Apr 30, 2013, at 4:31 PM, "May, Megan Marie" <mmmay at indiana.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Neal,2****
>>>>>>>    I’m really, really, really, really concerned that bugs that are
>>>>>>> not flagged as **blockers**  keep being brought up as candidates
>>>>>>> for release.****
>>>>>>>
>>>>>>> SAM-973 has existed since 2.6.3 . . . why isn’t it being flagged for
>>>>>>> 2.9.3 like some of the other issues that have been raised? ****
>>>>>>>
>>>>>>> Megan****
>>>>>>>
>>>>>>>
>>>>>>> *From:* samigo-team-bounces at collab.sakaiproject.org [mailto:samigo-
>>>>>>> team-bounces at collab.sakaiproject.org] *On Behalf Of *Neal Caidin
>>>>>>> *Sent:* Tuesday, April 30, 2013 4:18 PM
>>>>>>> *To:* Samigo Team
>>>>>>> *Cc:* sakai-qa at collab.sakaiproject.org QA;
>>>>>>> cle-release-team at collab.sakaiproject.org
>>>>>>> *Subject:* [samigo-team] [Important] SAM-973 for CLE 2.9.2 ?
>>>>>>> *Importance:* High****
>>>>>>> ** **
>>>>>>> Dear Samigo Team,****
>>>>>>> ** **
>>>>>>> The QA team reviewed opened Jiras that potentially need to be
>>>>>>> addressed for CLE 2.9.2 (and specifically to get into 2.9.2-rc02, release
>>>>>>> candidate 02).****
>>>>>>> ** **
>>>>>>> We have a question/suggestion for
>>>>>>> https://jira.sakaiproject.org/browse/SAM-973****
>>>>>>> ** **
>>>>>>> We believe that SAM-973 is actually two issues. One is a GUI issue
>>>>>>> which has been fixed and verified in trunk. The GUI issue was sometimes
>>>>>>> displaying incorrect answers as though they were the correct answers (with
>>>>>>> green checkmarks). We think that this issue should have the 2.9.x merge
>>>>>>> flag set and be merged into 2.9.2-rc02.****
>>>>>>> ** **
>>>>>>> The other issue in SAM-973 is a problem with grading Fill in the
>>>>>>> Blank answers. This does not yet appear to be fixed and may or may not need
>>>>>>> additional discussion. For this one we suggest opening a separate ticket.
>>>>>>> ****
>>>>>>> ** **
>>>>>>> Does that make sense to you too? ****
>>>>>>> ** **
>>>>>>> Thanks!****
>>>>>>> ** **
>>>>>>> Neal Caidin****
>>>>>>>
>>>>>>> Sakai CLE Community Coordinator****
>>>>>>> nealcaidin at sakaifoundation.org****
>>>>>>> Skype: nealkdin****
>>>>>>> AIM: ncaidin at aol.com****
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ** **
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> samigo-team mailing list
>>>>>>> samigo-team at collab.sakaiproject.org
>>>>>>> http://collab.sakaiproject.org/mailman/listinfo/samigo-team
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> samigo-team mailing list
>>>>>>> samigo-team at collab.sakaiproject.org
>>>>>>> http://collab.sakaiproject.org/mailman/listinfo/samigo-team
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> samigo-team mailing list
>>>>> samigo-team at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/samigo-team
>>>>>
>>>>>
>>>>
>>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-qa/attachments/20130722/311a018d/attachment-0001.html

More information about the sakai-qa mailing list