[Contrib: Evaluation System] 1.3.x next (and final?) RC
Beth Kirschner
bkirschn at umich.edu
Wed May 4 06:21:48 PDT 2011
Thanks Adam,
It sounds like you can help with the 1.3 testing effort by documenting the problems you've already found. Additionally, we have several volunteers willing to do some structured testing (Rick, Bryan and UM has someone to help out as well). I had asked for updates to the existing test plan by the end of this week, testing next week, and a decision on releasing v1.3 sometime the week of May 16th. Can you try to add the v1.3 bugs you've found to JIRA by the 16th?
I wouldn't worry too much about your "sins" -- we all share the problem of not enough hours in the day ;-)
Thanks,
- Beth
On May 3, 2011, at 11:18 AM, Adam Marshall wrote:
> We have effectively tested 1.3 in production (at Oxford). We have found quite a few issues, some of which are detailed in Jira and some which we have not had time to report yet.
>
> I don't believe there are any blockers but I do think some of the issues are fairly serious: we are still referring to our implementation as a Beta release! Regardless of this, 1.3 is way way better than 1.2 and we should bite the bullet.
>
> Examples of problems are a plethora of caching issues (esp with MSIE) the worst of which allows one use to see another users' rating scale questions which could potentially compromise sensitive information.
>
> For my sins, I still need to go through "Trunk" and check which of the bugs we fixed in 1.3 are present in "1.4". I also need to do this with the new bugs we've found and I need to propose "new features" and, if they are accepted, get our patches merged. There's just not enough hours in the day! Oh yes and we need to fix the newly discovered bugs.
>
> adam
>
>> -----Original Message-----
>> From: evaluation-bounces at collab.sakaiproject.org [mailto:evaluation-
>> bounces at collab.sakaiproject.org] On Behalf Of Beth Kirschner
>> Sent: 26 April 2011 19:19
>> To: Nicola Monat-Jacobs
>> Cc: evaluation at collab.sakaiproject.org
>> Subject: Re: [Contrib: Evaluation System] 1.3.x next (and final?) RC
>>
>> Hi Nicola,
>>
>> If we do decide to tag and release evaluations v1.3, I think the
>> outstanding questions are:
>> 1) Has there been any structured QA on this release? It sounds
>> like the answer is no, but a rough test plan is in place.
>> 2) Are there any outstanding 'blocker' issues? It sounds like
>> there is one, but the priority is always subject to debate.
>> 3) Has this release been tested in production anywhere? It sounds
>> like the answer is yes.
>>
>> From UM's point of view, I think it would be useful to have some
>> structured QA on this release, so that we can define a 'baseline' to
>> better identify what, if any, issues would be introduced in a future
>> v1.4 evaluations release. Rick Moyer has volunteered to do some QA
>> testing -- is there anyone else?
>>
>> I still think a phone conference would be a good idea, but I'd
>> prefer if we had some structured QA first -- what do others think?
>>
>> - Beth
>>
>> On Apr 25, 2011, at 9:22 PM, Nicola Monat-Jacobs wrote:
>>
>>> Beth, Rick, everyone -
>>>
>>> Apologies on not responding sooner. We do have the start of a QA Test
>> Plan here (really more of an outline at this point):
>>>
>>> https://confluence.sakaiproject.org/display/EVALSYS/Test+Plans
>>>
>>> If we feel like we still need more eyes on this, I can post to the
>> dev list and encourage people to see if they can find any issues.
>>>
>>> Since Lovemore didn't respond to the query about the one blocker
>> associated with 1.3.0 (EVALSYS-1000), I'm going to write him directly
>> about it.
>>>
>>> I can say that almost all our clients are now running the 1.3.x
>> branch and have been for a while and we've encountered no major issues
>> with basic functionality. (If you use very specific functionality in a
>> very specific way YMMV, of course.)
>>>
>>> If you're curious to see the outstanding issues for 1.3.x, take a
>> look here:
>>>
>>>
>> http://jira.sakaiproject.org/secure/IssueNavigator.jspa?reset=true&pid=
>> 10220&resolution=-
>> 1&version=11906&version=11919&sorter/field=issuekey&sorter/order=DESC&t
>> empMax=1000
>>>
>>> I'm happy to tag 1.3.0 tonight if we feel this has gone on long
>> enough and we just need to move on. At this point, 1.3.0 will be far,
>> far more reliable that 1.2.1, so it's an improvement no matter how you
>> slice it.
>>>
>>> Thanks,
>>> Nicola
>>>
>>>
>>> On Apr 19, 2011, at 6:35 AM, Beth Kirschner wrote:
>>>
>>>> Thanks Rick -- I've added some more comments below, but I wonder:
>>>>
>>>> 1) Is anyone planning on deploying 1.3 into production (or even a
>> modified 1.3 evaluations)?
>>>> 2) I think a phone conference would be helpful, but would it be best
>> to do before or after you (and others?!?) are able to do some testing?
>>>>
>>>> - Beth
>>>>
>>>> On Apr 15, 2011, at 3:17 PM, Rick Moyer wrote:
>>>>
>>>>> Beth,
>>>>>
>>>>> I can do some simple things such as create scales, items,
>> templates, hierarchy.
>>>>>
>>>> That would be good -- do we know who would want to be deploying 1.3
>> into production?
>>>>
>>>>> I don't see any course data, forgot to try via adhoc hoc.
>>>>>
>>>> There are some courses already defined on the server (e.g. 'CIS 101
>> A Spring 2011'), and you can always create new courses (demo course
>> sites are defined). The "Evaluations Systems" tool will be
>> automatically added to new users who login, but existing users will
>> have to manually add it to their My Workspace (Worksite Setup -> Edit -
>>> Edit Tools -> Select "Evaluation System" and Continue.
>>>>
>>>> OSP Evaluations are also defined on this server, so be sure not to
>> accidentally select that.
>>>>
>>>>> Any chance of creating a test course?
>>>>>
>>>> See above.
>>>>
>>>>> Rick.
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: evaluation-bounces at collab.sakaiproject.org
>> [mailto:evaluation-bounces at collab.sakaiproject.org] On Behalf Of Beth
>> Kirschner
>>>>> Sent: Friday, April 15, 2011 8:35 AM
>>>>> To: evaluation at collab.sakaiproject.org
>>>>> Subject: Re: [Contrib: Evaluation System] 1.3.x next (and final?)
>> RC
>>>>>
>>>>> The QA5 server has been running the 1.3.0-RC3 tag of evaluations
>> for over a month now -- has anyone had a chance to test? If not, should
>> we have a conference call to coordinate the testing effort? Otherwise,
>> we should probably be discussing any significant issues and then
>> picking a release date.
>>>>>
>>>>> Thanks,
>>>>> - Beth
>>>>>
>>>>> On Mar 4, 2011, at 5:27 PM, Beth Kirschner wrote:
>>>>>
>>>>>> Hi everyone -- the QA5 server has been updated with the latest
>> evaluations tag -- please feel free to take it for a test drive.
>>>>>> http://qa5-us.sakaiproject.org/portal
>>>>>>
>>>>>> - Beth
>>>>>>
>>>>>> On Mar 4, 2011, at 8:26 AM, Beth Kirschner wrote:
>>>>>>
>>>>>>> I'll send a note out when QA5 has been updated with the latest
>> tag.
>>>>>>>
>>>>>>> - Beth
>>>>>>>
>>>>>>> On Mar 2, 2011, at 9:10 PM, Nicola Monat-Jacobs wrote:
>>>>>>>
>>>>>>>> Beth and Everyone -
>>>>>>>>
>>>>>>>> I have now merged all 1.3.x Status = merge JIRAs that made sense
>> to merge in. There are currently two outstanding:
>>>>>>>>
>>>>>>>> EVALSYS-1026 - waiting for more information from Rick, but I
>> believe this should not be merged into 1.3.x and should have it's Fix
>> Status changed to 1.4.x.
>>>>>>>>
>>>>>>>> EVALSYS-870 - Folks suspect this may already be resolved. If
>> testing proves it's not, I can merge in.
>>>>>>>>
>>>>>>>> We still have about 75 issues that are tagged as affecting 1.3.x
>> that remain unresolved and without patches, but the only one that is a
>> blocker is:
>>>>>>>>
>>>>>>>> https://jira.sakaiproject.org/browse/EVALSYS-1000
>>>>>>>>
>>>>>>>> Lovemore, do you have more information on this?
>>>>>>>>
>>>>>>>> I've made a new RC:
>>>>>>>>
>>>>>>>> https://source.sakaiproject.org/contrib//evaluation/tags/1.3.0-
>> RC3/
>>>>>>>>
>>>>>>>> Perhaps this will be the final one if we don't find any major
>> issues.
>>>>>>>>
>>>>>>>> Beth, can you get it up on that UMich test server for others to
>> test? I think it was qa5-us.sakaiproject.org right?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Nicola
>>>>>>>>
>>>>>>>> On Mar 2, 2011, at 10:51 AM, Beth Kirschner wrote:
>>>>>>>>
>>>>>>>>> Hi Nicola and all,
>>>>>>>>>
>>>>>>>>> It looks like there are 11 evaluation issues with a "1.3.x
>> Status" of merge, and of these is resolved (EVALSYS-1026) but not
>> merged to 1.3.x. Does anyone have a good feel for the current status of
>> 1.3.x? Is JIRA up-to-date? Should EVALSYS-1026 be merged and the
>> remaining 10 issues deferred until the 1.4.x release?
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>> - Beth
>>>>>>>>>
>>>>>>>>> On Feb 18, 2011, at 8:30 PM, Nicola Monat-Jacobs wrote:
>>>>>>>>>
>>>>>>>>>> Thanks for your summary Adam, it's very helpful
>>>>>>>>>>
>>>>>>>>>>> EVALSYS-1061
>>>>>>>>>>>
>>>>>>>>>>> Display login screen when a login is needed to see results.
>>>>>>>>>>>
>>>>>>>>>>> * Nicola mentions this but it is still unresolved in Jira &
>> no there is comment about being applied to 1.3
>>>>>>>>>>
>>>>>>>>>> I've resolved it. Also, if you tab over to 'Subversion
>> Commits' you should see my commits to trunk and 1.3.x - sometimes they
>> take a little while show up and/or I think you have to be logged in to
>> see this.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> EVALSYS-1072
>>>>>>>>>>>
>>>>>>>>>>> PDF Report shows response rate of entire evaluation, rather
>> than selected groups
>>>>>>>>>>>
>>>>>>>>>>> * Nicola mentions this but it is still unresolved in Jira; it
>> appears to have been committed to trunk but not 1.3
>>>>>>>>>>
>>>>>>>>>> You should now see my 1.3.x commit under 'Subversion Commits'
>> and I've resolved the ticket, as Will clarified that Rick had already
>> committed the patch to trunk.
>>>>>>>>>>
>>>>>>>>>>> Patches that have been applied but which aren't yet set to
>> resolved
>>>>>>>>>>
>>>>>>>>>> A number of patches have comments from Rick about committing
>> them - he clarified today that he only committed to trunk. So, I'll be
>> committing them to 1.3.x and then setting 1.3.x status = resolved and
>> the JIRA to resolved once I've done so.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> evaluation mailing list
>>>>>>>>>> evaluation at collab.sakaiproject.org
>>>>>>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>>>>>>>
>>>>>>>>>> TO UNSUBSCRIBE: send email to evaluation-
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> evaluation mailing list
>>>>>> evaluation at collab.sakaiproject.org
>>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>>>
>>>>>> TO UNSUBSCRIBE: send email to evaluation-
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> evaluation mailing list
>>>>> evaluation at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>>
>>>>> TO UNSUBSCRIBE: send email to evaluation-
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> evaluation mailing list
>>>> evaluation at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>>>
>>>> TO UNSUBSCRIBE: send email to evaluation-
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>>
>>>
>>>
>>
>> _______________________________________________
>> evaluation mailing list
>> evaluation at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>
>> TO UNSUBSCRIBE: send email to evaluation-
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>
More information about the evaluation
mailing list