[Contrib: Evaluation System] 1.3.x next (and final?) RC

Adam Marshall adam.marshall at oucs.ox.ac.uk
Fri May 6 10:04:05 PDT 2011 

I have now logged all the bugs that i know about.

adam

> -----Original Message-----
> From: Beth Kirschner [mailto:bkirschn at umich.edu]
> Sent: 04 May 2011 14:22
> To: Adam Marshall
> Cc: Nicola Monat-Jacobs; evaluation at collab.sakaiproject.org
> Subject: Re: [Contrib: Evaluation System] 1.3.x next (and final?) RC
> 
> Thanks Adam,
> 
>    It sounds like you can help with the 1.3 testing effort by
> documenting the problems you've already found. Additionally, we have
> several volunteers willing to do some structured testing (Rick, Bryan
> and UM has someone to help out as well). I had asked for updates to the
> existing test plan by the end of this week, testing next week, and a
> decision on releasing v1.3 sometime the week of May 16th. Can you try
> to add the v1.3 bugs you've found to JIRA by the 16th?
> 
>    I wouldn't worry too much about your "sins" -- we all share the
> problem of not enough hours in the day ;-)
> 
> Thanks,
> - Beth
> 
> On May 3, 2011, at 11:18 AM, Adam Marshall wrote:
> 
> > We have effectively tested 1.3 in production (at Oxford). We have
> found quite a few issues, some of which are detailed in Jira and some
> which we have not had time to report yet.
> >
> > I don't believe there are any blockers but I do think some of the
> issues are fairly serious: we are still referring to our implementation
> as a Beta release! Regardless of this, 1.3 is way way better than 1.2
> and we should bite the bullet.
> >
> > Examples of problems are a plethora of caching issues (esp with MSIE)
> the worst of which allows one use to see another users' rating scale
> questions which could potentially compromise sensitive information.
> >
> > For my sins, I still need to go through "Trunk" and check which of
> the bugs we fixed in 1.3 are present in "1.4". I also need to do this
> with the new bugs we've found and I need to propose "new features" and,
> if they are accepted, get our patches merged. There's just not enough
> hours in the day! Oh yes and we need to fix the newly discovered bugs.
> >
> > adam
> >
> >> -----Original Message-----
> >> From: evaluation-bounces at collab.sakaiproject.org [mailto:evaluation-
> >> bounces at collab.sakaiproject.org] On Behalf Of Beth Kirschner
> >> Sent: 26 April 2011 19:19
> >> To: Nicola Monat-Jacobs
> >> Cc: evaluation at collab.sakaiproject.org
> >> Subject: Re: [Contrib: Evaluation System] 1.3.x next (and final?) RC
> >>
> >> Hi Nicola,
> >>
> >>   If we do decide to tag and release evaluations v1.3, I think the
> >> outstanding questions are:
> >> 	1) Has there been any structured QA on this release? It sounds
> >> like the answer is no, but a rough test plan is in place.
> >> 	2) Are there any outstanding 'blocker' issues? It sounds like
> >> there is one, but the priority is always subject to debate.
> >> 	3) Has this release been tested in production anywhere? It sounds
> >> like the answer is yes.
> >>
> >>   From UM's point of view, I think it would be useful to have some
> >> structured QA on this release, so that we can define a 'baseline' to
> >> better identify what, if any, issues would be introduced in a future
> >> v1.4 evaluations release.  Rick Moyer has volunteered to do some QA
> >> testing -- is there anyone else?
> >>
> >>   I still think a phone conference would be a good idea, but I'd
> >> prefer if we had some structured QA first -- what do others think?
> >>
> >> - Beth
> >>
> >> On Apr 25, 2011, at 9:22 PM, Nicola Monat-Jacobs wrote:
> >>
> >>> Beth, Rick, everyone -
> >>>
> >>> Apologies on not responding sooner. We do have the start of a QA
> Test
> >> Plan here (really more of an outline at this point):
> >>>
> >>> https://confluence.sakaiproject.org/display/EVALSYS/Test+Plans
> >>>
> >>> If we feel like we still need more eyes on this, I can post to the
> >> dev list and encourage people to see if they can find any issues.
> >>>
> >>> Since Lovemore didn't respond to the query about the one blocker
> >> associated with 1.3.0 (EVALSYS-1000), I'm going to write him
> directly
> >> about it.
> >>>
> >>> I can say that almost all our clients are now running the 1.3.x
> >> branch and have been for a while and we've encountered no major
> issues
> >> with basic functionality. (If you use very specific functionality in
> a
> >> very specific way YMMV, of course.)
> >>>
> >>> If you're curious to see the outstanding issues for 1.3.x, take a
> >> look here:
> >>>
> >>>
> >>
> http://jira.sakaiproject.org/secure/IssueNavigator.jspa?reset=true&pid=
> >> 10220&resolution=-
> >>
> 1&version=11906&version=11919&sorter/field=issuekey&sorter/order=DESC&t
> >> empMax=1000
> >>>
> >>> I'm happy to tag 1.3.0 tonight if we feel this has gone on long
> >> enough and we just need to move on. At this point, 1.3.0 will be
> far,
> >> far more reliable that 1.2.1, so it's an improvement no matter how
> you
> >> slice it.
> >>>
> >>> Thanks,
> >>> Nicola
> >>>
> >>>
> >>> On Apr 19, 2011, at  6:35 AM, Beth Kirschner wrote:
> >>>
> >>>> Thanks Rick -- I've added some more comments below, but I wonder:
> >>>>
> >>>> 1) Is anyone planning on deploying 1.3 into production (or even a
> >> modified 1.3 evaluations)?
> >>>> 2) I think a phone conference would be helpful, but would it be
> best
> >> to do before or after you (and others?!?) are able to do some
> testing?
> >>>>
> >>>> - Beth
> >>>>
> >>>> On Apr 15, 2011, at 3:17 PM, Rick Moyer wrote:
> >>>>
> >>>>> Beth,
> >>>>>
> >>>>> 	I can do some simple things such as create scales, items,
> >> templates, hierarchy.
> >>>>>
> >>>> That would be good -- do we know who would want to be deploying
> 1.3
> >> into production?
> >>>>
> >>>>> 	I don't see any course data, forgot to try via adhoc hoc.
> >>>>>
> >>>> There are some courses already defined on the server (e.g. 'CIS
> 101
> >> A Spring 2011'), and you can always create new courses (demo course
> >> sites are defined). The "Evaluations Systems" tool will be
> >> automatically added to new users who login, but existing users will
> >> have to manually add it to their My Workspace (Worksite Setup ->
> Edit -
> >>> Edit Tools -> Select "Evaluation System" and Continue.
> >>>>
> >>>> OSP Evaluations are also defined on this server, so be sure not to
> >> accidentally select that.
> >>>>
> >>>>> 	Any chance of creating a test course?
> >>>>>
> >>>> See above.
> >>>>
> >>>>> Rick.
> >>>>>
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: evaluation-bounces at collab.sakaiproject.org
> >> [mailto:evaluation-bounces at collab.sakaiproject.org] On Behalf Of
> Beth
> >> Kirschner
> >>>>> Sent: Friday, April 15, 2011 8:35 AM
> >>>>> To: evaluation at collab.sakaiproject.org
> >>>>> Subject: Re: [Contrib: Evaluation System] 1.3.x next (and final?)
> >> RC
> >>>>>
> >>>>> The QA5 server has been running the 1.3.0-RC3 tag of evaluations
> >> for over a month now -- has anyone had a chance to test? If not,
> should
> >> we have a conference call to coordinate the testing effort?
> Otherwise,
> >> we should probably be discussing any significant issues and then
> >> picking a release date.
> >>>>>
> >>>>> Thanks,
> >>>>> - Beth
> >>>>>
> >>>>> On Mar 4, 2011, at 5:27 PM, Beth Kirschner wrote:
> >>>>>
> >>>>>> Hi everyone -- the QA5 server has been updated with the latest
> >> evaluations tag -- please feel free to take it for a test drive.
> >>>>>> 	http://qa5-us.sakaiproject.org/portal
> >>>>>>
> >>>>>> - Beth
> >>>>>>
> >>>>>> On Mar 4, 2011, at 8:26 AM, Beth Kirschner wrote:
> >>>>>>
> >>>>>>> I'll send a note out when QA5 has been updated with the latest
> >> tag.
> >>>>>>>
> >>>>>>> - Beth
> >>>>>>>
> >>>>>>> On Mar 2, 2011, at 9:10 PM, Nicola Monat-Jacobs wrote:
> >>>>>>>
> >>>>>>>> Beth and Everyone -
> >>>>>>>>
> >>>>>>>> I have now merged all 1.3.x Status = merge JIRAs that made
> sense
> >> to merge in. There are currently two outstanding:
> >>>>>>>>
> >>>>>>>> EVALSYS-1026 - waiting for more information from Rick, but I
> >> believe this should not be merged into 1.3.x and should have it's
> Fix
> >> Status changed to 1.4.x.
> >>>>>>>>
> >>>>>>>> EVALSYS-870 - Folks suspect this may already be resolved. If
> >> testing proves it's not, I can merge in.
> >>>>>>>>
> >>>>>>>> We still have about 75 issues that are tagged as affecting
> 1.3.x
> >> that remain unresolved and without patches, but the only one that is
> a
> >> blocker is:
> >>>>>>>>
> >>>>>>>> https://jira.sakaiproject.org/browse/EVALSYS-1000
> >>>>>>>>
> >>>>>>>> Lovemore, do you have more information on this?
> >>>>>>>>
> >>>>>>>> I've made a new RC:
> >>>>>>>>
> >>>>>>>>
> https://source.sakaiproject.org/contrib//evaluation/tags/1.3.0-
> >> RC3/
> >>>>>>>>
> >>>>>>>> Perhaps this will be the final one if we don't find any major
> >> issues.
> >>>>>>>>
> >>>>>>>> Beth, can you get it up on that UMich test server for others
> to
> >> test? I think it was qa5-us.sakaiproject.org right?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Nicola
> >>>>>>>>
> >>>>>>>> On Mar 2, 2011, at  10:51 AM, Beth Kirschner wrote:
> >>>>>>>>
> >>>>>>>>> Hi Nicola and all,
> >>>>>>>>>
> >>>>>>>>> It looks like there are 11 evaluation issues with a "1.3.x
> >> Status" of merge, and of these is resolved (EVALSYS-1026) but not
> >> merged to 1.3.x. Does anyone have a good feel for the current status
> of
> >> 1.3.x? Is JIRA up-to-date? Should EVALSYS-1026 be merged and the
> >> remaining 10 issues deferred until the 1.4.x release?
> >>>>>>>>>
> >>>>>>>>> Thoughts?
> >>>>>>>>> - Beth
> >>>>>>>>>
> >>>>>>>>> On Feb 18, 2011, at 8:30 PM, Nicola Monat-Jacobs wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks for your summary Adam, it's very helpful
> >>>>>>>>>>
> >>>>>>>>>>> EVALSYS-1061
> >>>>>>>>>>>
> >>>>>>>>>>> Display login screen when a login is needed to see results.
> >>>>>>>>>>>
> >>>>>>>>>>> * Nicola mentions this but it is still unresolved in Jira &
> >> no there is comment about being applied to 1.3
> >>>>>>>>>>
> >>>>>>>>>> I've resolved it. Also, if you tab over to 'Subversion
> >> Commits' you should see my commits to trunk and 1.3.x - sometimes
> they
> >> take a little while show up and/or I think you have to be logged in
> to
> >> see this.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> EVALSYS-1072
> >>>>>>>>>>>
> >>>>>>>>>>> PDF Report shows response rate of entire evaluation, rather
> >> than selected groups
> >>>>>>>>>>>
> >>>>>>>>>>> * Nicola mentions this but it is still unresolved in Jira;
> it
> >> appears to have been committed to trunk but not 1.3
> >>>>>>>>>>
> >>>>>>>>>> You should now see my 1.3.x commit under 'Subversion
> Commits'
> >> and I've resolved the ticket, as Will clarified that Rick had
> already
> >> committed the patch to trunk.
> >>>>>>>>>>
> >>>>>>>>>>> Patches that have been applied but which aren't yet set to
> >> resolved
> >>>>>>>>>>
> >>>>>>>>>> A number of patches have comments from Rick about committing
> >> them - he clarified today that he only committed to trunk. So, I'll
> be
> >> committing them to 1.3.x and then setting 1.3.x status = resolved
> and
> >> the JIRA to resolved once I've done so.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> evaluation mailing list
> >>>>>>>>>> evaluation at collab.sakaiproject.org
> >>>>>>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
> >>>>>>>>>>
> >>>>>>>>>> TO UNSUBSCRIBE: send email to evaluation-
> >> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> evaluation mailing list
> >>>>>> evaluation at collab.sakaiproject.org
> >>>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
> >>>>>>
> >>>>>> TO UNSUBSCRIBE: send email to evaluation-
> >> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> evaluation mailing list
> >>>>> evaluation at collab.sakaiproject.org
> >>>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
> >>>>>
> >>>>> TO UNSUBSCRIBE: send email to evaluation-
> >> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> >>>>>
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> evaluation mailing list
> >>>> evaluation at collab.sakaiproject.org
> >>>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
> >>>>
> >>>> TO UNSUBSCRIBE: send email to evaluation-
> >> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> >>>
> >>>
> >>>
> >>
> >> _______________________________________________
> >> evaluation mailing list
> >> evaluation at collab.sakaiproject.org
> >> http://collab.sakaiproject.org/mailman/listinfo/evaluation
> >>
> >> TO UNSUBSCRIBE: send email to evaluation-
> >> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> >
> >

More information about the evaluation mailing list