[Contrib: Evaluation System] QA Effort for the Evaluation System

Adam Marshall adam.marshall at oucs.ox.ac.uk
Mon Jun 28 04:08:21 PDT 2010


forgot to say we're running with sakai 2.6.2

From: evaluation-bounces at collab.sakaiproject.org [mailto:evaluation-bounces at collab.sakaiproject.org] On Behalf Of Adam Marshall
Sent: 28 June 2010 12:07
To: Matthew Jones; Stephen Marquard
Cc: evaluation at collab.sakaiproject.org
Subject: Re: [Contrib: Evaluation System] QA Effort for the Evaluation System

we're running 68241 (or rather we will be on 6 july) - we could give interested parties usernames to "WebLearn" for the purposes of QA.

adam

From: evaluation-bounces at collab.sakaiproject.org [mailto:evaluation-bounces at collab.sakaiproject.org] On Behalf Of Matthew Jones
Sent: 26 June 2010 19:12
To: Stephen Marquard
Cc: evaluation at collab.sakaiproject.org
Subject: Re: [Contrib: Evaluation System] QA Effort for the Evaluation System

I agree with all of this. Branch should be 1.3.x for now and 1.4.x once we get more stuff into trunk in a few months, like the umich, uct and umd merges.
The only question I had is what should 1.3.x be. Either head (68241), the version longsight lists as using (67522 -32 commits) or the version umd lists as using (63303 -79 commits), or some other version? [1]. Going with a version someone is actually running could be safer as some QA probably had already been done. However it would mean merging in some changes immediately. The tech group should decide and make the 1.3.x branch and first RC tag.

I can then put this RC up on QA6.

Would probably also be useful to get the trunk on nightly2 trunk experimental (or some other server), but we'd need it to build under 2.7 first, which I could work on.

All sounds good to me.

[1] http://confluence.sakaiproject.org/display/EVALSYS/Adopters

On Sat, Jun 26, 2010 at 1:52 PM, Stephen Marquard <stephen.marquard at uct.ac.za<mailto:stephen.marquard at uct.ac.za>> wrote:
Hi all,

Great to see some forward effort, and thanks to Nicola and others volunteering.

I'd like to see an actual maintenance branch, i.e. 1.3.x, and then we do point maintenance releases like 1.3.0, 1.3.1, 1.3.2 (which could include RC candidates) from the maintenance branch rather than trunk.

At some point when enough new features are in trunk, we create new maintenance branches (1.4.x, etc.).

This both lets us do successive point releases with bugfixes only, and insulates production sites from other changes in trunk.

For QA, I'd suggest we follow the mainline Sakai practice, i.e. bugfixes should get merged to a maintenance branch (ideally by someone other than the original developer) when they've been verified and closed by a QA tester. Thus:

- Developer changes issue to Resolved (against trunk) once the bugfix is committed in trunk
- QA tester changes issue to Closed once the fix is confirmed against a QA instance
- Branch manager merges the fix from trunk to the maintenance branch

This may take some updating of the JIRA project to include appropriate versions, and I'm not sure what the implications are for the poms, but as there are other projects that do releases from maintenance branches, I don't think it's too difficult.

Regards
Stephen

>>> Jim Eng <jimeng at umich.edu<mailto:jimeng at umich.edu>> 6/26/2010 3:40 PM >>>
+1 to all points below from Nicola and Aaron.


On Jun 26, 2010, at 2:03 AM, Aaron Zeckoski wrote:

> This all sounds great to me.
> I would suggest the following approach:
>
> 1) Create a branch called 1.3.0-RC
> 2) (not strictly needed) From that branch, create a tag: 1.3.0-RC1
> 3) Merge changes into the branch from trunk
> 4) At some point create a new RC# tag from the branch
> 5) At some other point agree the RC tag is the release
>
> -AZ
>
>
> On Sat, Jun 26, 2010 at 12:07 AM, Nicola Monat-Jacobs
> <nicola at longsight.com<mailto:nicola at longsight.com>> wrote:
>> This week I was given the go-ahead to spend a few hours each week on EvalSys QA coordination efforts, as I offered in Denver. I don't want to step on any toes, but I have some suggestions on how we might precede.
>>
>> I've been following the thread on merging EVALSYS-861 into trunk with interest and I think the suggestions from Adam, Aaron and Jim all make sense and seem to be in line with our conversations from Denver.
>>
>> The real sticking point relates back to whether or not developers can commit to trunk without approval (partial or unanimous) from the newly-anointed PMC, which ultimately relates back to, I think, QA.
>>
>> As we all know, without some oversight, trunk can become hazardous to the health of institutions running EvalSys, but waiting for the 4 very busy people on the PMC to fully review something before check-in risks putting us back where we started: with no forward motion.
>>
>> I'd like to propose the solution offered in Denver by (I think) Stephen M. Let's cut an RC tag right now - it doesn't have to be perfect (should build though :) ), but it will give institutions something to run that isn't trunk and can be the 'go to' check out for those looking to work with EvalSys. It almost doesn't matter which rev it is, waiting is worse at this point then picking the wrong rev, I think.
>>
>> Then, I can work with those groups from Denver who were willing to do QA and develop processes, Test Plans and a repeatable approach for QA going forward.
>>
>> I agree with Aaron that every change (to trunk or whatever) should have a JIRA going forward. I'd also like to request that whoever has the magic to hook our JIRA project up with SVN should perform that particular incantation, if possible.
>>
>> Bug fixes will need to be committed to both trunk and the tag, just like Sakai and other mature projects. I'd like to to take those efforts as a model for ours, if possible. I do have some process questions though, since I'm not intimately familiar with the mechanics of the normal Sakai QA process:
>>
>> Once an RC tag is cut, and patches need to be made, I assume those patches aren't made to the tag (since that sort of goes against the nature of a tag). Are they normally made against a corresponding branch? We don't want to hold up dev work in trunk, but I assume patches will need to be made to graduate from RC to actual release.
>>
>> Additionally, I'm happy to cut the RC tag, if someone can give me permission to do so. I also have limited permissions in JIRA, will probably need to be able to edit versions and assign issues..... I should probably go to Anthony with this, right, unless anyone has any objections?
>>
>> I also think Aaron's most recent email about what types of commits require what kind of communication is great and should be the model we follow, regardless of what form QA takes.
>>
>> Please let me know your thoughts!
>>
>> Thanks,
>>
>> Nicola Monat-Jacobs
>> Senior Technical Manager
>> The Longsight Group
>> nicola at longsight.com<mailto:nicola at longsight.com>



###
UNIVERSITY OF CAPE TOWN

This e-mail is subject to the UCT ICT policies and e-mail disclaimer published on our website at http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 21 650 4500. This e-mail is intended only for the person(s) to whom it is addressed. If the e-mail has reached you in error, please notify the author. If you are not the intended recipient of the e-mail you may not use, disclose, copy, redirect or print the content. If this e-mail is not related to the business of UCT it is sent by the sender in the sender's individual capacity.

###

_______________________________________________
evaluation mailing list
evaluation at collab.sakaiproject.org<mailto:evaluation at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/evaluation

TO UNSUBSCRIBE: send email to evaluation-unsubscribe at collab.sakaiproject.org<mailto:evaluation-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/evaluation/attachments/20100628/b7c6d157/attachment-0001.html 


More information about the evaluation mailing list