[Contrib: Evaluation System] QA Effort for the Evaluation System

Matthew Jones jonespm at umich.edu
Sat Jun 26 11:12:25 PDT 2010


I agree with all of this. Branch should be 1.3.x for now and 1.4.x once we
get more stuff into trunk in a few months, like the umich, uct and umd
merges.

The only question I had is what should 1.3.x be. Either head (68241), the
version longsight lists as using (67522 -32 commits) or the version umd
lists as using (63303 -79 commits), or some other version? [1]. Going with a
version someone is actually running could be safer as some QA probably had
already been done. However it would mean merging in some changes
immediately. The tech group should decide and make the 1.3.x branch and
first RC tag.

I can then put this RC up on QA6.

Would probably also be useful to get the trunk on nightly2 trunk
experimental (or some other server), but we'd need it to build under 2.7
first, which I could work on.

All sounds good to me.

[1] http://confluence.sakaiproject.org/display/EVALSYS/Adopters

On Sat, Jun 26, 2010 at 1:52 PM, Stephen Marquard <
stephen.marquard at uct.ac.za> wrote:

> Hi all,
>
> Great to see some forward effort, and thanks to Nicola and others
> volunteering.
>
> I'd like to see an actual maintenance branch, i.e. 1.3.x, and then we do
> point maintenance releases like 1.3.0, 1.3.1, 1.3.2 (which could include RC
> candidates) from the maintenance branch rather than trunk.
>
> At some point when enough new features are in trunk, we create new
> maintenance branches (1.4.x, etc.).
>
> This both lets us do successive point releases with bugfixes only, and
> insulates production sites from other changes in trunk.
>
> For QA, I'd suggest we follow the mainline Sakai practice, i.e. bugfixes
> should get merged to a maintenance branch (ideally by someone other than the
> original developer) when they've been verified and closed by a QA tester.
> Thus:
>
> - Developer changes issue to Resolved (against trunk) once the bugfix is
> committed in trunk
> - QA tester changes issue to Closed once the fix is confirmed against a QA
> instance
> - Branch manager merges the fix from trunk to the maintenance branch
>
> This may take some updating of the JIRA project to include appropriate
> versions, and I'm not sure what the implications are for the poms, but as
> there are other projects that do releases from maintenance branches, I don't
> think it's too difficult.
>
> Regards
> Stephen
>
> >>> Jim Eng <jimeng at umich.edu> 6/26/2010 3:40 PM >>>
> +1 to all points below from Nicola and Aaron.
>
>
> On Jun 26, 2010, at 2:03 AM, Aaron Zeckoski wrote:
>
> > This all sounds great to me.
> > I would suggest the following approach:
> >
> > 1) Create a branch called 1.3.0-RC
> > 2) (not strictly needed) From that branch, create a tag: 1.3.0-RC1
> > 3) Merge changes into the branch from trunk
> > 4) At some point create a new RC# tag from the branch
> > 5) At some other point agree the RC tag is the release
> >
> > -AZ
> >
> >
> > On Sat, Jun 26, 2010 at 12:07 AM, Nicola Monat-Jacobs
> > <nicola at longsight.com> wrote:
> >> This week I was given the go-ahead to spend a few hours each week on
> EvalSys QA coordination efforts, as I offered in Denver. I don't want to
> step on any toes, but I have some suggestions on how we might precede.
> >>
> >> I've been following the thread on merging EVALSYS-861 into trunk with
> interest and I think the suggestions from Adam, Aaron and Jim all make sense
> and seem to be in line with our conversations from Denver.
> >>
> >> The real sticking point relates back to whether or not developers can
> commit to trunk without approval (partial or unanimous) from the
> newly-anointed PMC, which ultimately relates back to, I think, QA.
> >>
> >> As we all know, without some oversight, trunk can become hazardous to
> the health of institutions running EvalSys, but waiting for the 4 very busy
> people on the PMC to fully review something before check-in risks putting us
> back where we started: with no forward motion.
> >>
> >> I'd like to propose the solution offered in Denver by (I think) Stephen
> M. Let's cut an RC tag right now - it doesn't have to be perfect (should
> build though :) ), but it will give institutions something to run that isn't
> trunk and can be the 'go to' check out for those looking to work with
> EvalSys. It almost doesn't matter which rev it is, waiting is worse at this
> point then picking the wrong rev, I think.
> >>
> >> Then, I can work with those groups from Denver who were willing to do QA
> and develop processes, Test Plans and a repeatable approach for QA going
> forward.
> >>
> >> I agree with Aaron that every change (to trunk or whatever) should have
> a JIRA going forward. I'd also like to request that whoever has the magic to
> hook our JIRA project up with SVN should perform that particular
> incantation, if possible.
> >>
> >> Bug fixes will need to be committed to both trunk and the tag, just like
> Sakai and other mature projects. I'd like to to take those efforts as a
> model for ours, if possible. I do have some process questions though, since
> I'm not intimately familiar with the mechanics of the normal Sakai QA
> process:
> >>
> >> Once an RC tag is cut, and patches need to be made, I assume those
> patches aren't made to the tag (since that sort of goes against the nature
> of a tag). Are they normally made against a corresponding branch? We don't
> want to hold up dev work in trunk, but I assume patches will need to be made
> to graduate from RC to actual release.
> >>
> >> Additionally, I'm happy to cut the RC tag, if someone can give me
> permission to do so. I also have limited permissions in JIRA, will probably
> need to be able to edit versions and assign issues..... I should probably go
> to Anthony with this, right, unless anyone has any objections?
> >>
> >> I also think Aaron's most recent email about what types of commits
> require what kind of communication is great and should be the model we
> follow, regardless of what form QA takes.
> >>
> >> Please let me know your thoughts!
> >>
> >> Thanks,
> >>
> >> Nicola Monat-Jacobs
> >> Senior Technical Manager
> >> The Longsight Group
> >> nicola at longsight.com
>
>
>
>
>
> ###
> UNIVERSITY OF CAPE TOWN
>
> This e-mail is subject to the UCT ICT policies and e-mail disclaimer
> published on our website at
> http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from
> +27 21 650 4500. This e-mail is intended only for the person(s) to whom it
> is addressed. If the e-mail has reached you in error, please notify the
> author. If you are not the intended recipient of the e-mail you may not use,
> disclose, copy, redirect or print the content. If this e-mail is not related
> to the business of UCT it is sent by the sender in the sender's individual
> capacity.
>
> ###
>
> _______________________________________________
> evaluation mailing list
> evaluation at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>
> TO UNSUBSCRIBE: send email to
> evaluation-unsubscribe at collab.sakaiproject.org with a subject of
> "unsubscribe"
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/evaluation/attachments/20100626/d1a71900/attachment-0001.html 


More information about the evaluation mailing list