[WG: Sakai QA] Request for inclusion in 2.6.1: SAK-16416
Stephen Marquard
stephen.marquard at uct.ac.za
Fri Jun 5 08:37:17 PDT 2009
I'm not convinced that the "selective component" approach to maintenance releases is really useful. To me it adds a lot of complexity, and means that the maintenance releases bear no resemblance to the 2-6-x branches that people are running in production. It's confusing when an issue is fixed in the maintenance branch but not in a release which post-dates the fix.
I would rather see the testing happen as and when issues are moved into the 2-6-x branch, so that it's safe to release a 2.6.x from the maintenance branch at any time.
The only exception to this IMO should be security releases, which are strictly the last .x release plus the security fixes, so that they can be done immediately.
Also in general I think we'd be better off putting effort into building more automated tests than trying to get a diffuse pool of volunteers to do more regression testing to a release schedule.
Regards
Stephen
>>> Michael Korcuska <mkorcuska at sakaifoundation.org> 06/05/09 4:19 PM >>>
I'm not going to get into the procedures (that's up to the QA group),
but let me outline a few things we've discussed in the past:
* QA is a (possible the most important) bounding factor on how many
and which issues can going into a maintenance release. If we try to
include everything that is fixed we create a large QA testing effort.
Especially regression testing. This delays maintenance releases. So
there is an important balance between the number of things we include
and the speed with which we get maintenance releases out the door. The
more issues fixed, the longer it will take.
* It is my view that a maintenance release that introduces regressions
is a very bad thing. So they need to be well tested. Unfortunately the
lack of unit test coverage (or automated testing of any kind) in the
2.x code base makes regression testing a manual process.
* To reduce the surface area for testing we (the foundation) has
adopted a strategy of targeting a limited number of components for
maintenance releases. We test those few well and then cut a release.
Hopefully we hop onto the next set of components and can cut a second
maintenance release "soon" after.
* Security issues change the schedule. They need to happen quickly
and so we may change our plans entirely based on the severity of the
security issue and how close we are to cutting a release. And when one
of these issues is involved it will now mean (usually) simultaneous
releases on the 2.5 and 2.6 series (at least).
Now we haven't been fully successful in executing this approach for a
variety of reasons. And it is only a good idea to limit the components
involved if, in fact, we can get maintenance releases out fairly
quickly I'm expecting Pete P to get things moving more smoothly. But
right now the community is placing too much reliance on the
Foundation. We simply need some help :-).
Mostly this means help testing maintenance releases. Those who want to
help test will certainly get a bigger voice in what issues/components
are addressed first. Makes sense, yes?
Another issue is the work involved in release management. It is
increasingly the case that the work of release management has been
taken up by Anthony. We could really use some help with that. I'd
love to have someone in the community start to manage the maintenance
releases--usually the .0 release is the hardest and once things are
set up, the maintenance release process is much less burdensome. Any
volunteers to manage the upcoming maintenance releases for the 2.6 or
2.5 series? Anthony will certainly guide you through the first one
and be there to assist.....
Best,
Michael
More information about the sakai-qa
mailing list