[WG: Sakai QA] Release planning update

[Clay Fenlason]

It's time again to try to summarize a range of discussion and
activity. I'm sure to fail to be comprehensive, and I'm not quite
comfortable with some areas that don't have clear resolution, yet it
feels more important to talk about things as we go than to tie up all
the loose ends before speaking out and inviting comment.  There is
much to say. What follows is my understanding of the current state of
affairs for Sakai releases in the coming year.

2.6 MAINTENANCE RELEASES

The first point is that the 2.6 branch of code still needs attention,
including maintenance releases, and that needs to be a priority.  The
current plan is to have two such releases before the end of the year,
the first of which should be ready by the end of September.  2.6.x QA
servers with known revisions are already starting to go up.

2.7 FEATURE RELEASE(S)

The initial proposal to have two full Sakai feature releases in the
coming year [1] has been abandoned, for reasons more or less laid out
in an earlier thread [2]. We are now moving ahead with a rough plan to
have a 2.7 by the end of April which will include some independent
tool releases, very likely including entirely new tools or sets of
capabilities if they can work their way through the new product
development process in time.

What "in time" means is less clear than it has been in the past, and
that's a problem, but I think that, on balance, this reflects an
improvement in our state of affairs. We can begin testing with tools
and components that are ready to go now, and provide a measured way
(i.e. the independent release process) for testing to incorporate them
later, one that takes into account the scope of change and risk that
these capabilities represent, along with the amount of QA time and
resource that should go toward them.

In more specific terms, I think we can begin very soon (before the
next week is out) to test the set of changes in view for Forums (or,
more precisely, msgcntr) against a branch of trunk, then once that's
finished bring in a release of Tests&Quizzes, do more regression
testing, and so forth.  The details are under the purview of Pete
Peterson's coordination of QA resources, along with the participation
of particular development teams, and much of that still needs to be
squared away in conversation on the details, which has so far been
happening behind the scenes. Part of the delay is the convergence of
vacations and start-of-term emergencies for key players, something
that always happen this time of year, but there are ways we can make a
productive start without knowing exactly how we're going to finish. My
specific hope, again, is to see a clearer plan developed by the end of
next week, and to jumpstart testing for an independent Messages+Forums
release at about the same time.

Anthony Whyte has analyzed comparisons of 2.6.x against 2.6.0 as well
as trunk against 2.6.x.  He will offer details once his laptop dries
out (!), but the upshot appears to be that we should have a
maintenance release based on a known revision of 2.6.x before the
month is out, and also that we can and should base any independent
tool releases against a branch of trunk (i.e. rather than against
2.6.x).

MAINTENANCE TEAM

Anthony Whyte is taking the lead on recruiting dedicated participation
for a maintenance team, which would take on challenges of bugfixing in
areas of the code where it's most needed. I mention it here because
this team will doubtless be a significant factor in issue resolution
during the coming year, it should accelerate our ability to produce
quality releases as a result, and it will have some voice in the
question of release-readiness of a body of code.  More details should
be coming out about this in the near future.

2.7 INCLUSION CRITERIA

We have also developed a set of expectations for the evolutionary
changes of existing code [3]. The criteria are largely matters of
self-documentation which seem less than controversial, apart from some
ambiguity about what good coding practices are: that needs to get
sorted out. Nevertheless, the basic expectation is that a failure to
have this documentation in hand will oblige the release team to either
not include a change/patch, or roll back the code/release to a prior,
known stable point.  In most cases this would likely mean reverting to
the 2.6.x version of the code.

At the time of this writing, a breakdown of open questions about
planned changes is available in a Google Spreadsheet. [4] Its contents
are limited for the moment to Forums, Tests & Quizzes, and Site Info,
the three core tools with the biggest set of planned changes.  Where
you see a question mark is a point where a change does not yet have
sufficient documentation for inclusion. This document of course also
needs some work, and needs to be further informed by Anthony's branch
analysis, but it offers a fair indication of what's needed at this
stage.

RELEASE TEAM GAPS

We do not yet have a branch manager for 2.7. This is concerning, it
impedes our ability to move ahead with testing and release management
for the cycle, and it may even raise the question of whether the
community is sufficiently motivated to have a 2.7 at all. It's also
been suggested that we need more than one branch manager, though
presumably some of the pressure will be relieved if a few tools are
spun off into their own release management. Other efforts to recruit
volunteers will continue.

Apologies for the length, and where I've made omissions or expressed
things hastily, but I trust to questions and comments to tease those
out and correct me.

~Clay

[1] http://confluence.sakaiproject.org/x/CoLgAw
[2] http://www.nabble.com/-Building-Sakai--Release-proposal-revisited-to25046188.html
[3] http://confluence.sakaiproject.org/x/6ATtAw
[4] https://spreadsheets.google.com/ccc?key=0AlfbHxo2qpHEdHZYeVNjekZOTFN3bVdmWnk0ZmwxUGc&hl=en