[WG: Sakai QA] "Layers" of QA

[Seth Theriault]

Hello,

Here at the conference, there has been renewed discussion about
simplifying and strengthening the QA process. Some of the
specifics include cutting maintenance releases directly from the
maintenance branch and trying to better harness the power of
local QA efforts that are often cyclical. Here's one proposal
that I briefly fleshed in a conversation.

When Sakai 2.6.0 is released, all existing QA servers should move
to running a known revision of the 2.6.x maintenance branch.
These servers would be refreshed on a regular (weekly or
bi-weekly basis) with another known revision (determined by the
QA director) of the same branch.

When 2.7 code freeze happens and a 2.7.x branch becomes
available, a majority of these QA servers gradually migrate to
using this new branch. Eventually, when the alphas/betas/RCs
become available, most of the QA servers will be running these
tags in anticipation of a release.

At least two QA servers would continue to run the 2.6 maintenance
branch during the "official" support period after the 2.7 release.

Lather, rinse, repeat.

I think this approach provides some advantages for the community:

1) Stability and predictability for the QA process. People no
longer have to "ramp up" for a specific release. Time can be
better spent on testing as issues are fixed in the maintenance
branch. Server admins have a predictable schedule for new
deployments. Managers may have more flexibility to allow people
to work on QA that offers both local and community benefit.

2) Perimeter defense. With this approach, there are multiple
layers of QA. When an fix or feature is first committed to a
branch, it is picked up within 24 hours on Nightly2. Then with a
week or so, ti would appear on the QA servers running the
maintenance branch. Finally, it would appear in the various
alpha/beta/RC tags later in the cycle.

3) Support. Since QA servers will be running a known revision of
the current maintenance branch, important bug and security fixes
can be targeted, tested, verified, and communicated more easily.
The QA server that are "left behind" assure the necessary testing
environment for these fixes against known revisions (we need some
more QA servers or existing ones with more capacity to make this
part work).

Seth