[WG: Sakai QA] Release proposal revisited

Clay Fenlason clay.fenlason at et.gatech.edu
Wed Aug 19 08:17:59 PDT 2009


The last couple weeks of conversation around the release proposal [1]
circulated earlier have shifted things in a different direction, and I
want to offer an update on the latest thinking in the hope of
garnering further comment.

The original goals are still in force - we want to ramp up the new
product development process without being disruptive, and we want to
have a clear plan of execution - but there is more than one way to
achieve them.  There were also a couple pragmatic reasons for wanting
a 2.7 by the end of the year:

1) a solid release in time for institutions in the southern hemisphere
that would be looking to deploy in the December-January timeframe. Not
least because these schools (UCT in particular) have been carrying our
water for the last couple years by being first movers. We owe them
some peace of mind.
2) a hedge against the risk of significant new functionality going
into 2.8.  If we ran into another situation, say, where the 2.8
release would be delayed, there would still be a good step forward for
schools or organizations needing to move in April or May.

The first of those motivations has begun to unravel.  CSU is not
planning to upgrade, while UCT suggests that they'd be more interested
in a new Forums tool release than the spread of new features planned
for 2.7.  If there is a strong interest in a 2.7 by the end of the
year I've not yet encountered it, while the second motivation above
can be satisfied in another way.

Upon reviewing the planned changes for 2.7 in greater detail, it also
became clear that the more significant sets of changes were rather
localized in particular tools.  Forums in particular, followed by
Tests & Quizzes. It started to look to me that a reasonable QA plan
might devote the bulk of its energy to a focus on two or three tools.

What's more, in the release planning meetings we've also been
reviewing lessons learned, and the point was made that a comprehensive
code freeze has been too blunt an instrument: development teams on
some projects have rushed to get things in before they were ready,
while on the other hand some projects have been ready to go well ahead
of time and have not been taken advantage of.  A less monolithic
release management might find a more productive way to harness this
high degree of variability in resource and timeframe, presenting more
options to deployers and a better distribution of community activity.
Not to mention that many of the significant new capabilities primed
for consideration by the Product Council could start early to iterate
on their development in a measured way that followed good release
management practice, such that even if they were not included in the
next release they might still offer good options for deployers
prepared to assume some extra risk.  This is not a new idea, but the
practical features of our situation are I think combining to give it
new force.

So a new release proposal is emerging in conversation, one which uses
(some) independent tool releases as its differentiating mechanism
rather than a lightweight 2.7 by the end of the year.  We could begin
with a few tools prepared to work in this way, and not try to force it
rigorously across all modules.  The upshot for QA is that there would
be two phases of testing: the first focused on the individual
tool/capability and any internal regressions, and the second looking
for regressions where all the new independent releases are bundled
together with everything else.  There will still be a comprehensive
testing effort on a full Sakai release bundle, as there will be
objective criteria for included project releases, but the foundation
of general testing will be the latest stable iterations of its
components (and where a component proves to be unstable, it may be
rolled back).

There's far more to say about the details: Anthony and Pete have been
working hard to think through them and it will be a main topic of
conversation for this week's release planning call.  I also haven't
begun to talk about the benefits for the product development process
or concerns about whether this is a practice that may carry over into
the world of Sakai 3. Nevertheless, let me stop there for comment,
since I've gone on long enough already.

~Clay

[1] http://confluence.sakaiproject.org/x/CoLgAw


More information about the sakai-qa mailing list