[WG: Sakai QA] [Management] Release Process Proposal

[Clay Fenlason]

The week is at an end, and I can't see any fundamental barriers to
moving ahead as outlined. [1] Where there are questions, they're in
the details we need to work out in the open as we go. Forgive the
officiousness, but here are some procedural details for how I propose
we do so. I'm rattling this off at the end of the day on a Friday, so
where they don't make sense or leave important questions unanswered,
please just speak up and I'll work to improve upon it:

1) We'll have release planning phone calls in the same slot as the
weekly QA meetings, including the product manager (myself), the
release management team, and anyone else who cares to participate,
from now until code freeze. On Monday I'll be posting what I think the
agenda for the first meeting should be, so that there is ample time to
discuss and refine it before the call.  After the call (Thursday)
there should also be a thorough report, including a proposal of
consensus points for further comment on-list, following which there
will be another proposed agenda the next Monday, and so forth. I
expect to repeat this pattern until code freeze for 2.7, which should
be at some point in September (the precise date of which will be
determined by release planning yet to come).  I expect also to keep
things organized and readable in the Management/Project Coordination
space of Confluence.

2) The latest summary documentation about development work being
reviewed for 2.7 is available in Confluence [2].  For the moment this
page includes both significant new capabilities and lightweight
refinements; 2.7 is not intended to include the former, but triage
over what counts as 'significant' is what's going to occupy the
meeting at first, and so items may shift around in discussion. If you
have items to add or comments to make about any of these, by all means
do so either on-list or by editing the Confluence page (or commenting
on it).  Understand, too, that significant new capabilities will go to
the Product Council for review, and will likely be postponed for 2.8,
which should be available in Q2 of 2010. [1]

3) The release planning meetings will not be engaged in a process of
approving or rejecting development work for 2.7.  Rather, they will be
focused on:
  (a) identifying which new capabilities are significant enough to be
routed to the product council (presumably for 2.8)
  (b) devising a workable plan to get everything else tested
adequately and released in 2.7 by the end of 2009

If the release planning meetings lead to the conclusion that this
scope for 2.7 cannot be tested adequately and released by the end of
2009, then the release team will come back to the community with a
proposal that involves extending the deadline, reducing features, or a
combination of the two - but that would be a future discussion.

4) I'll propose that there are only three ways that a set of planned
changes [2] will not be included in 2.7. Either it will be judged (by
the release team, in conversation with the product manager)
significant enough to require the Product Council's review, or its
project leads will fail to provide basic documentation by the end of
August, or it will contain 'blocker' issues that cannot be resolved in
time for release. The documentation need will be commensurate with the
kind and scope of proposed change, and the release planning meetings
will work to identify where these documentation needs are and
communicate them out early.  For certain changes (e.g. performance
improvements) there may be no documentation required beyond who is
committed to support the code, while others (e.g. UI changes) may need
user documentation as well as test cases for QA. I expect to be
working with project teams in most cases, and so between myself, the
release team and QA volunteers I don't expect this to be an issue
unless a particular project lead/team is utterly unresponsive.

5) We'll commit to a determination by the end of August (probably
sooner, and we'll start with the obvious candidates first) as to which
portions of development work are big enough to need to go to the
Product Council for 2.8.

~Clay

[1] http://confluence.sakaiproject.org/x/CoLgAw
[2] http://confluence.sakaiproject.org/x/j4LgAw

On Thu, Jul 30, 2009 at 11:49 AM, Clay
Fenlason<clay.fenlason at et.gatech.edu> wrote:
> Some comments in-line:
>
> On Thu, Jul 30, 2009 at 11:21 AM, David Haines<dlhaines at umich.edu> wrote:
>> As pointed out in the release proposal there are a couple of different kinds
>> of changes that can end up in a release.  Some are incremental changes and /
>> or fixes that aren't controversial should be in a release soon as general
>> improvements.  There are also bigger changes that require more design and
>> thought.
>>
>> It seems that the 2.7 release will have the first kind of changes.  It also
>> seems like over-planning to work at try to find out what ought be in 2.7 and
>> then plan dates from there.
>
> I don't think that's what the proposal lays out.  It's rather to:
>
> 1) Identify the bigger changes, and route them to the PC, then go back
> and survey the smaller changes
> 2) Document the smaller changes, including a rough risk assessment
> that gives QA planning some clues about how to organize its
> activities.  This will undoubtedly involve at some point some
> controversy about what really is a bigger change, and that will need
> to be dealt with in discussion - another reason this shouldn't just be
> left until a freeze date (and potential surprises during QA).
> 3) With all this information in hand, come up with a credible plan
> that includes realistic dates
>
>> It would be more realistic to establish
>> specific dates based on the practicalities of doing a release and make it
>> clear to the community when changes need to be ready to get into the release
>> cycle.
>
> Right, so I don't think we're disagreeing here. I'm just emphasizing
> the part where I don't think the "practicalities of doing a release"
> can be assessed without a close survey of the proposed set of changes.
> I'd also emphasize (I don't think you're leaving this out, I just want
> to be clear) that it's not simply a case of having code changes ready
> before code freeze, it's also making sure there is enough
> documentation and test cases prepared that QA can play its role, and
> contributing institutions can plan accordingly.
>
>> That allows for groups to plan on their own without an explicit or
>> implicit expectation that the Sakai foundation will be orchestrating the
>> work.  If a group want's to have their changes in the release they know the
>> deadlines, if one group wants to help out another with QA or such they will
>> be able to know when their efforts are likely to be needed.  This doesn't
>> mean there isn't release management, just that it is incumbent on community
>> members to take the lead on getting a specific change into a release.
>>
>> For a release with bigger plans it makes a lot of sense to make the release
>> date dependent on the necessary set of features being available.
>>
>> - Dave
>
> ~Clay
>