[sakai2-tcc] Decision point: sakai-2.8.0 community release review

Tue Feb 15 10:10:21 PST 2011

Today marks the "release readiness decision point" in the sakai-2.8.0 schedule and the TCC is tasked with reviewing and determining the readiness of the 2.8.x branch for release tagging.  

On 24 January I provided a brief summary on the state of the upcoming 2.8.0 release, noting the divergence between the published schedule and the beta character of generated QA tags to date.  The schedule called for the tagging of sakai-2.8.0-rc01 on 11 January 2011.  The existence of unresolved blockers as well as concerns over a lack of progress with respect to functionality testing necessitated the issuance of additional beta tags on 13 Jan, 26 Jan and 9 Feb respectively. 

As of today a number of blockers remain unresolved (see below) and functionality testing coverage appears attenuated to me (see below), even when acknowledging that not all functionality may require comprehensive testing.  The next tag is scheduled for 22/23 Feb and without a push to eliminate the remaining blockers we could well see another beta tag rather than a release candidate.  

In my view releasing sakai-2.8.0 on 1 March 2011 is no longer a realistic proposition.  Starting tomorrow there remain only 9 business days before the scheduled release date, a period that in my view leaves us too little time to kick the tires of a release candidate before stamping it as worthy of public consumption.

Earlier, I recommended that we revise the schedule and target the end of March 2011 for a release.  I use the word "target" deliberately as none of us are in a position to compel anyone other than ourselves to meet particular deadlines.  That said, I believe that the blockers listed below can be addressed (or if necessary downgraded) within the next couple of weeks if the requisite energy is applied.  A late March release is possible but it only allows for three QA tags. 

22/23 Feb QA tag
8/9 March QA tag
22/23 March QA tag
24-30 March prep for release 
31 March release ?

My recommendations:

1.  Blocker handling: continue to triage the blocker list.  If a ticket is not truly a blocker, downgrade it.  Communicate with blocker assignees regarding progress on tickets (Alan and I do this regularly).  Again, I suggest that TCC members review the existing set of blockers and comment on whether or not they are worthy of holding up the release.

2.  Schedule: cut QA tags every two weeks until blockers are eliminated.  If an opportunity exists to cut a QA tag early, do it.  When no blockers exist cut an rc tag and prep for release.  Target release for end of March 2011 but recognize that timelines are dependent on community action not words or desires.  

3.  Functionality testing: encourage functionality testing but do not allow shortfalls or delays in testing to hold up the release.  QA under-staffing is an issue of longstanding that will not be solved in this phase of the release cycle and as such should be treated in the same manner we treat long-standing bugs, i.e., it's not a blocker.

Cheers,

Anthony

Schedule
https://confluence.sakaiproject.org/display/REL/Sakai+2.8+Release+Schedule

2.8 Test Fest results

https://confluence.sakaiproject.org/display/QA/2.8+Test+Fest

2.x Blockers

https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&requestId=12708

1.  Announcements
https://jira.sakaiproject.org/browse/SAK-20020

2. Site Info
https://jira.sakaiproject.org/browse/SAK-19686
	relates to https://jira.sakaiproject.org/browse/SAK-18616

3. DDL-related
https://jira.sakaiproject.org/browse/STAT-270
https://jira.sakaiproject.org/browse/SAK-20057
https://jira.sakaiproject.org/browse/SAK-19965 (should be fixed today)
https://jira.sakaiproject.org/browse/SAK-19945 
https://jira.sakaiproject.org/browse/SAK-19088 (parent)

4.  WYSIWIG editor
https://jira.sakaiproject.org/browse/SAK-20025

5. Kernel 
https://jira.sakaiproject.org/browse/KNL-648  (I do not regard this ticket as a blocker)

6. i18n (sections)

A set of commits that overlay a regression introduced by r80384 that is further complicated by unacceptable code changes that I declined to commit to 2.8.x and asked to be reverted in trunk.  The /config changes were reverted on 9 Feb (r88247) after 2.8.0-b04 was tagged; hopefully a final review of the commits associated with these three tickets will prove positive and I'll merge the changes to 2.8.x.

https://jira.sakaiproject.org/browse/SAK-19910 
	related to 
		https://jira.sakaiproject.org/browse/SAK-18850
		https://jira.sakaiproject.org/browse/SAK-18859

4. Ignore

https://jira.sakaiproject.org/browse/HYB-107 (blocker for next Hybrid release only)

2x critical
https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&requestId=12709

Begin forwarded message:

> From: Anthony Whyte <arwhyte at umich.edu>
> Date: January 24, 2011 2:52:03 PM EST
> To: "sakai2-tcc at collab.sakaiproject.org Committee" <sakai2-tcc at collab.sakaiproject.org>
> Subject: Update: Sakai CLE 2.8.0 Community Release
> 
> 15 February 2011 is the date when the TCC is scheduled to review the readiness of the 2.8.x branch for release as sakai-2.8.0.  I want to provide you with a quick briefing of where we are at today in order to better inform our decision regarding readiness to release.
> 
> 
> RC WHEN? (https://confluence.sakaiproject.org/display/REL/Sakai+2.8+Release+Schedule)
> 
> The first release candidate was scheduled for tagging on 11 Jan 2011.  Instead, I issued sakai-2.8.0-b03 (beta) instead of an RC.  I did so for the following reasons:
> 
> 1) unresolved blockers (including conversion script testing)
> 2) large numbers of fixes flagged for merging (suggesting code instability)
> 3) a lack of functionality testing compounded by an unclear understanding of what has/has not been tested
> 
> The next QA tag is schedule for 25 Jan 2011.  The conditions above have hardly changed although I should note that the first in a series of promising Functionality Test Fest events took place last week (see https://confluence.sakaiproject.org/display/QA/2.8+Test+Fest).  Nevertheless, blockers still exist, merge requests have yet to stabilize, conversion scripts remain untested, while functionality testing is only just commencing.
> 
> Given the above, I plan to issue another beta tag rather than a release candidate on Tue/Wed this week (sakai-2.8.0-b04).  As a result we will be off-schedule by some four weeks when I issue the next tag on 8 February 2011.
> 
> 
> BLOCKERS (https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&requestId=12708)
> 
> Given the tag slippage described above the release candidate phase of the schedule will need to be compressed in order to produce a Community 2.8.0 release by 1 March 2011.  This raises questions regarding our tolerance for risk; however, I recommend that we focus on the existing set of blockers and decide which ones, if any, either introduce embarrassing regressions or imperil the release from a performance or security point of view. 
> 
> I do not accept the status of Jira tickets uncritically.  One person's blocker is another person's critical ticket (the converse is also true).  Last week during the RM call Beth Kirschner, Megan May, Alan Berg, Jean-Francois Leveque and myself triaged the blocker list and downgraded the priority of a half dozen tickets.  I followed up this work with individuals emails to blocker assignees (not all have replied).  There is nothing new in this approach--we've been triaging tickets since 2.5 if not earlier.
> 
> Not all blockers currently listed need to hold up the 2.8.0 release in my opinion.  I will not go into further detail on this list regarding individual tickets but I encourage you to review the current blocker list and send me your opinions.
> 
> If the current set of blockers are not true blockers, we should downgrade their status and prep for release; otherwise, we should continue to issue QA tags every other week until all blockers are eliminated. 
> 
> 
> RUTGERS TIMELINE (OUR TIMELINE?)
> 
> Rutgers intends to deploy 2.8.0 in production in May 2011 with local testing planned for February and March.  I know of no other organizations planning a rollout of 2.8.0 during Q1/Q2 2011 but ignorance such as this is due to the fact that no one (including myself) has polled the Community on the question.   
> 
> We probably have two choices here: 
> 
> 1.  we adhere to the existing schedule, eliminating all true blockers in the next two weeks and then issuing sakai-2.8.0-rc01 for Rutger's use on 8 Feb 2011.  As with previous releases, sakai-2.8.0-rc01 will be tagged from a release branch in order to allow non-blocker fixes to flow to 2.8.x.  We then prep for release during  21-25 February and release sakai-2.8.0 on 1 March 2011.  However, this leaves us little or no room to respond to any blocker/critical issues that Rutgers (or the Marist/rSmart Test Fest) may discover during the month of February.  Such an approach could well doom us to an early 2.8.1 maintenance release--involving additional work not merely for the release team but also deployers.
> 
> 2.  we modify the current schedule to better incorporate Rutgers and Test Fest testing.  Again, we endeavor to eliminate all true blockers in the next two weeks, issuing sakai-2.8.0-rc01 on 8 Feb 2011.  Two weeks later on 22 February we issue sakai-2.8.0-rc02; if necessary we issue another RC tag on 8 March 2011.  We prep for release on 21-25 March 2011 and release the following week by 31 March 2011.  This latter approach provides us with an opportunity to produce a couple of RC tags that includes fixes produced in response to Rutgers and Test Fest testing.
> 
> I prefer option 2 as I believe it is better aligned to current realities including better hooks into Rutgers and Test Fest QA activities than option 1.
> 
> What say you?
> 
> Cheers,
> 
> Anthony

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20110215/de524fa8/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3829 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20110215/de524fa8/attachment-0001.bin 