[sakai2-tcc] Decision point: sakai-2.8.0 community release review

[Anthony Whyte]

Responses below.

> 
> -> If a JIRA is not being actively being worked on _and_ is not new to this release, it should be downgraded to Critical
>   This might apply to: STAT-270, SAK-19686, KNL-648
> -> If a JIRA does not affect the Sakai 2 product (e.g. HYB-107) it should be downgraded to Critical


Since December 2009 the 2.8 the release management work group has been applying what I call "the Hedrick rule" to tickets flagged as blocker and critical issues during our triage sessions.  If the issue is a bug of longstanding we have downgraded it to critical if there is no prospect that it will be addressed within the 2x release cycle.  The decision usually involves a brief exchange with interested project teams and is of course always open to reversal.  So the suggestions Beth makes are already in play.

The tickets highlighted in my email represent the next group to be triaged.  SAK-19686 may be covered by the Hedrick rule but STAT-270 is new and I am planning to assign it to myself if there are no takers at the upcoming combined RM/MT call this Thursday.

As for HYB-107 I talked to Lance about it yesterday.  It's not a blocker for 2.8 but it is a blocker for the next Hybrid release.  Hybrid is something of a special case since it straddles both the CLE and OAE worlds.  For 2.8 all we care about is that Hybrid-utils contains no blockers; the full Hybrid project lies outside the scope of 2.8.  That said, I include Hybrid in the Jira 2x blocker filter in order to track any issues reported that may affect hybrid-utils.  I could remove Hybrid from my filter but I'd prefer to leave it in.

> 
> The reality of it is  the list is fluid and needs someone who can monitor it daily and interact with the development teams to ensure people are working on the critical bugs and get estimates.   I've seen this repeatedly the past few years  - Sakai 2 needs someone in a project management/coordination role.   Anthony and Alan have been sharing in this but their focuses really should be elsewhere.   I'd like to see us recommend this role for subsequent release and aid in finding someone to fulfill it.  In the short term is there anyone on the TCC that can fulfill this role?


As for Megan's suggestion, I concur.  As Release lead I've been forced out of necessity to adopt something of the trappings of a project management/coordination role following Michael Korcuska's announcement some time ago that Peter Knoop in his role as project coordinator would no longer oversee Jira as he once did while Clay Fenlason in his former role as Product Manager focused on strategic concerns and product council facilitation.  It would be a net gain if one or more non-Foundation, community volunteers were to help out in this area.

There is an ancient job description in Confluence that describes the heroic role that Peter Knoop previously endeavored to fulfill for the Sakai Community.  

https://confluence.sakaiproject.org/display/MGT/Sakai+Projects+Coordinator+-+Position+Description

I use the words "heroic" and "endeavored" to highlight the challenging role that he was assigned--it is no easy task to assume a role that involves (perceived) responsibility but little authority.  If the role were to be resurrected for 2x I would recommend that the position be defined in way that recognizes both the volunteer nature of Sakai development activities and that it depends entirely on the deft use of "soft power" to achieve success.  In other words, it is not a "Manager" role--to describe it as such would misrepresent the role's power to coerce (i.e., none) and condemn it's holder to a set of unreasonable expectations.

Frankly, given the 2x ecosystem at present, I doubt we will find a single non-Foundation person would possess the time to fill a (redefined) project coordination/facilitation role, unless an institution is willing to volunteer someone for the role (a quarter to half time position at least).  I think instead we should look at forming a Jira secretariat, a group of individuals who would oversee ticket workflows, tracking and communication.

Anth



On Feb 16, 2011, at 6:56 AM, May, Megan Marie wrote:

> Hi Beth,
>   I like those guidelines for the short term.    
> 
> At the last RM/MT meeting I believe that Anthony and Alan were sharing contacting those with bugs on the list.     Do we have any feedback on that?
> 
> The reality of it is  the list is fluid and needs someone who can monitor it daily and interact with the development teams to ensure people are working on the critical bugs and get estimates.   I've seen this repeatedly the past few years  - Sakai 2 needs someone in a project management/coordination role.   Anthony and Alan have been sharing in this but their focuses really should be elsewhere.   I'd like to see us recommend this role for subsequent release and aid in finding someone to fulfill it.  In the short term is there anyone on the TCC that can fulfill this role?
> 
> As  for sliding the release date  . . . .I've said this a number of times. We need to have a concrete plan with a target.  I think the recommendations coupled with having someone in the afore mentioned role nudging the community along would go a long way.
> 
> Megan
> 
> 
> -----Original Message-----
> From: sakai2-tcc-bounces at collab.sakaiproject.org [mailto:sakai2-tcc-bounces at collab.sakaiproject.org] On Behalf Of Beth Kirschner
> Sent: Tuesday, February 15, 2011 9:41 PM
> To: Anthony Whyte
> Cc: sakai2-tcc at collab.sakaiproject.org Committee
> Subject: Re: [sakai2-tcc] Decision point: sakai-2.8.0 community release review
> 
> I would agree that a March 1st release date is unrealistic at this point. I also think we need a more objective set of guidelines for determining what is a blocker. I don't think we can put those guidelines together in the next couple of weeks, but would like to propose a couple suggestions for the near-term set of blockers:
> 
> -> If a JIRA is not being actively being worked on _and_ is not new to this release, it should be downgraded to Critical
>   This might apply to: STAT-270, SAK-19686, KNL-648
> -> If a JIRA does not affect the Sakai 2 product (e.g. HYB-107) it should be downgraded to Critical
> 
> We should contact each individual assigned to each remaining Blocker and request an ETA. If the ETA is not within the next couple of weeks, we should request help from others. 
> 
> In the meantime, it might be best if we hold off announcing any new dates until the remaining blockers are indeed fixed. Thoughts?
> 
> - Beth
> 
> On Feb 15, 2011, at 1:10 PM, Anthony Whyte wrote:
> 
>> Today marks the "release readiness decision point" in the sakai-2.8.0 schedule and the TCC is tasked with reviewing and determining the readiness of the 2.8.x branch for release tagging.  
>> 
>> On 24 January I provided a brief summary on the state of the upcoming 2.8.0 release, noting the divergence between the published schedule and the beta character of generated QA tags to date.  The schedule called for the tagging of sakai-2.8.0-rc01 on 11 January 2011.  The existence of unresolved blockers as well as concerns over a lack of progress with respect to functionality testing necessitated the issuance of additional beta tags on 13 Jan, 26 Jan and 9 Feb respectively. 
>> 
>> As of today a number of blockers remain unresolved (see below) and functionality testing coverage appears attenuated to me (see below), even when acknowledging that not all functionality may require comprehensive testing.  The next tag is scheduled for 22/23 Feb and without a push to eliminate the remaining blockers we could well see another beta tag rather than a release candidate.  
>> 
>> In my view releasing sakai-2.8.0 on 1 March 2011 is no longer a realistic proposition.  Starting tomorrow there remain only 9 business days before the scheduled release date, a period that in my view leaves us too little time to kick the tires of a release candidate before stamping it as worthy of public consumption.
>> 
>> Earlier, I recommended that we revise the schedule and target the end of March 2011 for a release.  I use the word "target" deliberately as none of us are in a position to compel anyone other than ourselves to meet particular deadlines.  That said, I believe that the blockers listed below can be addressed (or if necessary downgraded) within the next couple of weeks if the requisite energy is applied.  A late March release is possible but it only allows for three QA tags. 
>> 
>> 22/23 Feb QA tag
>> 8/9 March QA tag
>> 22/23 March QA tag
>> 24-30 March prep for release 
>> 31 March release ?
>> 
>> My recommendations:
>> 
>> 1.  Blocker handling: continue to triage the blocker list.  If a ticket is not truly a blocker, downgrade it.  Communicate with blocker assignees regarding progress on tickets (Alan and I do this regularly).  Again, I suggest that TCC members review the existing set of blockers and comment on whether or not they are worthy of holding up the release.
>> 
>> 2.  Schedule: cut QA tags every two weeks until blockers are eliminated.  If an opportunity exists to cut a QA tag early, do it.  When no blockers exist cut an rc tag and prep for release.  Target release for end of March 2011 but recognize that timelines are dependent on community action not words or desires.  
>> 
>> 3.  Functionality testing: encourage functionality testing but do not allow shortfalls or delays in testing to hold up the release.  QA under-staffing is an issue of longstanding that will not be solved in this phase of the release cycle and as such should be treated in the same manner we treat long-standing bugs, i.e., it's not a blocker.
>> 
>> Cheers,
>> 
>> Anthony
>> 
>> 
>> Schedule
>> https://confluence.sakaiproject.org/display/REL/Sakai+2.8+Release+Schedule
>> 
>> 
>> 2.8 Test Fest results
>> 
>> https://confluence.sakaiproject.org/display/QA/2.8+Test+Fest
>> 
>> 
>> 2.x Blockers
>> 
>> https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&requestId=12708
>> 
>> 1.  Announcements
>> https://jira.sakaiproject.org/browse/SAK-20020
>> 
>> 2. Site Info
>> https://jira.sakaiproject.org/browse/SAK-19686
>> 	relates to https://jira.sakaiproject.org/browse/SAK-18616
>> 
>> 3. DDL-related
>> https://jira.sakaiproject.org/browse/STAT-270
>> https://jira.sakaiproject.org/browse/SAK-20057
>> https://jira.sakaiproject.org/browse/SAK-19965 (should be fixed today)
>> https://jira.sakaiproject.org/browse/SAK-19945 
>> https://jira.sakaiproject.org/browse/SAK-19088 (parent)
>> 
>> 4.  WYSIWIG editor
>> https://jira.sakaiproject.org/browse/SAK-20025
>> 
>> 5. Kernel 
>> https://jira.sakaiproject.org/browse/KNL-648  (I do not regard this ticket as a blocker)
>> 
>> 6. i18n (sections)
>> 
>> A set of commits that overlay a regression introduced by r80384 that is further complicated by unacceptable code changes that I declined to commit to 2.8.x and asked to be reverted in trunk.  The /config changes were reverted on 9 Feb (r88247) after 2.8.0-b04 was tagged; hopefully a final review of the commits associated with these three tickets will prove positive and I'll merge the changes to 2.8.x.
>> 
>> https://jira.sakaiproject.org/browse/SAK-19910 
>> 	related to 
>> 		https://jira.sakaiproject.org/browse/SAK-18850
>> 		https://jira.sakaiproject.org/browse/SAK-18859
>> 
>> 
>> 4. Ignore
>> 
>> https://jira.sakaiproject.org/browse/HYB-107 (blocker for next Hybrid release only)
>> 
>> 
>> 2x critical
>> https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&requestId=12709
>> 
>> 
>> 
>> 
>> Begin forwarded message:
>> 
>>> From: Anthony Whyte <arwhyte at umich.edu>
>>> Date: January 24, 2011 2:52:03 PM EST
>>> To: "sakai2-tcc at collab.sakaiproject.org Committee" <sakai2-tcc at collab.sakaiproject.org>
>>> Subject: Update: Sakai CLE 2.8.0 Community Release
>>> 
>>> 15 February 2011 is the date when the TCC is scheduled to review the readiness of the 2.8.x branch for release as sakai-2.8.0.  I want to provide you with a quick briefing of where we are at today in order to better inform our decision regarding readiness to release.
>>> 
>>> 
>>> RC WHEN? (https://confluence.sakaiproject.org/display/REL/Sakai+2.8+Release+Schedule)
>>> 
>>> The first release candidate was scheduled for tagging on 11 Jan 2011.  Instead, I issued sakai-2.8.0-b03 (beta) instead of an RC.  I did so for the following reasons:
>>> 
>>> 1) unresolved blockers (including conversion script testing)
>>> 2) large numbers of fixes flagged for merging (suggesting code instability)
>>> 3) a lack of functionality testing compounded by an unclear understanding of what has/has not been tested
>>> 
>>> The next QA tag is schedule for 25 Jan 2011.  The conditions above have hardly changed although I should note that the first in a series of promising Functionality Test Fest events took place last week (see https://confluence.sakaiproject.org/display/QA/2.8+Test+Fest).  Nevertheless, blockers still exist, merge requests have yet to stabilize, conversion scripts remain untested, while functionality testing is only just commencing.
>>> 
>>> Given the above, I plan to issue another beta tag rather than a release candidate on Tue/Wed this week (sakai-2.8.0-b04).  As a result we will be off-schedule by some four weeks when I issue the next tag on 8 February 2011..
>>> 
>>> 
>>> BLOCKERS (https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&requestId=12708)
>>> 
>>> Given the tag slippage described above the release candidate phase of the schedule will need to be compressed in order to produce a Community 2.8.0 release by 1 March 2011.  This raises questions regarding our tolerance for risk; however, I recommend that we focus on the existing set of blockers and decide which ones, if any, either introduce embarrassing regressions or imperil the release from a performance or security point of view. 
>>> 
>>> I do not accept the status of Jira tickets uncritically.  One person's blocker is another person's critical ticket (the converse is also true).  Last week during the RM call Beth Kirschner, Megan May, Alan Berg, Jean-Francois Leveque and myself triaged the blocker list and downgraded the priority of a half dozen tickets.  I followed up this work with individuals emails to blocker assignees (not all have replied).  There is nothing new in this approach--we've been triaging tickets since 2.5 if not earlier.
>>> 
>>> Not all blockers currently listed need to hold up the 2.8.0 release in my opinion.  I will not go into further detail on this list regarding individual tickets but I encourage you to review the current blocker list and send me your opinions.
>>> 
>>> If the current set of blockers are not true blockers, we should downgrade their status and prep for release; otherwise, we should continue to issue QA tags every other week until all blockers are eliminated. 
>>> 
>>> 
>>> RUTGERS TIMELINE (OUR TIMELINE?)
>>> 
>>> Rutgers intends to deploy 2.8.0 in production in May 2011 with local testing planned for February and March.  I know of no other organizations planning a rollout of 2.8.0 during Q1/Q2 2011 but ignorance such as this is due to the fact that no one (including myself) has polled the Community on the question.   
>>> 
>>> We probably have two choices here: 
>>> 
>>> 1.  we adhere to the existing schedule, eliminating all true blockers in the next two weeks and then issuing sakai-2.8.0-rc01 for Rutger's use on 8 Feb 2011.  As with previous releases, sakai-2.8.0-rc01 will be tagged from a release branch in order to allow non-blocker fixes to flow to 2.8.x.  We then prep for release during  21-25 February and release sakai-2.8.0 on 1 March 2011.  However, this leaves us little or no room to respond to any blocker/critical issues that Rutgers (or the Marist/rSmart Test Fest) may discover during the month of February.  Such an approach could well doom us to an early 2.8.1 maintenance release--involving additional work not merely for the release team but also deployers.
>>> 
>>> 2.  we modify the current schedule to better incorporate Rutgers and Test Fest testing.  Again, we endeavor to eliminate all true blockers in the next two weeks, issuing sakai-2.8.0-rc01 on 8 Feb 2011.  Two weeks later on 22 February we issue sakai-2.8.0-rc02; if necessary we issue another RC tag on 8 March 2011.  We prep for release on 21-25 March 2011 and release the following week by 31 March 2011.  This latter approach provides us with an opportunity to produce a couple of RC tags that includes fixes produced in response to Rutgers and Test Fest testing.
>>> 
>>> I prefer option 2 as I believe it is better aligned to current realities including better hooks into Rutgers and Test Fest QA activities than option 1.
>>> 
>>> What say you?
>>> 
>>> Cheers,
>>> 
>>> Anthony
>> 
>> _______________________________________________
>> sakai2-tcc mailing list
>> sakai2-tcc at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
> 
> _______________________________________________
> sakai2-tcc mailing list
> sakai2-tcc at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3829 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20110216/6bc10b1e/attachment-0001.bin