[sakai2-tcc] Thoughts on 2.9.1 schedule

[Berg, Alan]

I have an online meeting with Anthony next week. I will see if I can help improve the deterministicness of the release automation. Anyone else is welcome.

Sounds like the use of the word padding was unfortunate. Hidden bottlenecks and all. Keep up up the good/hard work.


Regards Alan,

Sent from somewhere interesting via a Mobile device.



-------- Original message --------
Subject: Re: [sakai2-tcc] Thoughts on 2.9.1 schedule
From: Matthew Jones <matthew at longsight.com>
To: Neal Caidin <nealcaidin at sakaifoundation.org>
CC: "sakai2-tcc at collab.sakaiproject.org 2" <sakai2-tcc at collab.sakaiproject.org>


TL;DR - Release process is too complex, local issues are part of the delay, perhaps we should just skip release candidates for minor releases like we did in the past and put out 2.9.1.

I haven't caught up in depth on this is entire thread as this is my first day back from Phoenix and still jet lagged. However nobody who commented besides Neal regularly puts any predictable effort into the actual release process. And as far as the process of actually cutting the release and merging the tickets, myself and Sam currently do the vast majority of that work. So the schedule is very dependent on our local workload. This week I was out almost all week catching up doay, and Sam will be out most of next week, putting some extra local pressure on me. Some of these delays are built into the overly padded schedule that we've kept Neal updated on.

However some of the padding is just because of the actual known difficulty and anticipated delays from the past 3 months in the time required to perform the actual release and getting people on-board with the QA effort when needed so we're ready for the next cycle. We could smoke test on nightly and just skip rc01 going straight to 2.9.1 and for sure save a lot of time cutting the schedule in half. This was what we did in the past for previous minor releases (2.7.1, 2.8.1, 2.8.2), just verifying if the software actually starts up.

Some known delays and thoughts.

- CLE Release Team currently all meet once every Thursday morning, which generally puts at *least* a week between making any actual decisions. We could make decisions via email asynchronously with time limits, but this doesn't necessarily mean things will happen faster. We could potentially meet more often when the release is about to happen, but it's difficult to time times where all interested parties can meet.

- Currently, as mentioned, Sam and I are important to the process of both branching and releasing. We're happy to distributed this more but the only person who's done any major branching lately is Greg at IU, though not consistently, and he's seems busier now. 2.9 has 63 issues waiting to merge, 2.8 has 65, most of these verified. Most of these will be pushed to 2.9.2 or 2.9.3 even though they're probably good fixes.  Steve Swinsburg does help on the releases (especially 2.8), but he hasn't done a full release of 2.9.

- The release process for 2.9 in general is overly complex. When Anthony was releasing we had half of the tools set up as indies as we do now and everything was done on command line. Now it's all done through Jenkins. The idea with this is it would make it easier for other people to do off cycle releases (Like Bryan does occasionally since I've showed him how to do it last year at the conference, and Steve does occasionally) but really just introduces random complexity because the Jenkins M2 Release Plugin has a lot of bugs, and Maven still has a lot of missing capability as well.
  There's going to need to be a separate email addressing this, but I'm almost reluctant to do *any* releases both because of the actual time it takes (2-3 hours one day, waiting ~8 hours for Jenkins, then 2-3 hours the next day) as well as the confusion it ends up causing those who use msub to customize builds and anyone who uses the 2.9.x branch. When versions get bumped, the SNAPSHOT version also gets dumped which REQUIRES that change be pushed through the entire local code-base. When Jenkins updates it often does something new and unpredictable each release cycle that I get to experience and attempt to figure out! Great fun! :)

Much of this is stuff we'd like to have Neal do, and it's part of his 3+ month on-boarding, but it really requires him to go some where and get training from someone that knows it (or someone coming to him). Doing this over Skype isn't very convenient.  I have plans for this to happen (If I'm going to be training again) by at least April. Though I'm not sure with the way the release process works and the randomness of Jenkins that it would be possible for someone who doesn't have a lot of experience troubleshooting maven errors (Something I've wrangling with years) would be able to solve easily.

On Fri, Jan 18, 2013 at 8:27 AM, Neal Caidin <nealcaidin at sakaifoundation.org<mailto:nealcaidin at sakaifoundation.org>> wrote:
Yes, no new blockers regressions, and have a fairly high bar for a show stopper. I haven't checked this morning to see if they have been fixed/merged, but we do have a few security issues that possibly won't get done in time.

I'm at EuroSakai the week of Jan 28, that might make the coordination tricky. Unlike Educause where I was barely a blip, I expect I will be getting a lot of attention in Paris and will be hard pressed to find time for QA coordination/check-in. Would be nice to have a backup QA lead, but I don't.

So maybe the schedule range is more of the first or second week in February. But if we need an RC02, then that schedule fails, correct? And I need to have a QA lead/check in backup plan. If I don't have that, it could delay things a week and we are back to a mid-February.

- Neal



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai2-tcc/attachments/20130119/b63afc23/attachment-0003.html