[WG: Sakai QA] [cle-release-team] Proposal: towards a releasable trunk for Sakai 10 (and beyond)

Thu Nov 14 10:20:47 PST 2013

Thanks for this proposal Anthony. It brings up some important points in the evolution of managing Sakai development.

I am hearing a lot of concerns about how quality assurance (QA) will be managed with this type of proposal. I would like to share what I am thinking on how QA testing could fit into this plan.

Before the branching work at the end of January, we could start testing. Already we have had some QA testing of Jira issues that are already merged to trunk, to verify them (special shout out to Derek Ramsey of Longsight for his work in this area! Derek’s been doing most of this testing to-date.)  As mentioned in the proposal, we would generate a QA tag directly from trunk and deploy to QA servers for testing. I would think this could start in another week or so (+/-, as American Thanksgiving is coming up fast, which could push this out a little). At that time we could do some additional Jira testing on already merged issues plus start Regression testing. Regression testing would start by testing tools not likely to have significant additional changes made to them (as best as we can identify this).  As we get closer to what I am calling a Beta branch cut (late January), we can focus more on tools and areas of Sakai which have had significant work done during this QA phase (making up another name, for now).  

I would expect more intensive testing during the Beta phase starting at the end of January/ beginning of February, but the more testing we can get done during QA phase (what I’m calling the phase before the Beta branch), the better quality release we will have for the Beta and the more likely we will meet our schedule goals.

We should be looking at significant QA testing commitment starting shortly from our community institutions and commercial affiliates. Consider this a heads up ! :-) 

The folks who participate on the QA team calls (which is very small and would appreciate more participation) is thinking we should run some synchronous testing sessions so that we can test together. True “Test Fests”. I’m willing to coordinate groups across various time zones if there is interest. I’ve also had some positive, and no negative, response to the idea of an Adopt-a-Tool approach going into the Beta and Release candidate phase. I plan on publicizing this idea more as well and seeing if it might catch on.

Ideas for QA are welcome, commitments to participate in testing are even more welcome!

Thanks,
Neal

Neal Caidin
Sakai Community Coordinator
neal.caidin at apereo.org
Skype: nealkdin
Twitter: ncaidin

On Nov 14, 2013, at 10:00 AM, Anthony Whyte <arwhyte at umich.edu> wrote:

> Proposal
> As a community we shift to a "releaseable trunk" mindset.   For the Sakai 10 release, we commence this mind shift by branching trunk to 10.x at the end of January 2014 rather than now, targeting ApereoCamp as the moment we freeze development and branch with a trailing string freeze date set accordingly.  Between now and then we focus development and quality assurance efforts on trunk, generating QA tags at regular intervals directly from trunk for deployment on community-provided/funded QA servers.  Contributors should focus on 10 activities; post-10 development, if any, should be held back in a working branch during this cycle.  Going forward we continue to refine our practices in order to ensure a high quality trunk capable of being released quickly and efficiently [1].
> 
> Rationale
> By long tradition trunk is branched months in advance of a Sakai *.0 release.  This practice I regard as counterproductive.  Branching prematurely is costly.   It lowers our velocity as well as our agility.  We let trunk quality slip as we attempt to focus our limited quality assurance efforts on the *.x release/maintenance branch.  We burden ourselves with more work; unproductive work.  Our penchant for alpha/beta phasing also has us thinking we need to branch early (I suggest we simply issue sakai-10.0-qaX labeled tags between now, the freeze date and beyond until we've eliminated all blockers and issue the first RC).  We incur additional branch management overhead in the guise of branch managers, additional Jenkins build jobs and branch-specific QA servers long before it is truly necessary to do so.  We let trunk sink to the status of a pass through branch for fixes destined for the *.x release/maintenance branch for far too long a period of time. 
> 
> There are also timing issues to consider as regards 10 work.  There are a number of potential new capabilities under consideration for 10 that need to be prepped for inclusion, evaluated and moved to the core svn.  I doubt we can get all that done before December.
> 
> In short, I propose that we endeavor to put trunk into a releasable state first before we shift our focus to branch activities.  The trick of course is in the determination of when to branch; our ability to measure certainly requires refinement but I believe we are currently capable of moving towards a releasable trunk without adding undue risk to the project.  Moreover, if we shift our QA efforts to trunk earlier an opportunity exists to reinforce the view that Sakai QA is a vital year-round activity rather than a cyclical affair that slowly cranks itself up and down according to our long drawn out release cycles.  
> 
> Implications
> Developers engaging in post-10 work will need to hold back their changes in a working branch until after trunk is branched to 10.x.  That said, developers looking beyond 10 should weigh carefully considerations of 10 throughput if new work is prioritized over bug fixes and other tasks required to prep trunk for the next release. 10 dev activities will require more on-list visibility, the trunk commit stream will require closer scrutiny and the upcoming code and string freeze dates will need to be regularly communicated.
> 
> To date, two QA servers (Longsight, UvA) have been committed to the 10 effort.  More will be needed.  Neal Caidin will coordinate the server build up as well as general QA efforts directed at trunk.
> 
> 
> A material objection raised by a Sakai PMC member will block this proposal.  Other opinions are welcome, indeed encouraged.  Silence equals consent. 
> 
> Cheers,
> 
> Anth 
> 
> [1] As Zach Thomas has pointed out we can and should refine our trunk development techniques. See his suggested readings for an outline of new approaches that we can incorporate:  
> 
> http://paulhammant.com/2013/04/05/what-is-trunk-based-development/
> http://continuousdelivery.com/2011/05/make-large-scale-changes-incrementally-with-branch-by-abstraction/
> http://martinfowler.com/bliki/FeatureToggle.html
> 
> 
> anthony whyte | its and mlibrary | university of michigan | arwhyte at umich.edu | 517-980-0228
> 
> 
> _______________________________________________
> cle-release-team mailing list
> cle-release-team at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-qa/attachments/20131114/5389365c/attachment.html 