[WG: Sakai QA] [Management] [Building Sakai] Release planning update

Eli Cochran eli at media.berkeley.edu
Sun Sep 6 16:23:57 PDT 2009


I agree with John here on both counts.

_User needs_ and _user effects_ should be considered for any change.  
There are very few changes which don't come from the former and have  
the latter. Why fix bugs or optimize code unless that fix or that  
optimization makes life better for someone, even if it's just the IT  
guy.

Good product definition and design takes time, attention, and/or money  
(either in the form of in-house salary or contractor costs). But those  
costs are necessary and must be accounted for when approaching new  
development. Otherwise we end up with mediocre product or the wrong  
product.

But to Clay's point, the 11th hour is usually too late to *fix* the  
user experience. The biggest contribution to good UX is defining the  
real problem and figuring out the right solution. Designing screens is  
critically important but it's a by-product of good product definition  
and is much easier when your designing for the right solution. Which  
brings us back to expressing the project in terms of user need and  
desired user effect.

- Eli


On Sep 5, 2009, at 12:51 AM, John Norman wrote:

> I could read this as dismissing the notion as too hard for Sakai 2. I
> hope that is a wrong interpretation. I think Chuck Hedrick made a
> point about changes that affect the user experience needing to be
> treated differently to changes that fix things behind the scenes.
>
> Personally, I would like to see this approach applied if any change is
> proposed that affects the user interaction. If a team doesn't have UX
> resource, IMHO it should recruit some. Perhaps publishing wireframes
> (as Indiana has done in the past) so others can comment on the change.
>
> John
>
> On 4 Sep 2009, at 02:32, Clay Fenlason wrote:
>
>> Terrific :) That's certainly the sort of thing we aspire to.
>>
>> If compromises are made in the near term on the design-led
>> considerations you champion, it's because we also don't want to  
>> derail
>> development work by introducing deep new requirements at the 11th
>> hour. Not every Sakai team has had the people or skillset chemistry  
>> to
>> do its work in this way, nor the time to retool its approach at this
>> stage. We're trying to strike the right balance between desirability
>> and difficulty, given the resource pressures we all feel. That  
>> balance
>> could swing a different direction in other circumstances.
>>
>> The work going toward Sakai 3, for example, since it has the luxury  
>> of
>> starting with a clean slate, will be measured more deeply against its
>> design considerations. Some things are easier to start with than
>> retrofit for, and so we'd be well-advised to lead with them, as you
>> say.
>>
>> ~Clay
>>
>> On Thu, Sep 3, 2009 at 9:04 PM, Keli Amann<kamann at stanford.edu>  
>> wrote:
>>> Hi  Clay
>>> Regarding [3], inclusion criteria
>>> (http://confluence.sakaiproject.org/display/MGT/New+Feature+Documentation
>>> ),
>>> that is certainly the minimum for consideration. However, Stanford  
>>> is
>>> considering trying to see if we can build in a user experience
>>> review as
>>> early in this process as possible, in a way we weren't able to do
>>> for 2.6.
>>>
>>> Some user needs are universal and some are very unique. We don't
>>> want to
>>> stop other teams from addressing unique needs, but we also don't
>>> want to
>>> disrupt or confuse current users with additional choices they
>>> didn't have to
>>> make in the past. We don't yet have set standards, but reflecting
>>> on our
>>> past experience with contributed code, a few guidelines seem to be
>>> emerging.
>>>
>>> 1)   When adding new choices for users, we want to design it so it
>>> doesn't
>>> break the default behavior of the system--or the user. The new
>>> choice should
>>> be discoverable, but it should not become the default, either
>>> directly or
>>> indirectly (ex. a new button is in same position as an older, more
>>> commonly
>>> used one). The exception to this would be if the change is fixing a
>>> usability issue
>>>
>>> 2) We want to think about how a change affects users other than the
>>> type of
>>> user that requested the change. For instance, a change made for
>>> instructors
>>> might affect students.
>>>
>>> These guidelines should be woven through out conception and
>>> planning, not
>>> just testing (QA or UX) when it is harder to make changes. However,
>>> it's
>>> sometimes hard for a new team contributing functionality to know the
>>> implications of the changes they are thinking about without hearing
>>> the
>>> perspectives of others who are either more experienced with the
>>> tool or who
>>> have different perspectives. Even Stanford has a hard time keeping
>>> track of
>>> all the implication of changes within Tests & Quizzes, so I can't
>>> imagine
>>> what it's like for other teams.
>>>
>>> Stanford has been advised to look at the practices of the Open  
>>> Source
>>> Portfolio. Features do not get included without vetting by a
>>> committee. In
>>> fact, the committee encourages new contributions to be reviewed
>>> first as
>>> wireframes and specs, not code, so they can give feedback before it
>>> is
>>> built, saving developers time.
>>>
>>> So yes, I think we'd agree that documentation must be submitted at
>>> minimum
>>> for consideration, but we may even encourage the sharing of
>>> documentation
>>> before code is written. Regardless of *when* we review
>>> documentation though,
>>> our biggest challenge will be managing the time it takes to do so,
>>> as more
>>> teams begin to contribute code. It's a necessary step, but we may
>>> need to
>>> think harder about how to find the resources to make sure that  
>>> review
>>> occurs.
>>>
>>> Keli Amann
>>> User Experience Specialist, CourseWork
>>> Academic Computing, Stanford University
>>>
>>>
>>> On Sep 3, 2009, at 11:27 AM, Clay Fenlason wrote:
>>>
>>>> It's time again to try to summarize a range of discussion and
>>>> activity. I'm sure to fail to be comprehensive, and I'm not quite
>>>> comfortable with some areas that don't have clear resolution, yet  
>>>> it
>>>> feels more important to talk about things as we go than to tie up
>>>> all
>>>> the loose ends before speaking out and inviting comment.  There is
>>>> much to say. What follows is my understanding of the current state
>>>> of
>>>> affairs for Sakai releases in the coming year.
>>>>
>>>> 2.6 MAINTENANCE RELEASES
>>>>
>>>> The first point is that the 2.6 branch of code still needs
>>>> attention,
>>>> including maintenance releases, and that needs to be a priority.
>>>> The
>>>> current plan is to have two such releases before the end of the
>>>> year,
>>>> the first of which should be ready by the end of September.  2.6.x
>>>> QA
>>>> servers with known revisions are already starting to go up.
>>>>
>>>> 2.7 FEATURE RELEASE(S)
>>>>
>>>> The initial proposal to have two full Sakai feature releases in the
>>>> coming year [1] has been abandoned, for reasons more or less laid
>>>> out
>>>> in an earlier thread [2]. We are now moving ahead with a rough
>>>> plan to
>>>> have a 2.7 by the end of April which will include some independent
>>>> tool releases, very likely including entirely new tools or sets of
>>>> capabilities if they can work their way through the new product
>>>> development process in time.
>>>>
>>>> What "in time" means is less clear than it has been in the past,  
>>>> and
>>>> that's a problem, but I think that, on balance, this reflects an
>>>> improvement in our state of affairs. We can begin testing with  
>>>> tools
>>>> and components that are ready to go now, and provide a measured way
>>>> (i.e. the independent release process) for testing to incorporate
>>>> them
>>>> later, one that takes into account the scope of change and risk  
>>>> that
>>>> these capabilities represent, along with the amount of QA time and
>>>> resource that should go toward them.
>>>>
>>>> In more specific terms, I think we can begin very soon (before the
>>>> next week is out) to test the set of changes in view for Forums  
>>>> (or,
>>>> more precisely, msgcntr) against a branch of trunk, then once  
>>>> that's
>>>> finished bring in a release of Tests&Quizzes, do more regression
>>>> testing, and so forth.  The details are under the purview of Pete
>>>> Peterson's coordination of QA resources, along with the
>>>> participation
>>>> of particular development teams, and much of that still needs to be
>>>> squared away in conversation on the details, which has so far been
>>>> happening behind the scenes. Part of the delay is the convergence  
>>>> of
>>>> vacations and start-of-term emergencies for key players, something
>>>> that always happen this time of year, but there are ways we can
>>>> make a
>>>> productive start without knowing exactly how we're going to
>>>> finish. My
>>>> specific hope, again, is to see a clearer plan developed by the
>>>> end of
>>>> next week, and to jumpstart testing for an independent Messages
>>>> +Forums
>>>> release at about the same time.
>>>>
>>>> Anthony Whyte has analyzed comparisons of 2.6.x against 2.6.0 as
>>>> well
>>>> as trunk against 2.6.x.  He will offer details once his laptop  
>>>> dries
>>>> out (!), but the upshot appears to be that we should have a
>>>> maintenance release based on a known revision of 2.6.x before the
>>>> month is out, and also that we can and should base any independent
>>>> tool releases against a branch of trunk (i.e. rather than against
>>>> 2.6.x).
>>>>
>>>> MAINTENANCE TEAM
>>>>
>>>> Anthony Whyte is taking the lead on recruiting dedicated
>>>> participation
>>>> for a maintenance team, which would take on challenges of
>>>> bugfixing in
>>>> areas of the code where it's most needed. I mention it here because
>>>> this team will doubtless be a significant factor in issue  
>>>> resolution
>>>> during the coming year, it should accelerate our ability to produce
>>>> quality releases as a result, and it will have some voice in the
>>>> question of release-readiness of a body of code.  More details
>>>> should
>>>> be coming out about this in the near future.
>>>>
>>>> 2.7 INCLUSION CRITERIA
>>>>
>>>> We have also developed a set of expectations for the evolutionary
>>>> changes of existing code [3]. The criteria are largely matters of
>>>> self-documentation which seem less than controversial, apart from
>>>> some
>>>> ambiguity about what good coding practices are: that needs to get
>>>> sorted out. Nevertheless, the basic expectation is that a failure  
>>>> to
>>>> have this documentation in hand will oblige the release team to
>>>> either
>>>> not include a change/patch, or roll back the code/release to a
>>>> prior,
>>>> known stable point.  In most cases this would likely mean
>>>> reverting to
>>>> the 2.6.x version of the code.
>>>>
>>>> At the time of this writing, a breakdown of open questions about
>>>> planned changes is available in a Google Spreadsheet. [4] Its
>>>> contents
>>>> are limited for the moment to Forums, Tests & Quizzes, and Site
>>>> Info,
>>>> the three core tools with the biggest set of planned changes.   
>>>> Where
>>>> you see a question mark is a point where a change does not yet have
>>>> sufficient documentation for inclusion. This document of course  
>>>> also
>>>> needs some work, and needs to be further informed by Anthony's
>>>> branch
>>>> analysis, but it offers a fair indication of what's needed at this
>>>> stage.
>>>>
>>>> RELEASE TEAM GAPS
>>>>
>>>> We do not yet have a branch manager for 2.7. This is concerning, it
>>>> impedes our ability to move ahead with testing and release
>>>> management
>>>> for the cycle, and it may even raise the question of whether the
>>>> community is sufficiently motivated to have a 2.7 at all. It's also
>>>> been suggested that we need more than one branch manager, though
>>>> presumably some of the pressure will be relieved if a few tools are
>>>> spun off into their own release management. Other efforts to  
>>>> recruit
>>>> volunteers will continue.
>>>>
>>>> Apologies for the length, and where I've made omissions or  
>>>> expressed
>>>> things hastily, but I trust to questions and comments to tease  
>>>> those
>>>> out and correct me.
>>>>
>>>> ~Clay
>>>>
>>>> [1] http://confluence.sakaiproject.org/x/CoLgAw
>>>> [2]
>>>> http://www.nabble.com/-Building-Sakai--Release-proposal-revisited-to25046188.html
>>>> [3] http://confluence.sakaiproject.org/x/6ATtAw
>>>> [4]
>>>> https://spreadsheets.google.com/ccc?key=0AlfbHxo2qpHEdHZYeVNjekZOTFN3bVdmWnk0ZmwxUGc&hl=en
>>>> _______________________________________________
>>>> sakai-dev mailing list
>>>> sakai-dev at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>>>>
>>>> TO UNSUBSCRIBE: send email to
>>>> sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of
>>>> "unsubscribe"
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>> _______________________________________________
>> sakai-qa mailing list
>> sakai-qa at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-qa
>>
>> TO UNSUBSCRIBE: send email to sakai-qa-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>
> _______________________________________________
> management mailing list
> management at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/management
>
> TO UNSUBSCRIBE: send email to management-unsubscribe at collab.sakaiproject.org 
>  with a subject of "unsubscribe"

. . . . . . . . . . .  .  .   .    .      .         .              .                     .

Eli Cochran
user interaction developer
ETS, UC Berkeley




More information about the sakai-qa mailing list