[sakai2-tcc] Test plans for sakai 2.8

[Berg, Alan]

The approach I will  take is to ask Project leads to update links in a confluence page for test plans per project before going to beta via pointing to old test plans that still work, new test plans or a goal driven test pointing to links in help. 

This process needs to gain momentum, therefore I ask for a vote on:

      Do we stop moving to beta tagging if there is not enough test plan coverage.

If there is a negative vote I would like to also hear about any proposed improvement in the process of obtaining decent coverage.

Alan

Alan Berg
QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg

________________________________________
From: Jean-Francois Leveque [jean-francois.leveque at upmc.fr]
Sent: 06 September 2010 10:46
To: Aaron Zeckoski
Cc: Berg, Alan; sakai2-tcc at collab.sakaiproject.org
Subject: Re: [sakai2-tcc] Test plans for sakai 2.8

I do.

J-F

Aaron Zeckoski a écrit :
> I think this is worth considering as part of the release process and
> at least warrants discussion. Keeping help files and/or test plans up
> to date seems like it deserves a line item in the release process at a
> minimum. Does anyone else agree?
>
> -AZ
>
>
> On Fri, Sep 3, 2010 at 9:20 AM, Berg, Alan <A.M.Berg at uva.nl> wrote:
>>  Hi TCC,
>>
>> In the 2.7 QA push there was inconsistent coverage of the functionality by
>> test plans (if you can define great holes in the coverage inconsistent).
>>  Worse still some of the plans were out of date or incorrect. Test plan
>> generation is a burden for hard working developers and I assume the
>> generation is seen as an unwelcome task or an optional extra. I also noticed
>> that some of the context help was starting to drift as features changed.
>>
>> I want to be realistic and find a middle ground that allow functional
>> testers to do their job without me nagging the project leaders and that is
>> not overly costly. I have the following suggestions for debate.
>>
>> The minimum
>>
>> 1)      1) Any Test plans that still work from the 2.7 cycle are pointed at
>> by the relevant project leads in a QA page on confluence. The page is
>> finished by the alpha beta tagging boundary.
>>
>> 2)      2) At the alpha beta boundary help is up to date qua any feature
>> changes. Help can then be used as a basis for Goal orientated tests such as
>> make an assignment, add a grade book etc. Simple plans of less than a
>> paragraph pointing to help links are made by the projects and are updated in
>> a page under QA on confluence. Links to testing resources such as zip files
>> point into subversion. This naturally forces help to remain current.
>>
>> 3)      3) Any new feature changes (from 2.7 to 2.8) are documented in a
>> thorough test plan. For example of what is acceptable, see the test plan
>> from BasicLTI in subversion.
>>
>>
>>
>> If the TCC can help enforce this as a standard especially when it comes down
>> to the alpha beta boundary then the functional testers will have enough
>> information for wide coverage of testing. I would even strongly suggest we
>> cannot go to beta without specifics of what to test.
>>
>>
>>
>> I still consider the work of automated Functional testing via Hudson and
>> perhaps selenium worth driving, but others need to develop the specifics of
>> this effort due to time constraints.
>>
>> Please have an opinion it improves the quality of QA itself.
>>
>> Alan
>>
>> Alan Berg
>> QA Director - The Sakai Foundation
>>
>> Senior Developer / Quality Assurance
>> Group Education and Research Services
>> Central Computer Services
>> University of Amsterdam
>>
>> http://home.uva.nl/a.m.berg
>> _______________________________________________
>> sakai2-tcc mailing list
>> sakai2-tcc at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc