[cle-release-team] [Building Sakai] [WG: Sakai QA] Sakai CLE 2.9.0 release status
Steve Swinsburg
steve.swinsburg at gmail.com
Mon Mar 26 16:39:53 PDT 2012
Yeah, I know, I thought we were talking about the testing part. I just just showing the whole workflow.
S
On 27/03/2012, at 10:35 AM, Aaron Zeckoski wrote:
> Steve,
> Pretty much already the case (except the testing part). Check the
> workflow here (or in jira itself):
> https://confluence.sakaiproject.org/display/MGT/Sakai+Jira+Guidelines
>
> -AZ
>
>
> On Mon, Mar 26, 2012 at 7:15 PM, Steve Swinsburg
> <steve.swinsburg at gmail.com> wrote:
>> Oops, I forgot a status of ours. We have Testing in between Ready for Test
>> and Closed.
>>
>> Anyway, I am -100 the idea of closing tickets as an indicator they are
>> tested. It was what we used to do in Jira 3 and worked ok, but in Jira 4,
>> closing the tickets means they need to be reopened before you can edit them,
>> ie to change the merge flag. This just disrupts everything and sends out new
>> notifications. Closing should mean its done, all work including QA is done,
>> all merges are done, move on. I know you aren't suggesting this Matthew,
>> just wanted to put it out there as an issue.
>>
>> I think a new field that indicates that an item is being tested is fine. But
>> you are also going to need one to indicate testing is complete.
>>
>> So the workflow could be:
>>
>> Ticket created - Open/Unresolved
>> Developer works on ticket - In progress (not always used, doesn't matter too
>> much)
>> Developer resolves ticket - Resolved
>> QA person takes ticket - Testing
>>
>> If issues in testing, reopen.
>>
>> If no issues from testing, Verified/Testing complete
>> Merges are done and issue edited to set appropriate values on branch merge
>> status fields
>> Issue closed.
>>
>>
>> cheers,
>> S
>>
>> On 27/03/2012, at 10:06 AM, Matthew Jones wrote:
>>
>> Well, as far as merging, the merging is indicated by the Merge status
>> dropdown which mirrors the statuses of the the other fields including Merge,
>> Resolved (Ready for Testing?), Closed (Tested).
>>
>> I believed that the ideal workflow involved closing the original ticket
>> (which means it was verified) before the merge could happen. The branch
>> manager would then set the merge drop down to Resolved. Then QA would test
>> the branch, and set the Merge flag to Closed after they verified a branch.
>> But we never had enough QA lately to test the initial issue, let alone
>> whether or not the merge worked. And Sam is the only one really doing branch
>> management.
>>
>> So we'd basically just merge as soon as that flag was set, if the issue was
>> really a bug.
>>
>> I'm not against this new status, I just feel like it would be used as much
>> as the "In Progress" status is. I can definitely see the value if there are
>> multiple QA testers and they don't want other people duplicating effort.
>> This rarely happens when fixing bugs though.
>>
>> -Matthew
>>
>> On Mon, Mar 26, 2012 at 6:27 PM, Aaron Zeckoski <azeckoski at unicon.net>
>> wrote:
>>>
>>> We have an extra step of merging into branches at the moment which
>>> means we (Sakai) have something between Testing complete and Closed so
>>> that adds a bit more complexity and my preference is to take baby
>>> steps and not overcomplicate things.
>>>
>>> For now, I think we should just add an extra step in between the
>>> current "Resolved" and "Tested" statuses which indicates it is being
>>> tested. I don't think we should reassign it because that has some
>>> negative consequences in terns of reopening and it seems overly
>>> complex to create a subtask every time a ticket is being verified.
>>>
>>> If anyone is hugely against trying this for now then please let me
>>> know. Otherwise I will proceed with this first thing tomorrow.
>>>
>>> This stuff can all be changed later of course.
>>>
>>> -AZ
>>>
>>>
>>> On Mon, Mar 26, 2012 at 6:13 PM, Steve Swinsburg
>>> <steve.swinsburg at gmail.com> wrote:
>>>> At ANU we renamed some of the existing statuses and added a couple more.
>>>>
>>>> So our workflow is:
>>>>
>>>> Open/Unresolved
>>>> In Progress
>>>> Resolved
>>>> Staged for Test (you could eliminate this one, it just signals that an
>>>> issue
>>>> is on the testing server)
>>>> Ready for Test
>>>> Closed
>>>>
>>>> In regards to reassigning tickets when working on them, I think that is
>>>> important so that people know what is happening. If more work needs to
>>>> be
>>>> done, just reassign back to the original developer who worked on it.
>>>>
>>>> Alternatively we could start using the Project Roles more, and have a
>>>> defined set of QA people, then adjust the notification scheme so that
>>>> those
>>>> people get notified at the appropriate time.
>>>>
>>>> cheers,
>>>> Steve
>>>>
>>>> On 27/03/2012, at 4:51 AM, Matthew Jones wrote:
>>>>
>>>> Yea Jira never really had any good way of dealing with this (identify
>>>> who is
>>>> testing it), and we've talked about it in the past. At Michigan we ended
>>>> up
>>>> using a completely separate system (Pivotal Tracker) to manage the QA
>>>> process because Jira alone just was not doing it. I believe that
>>>> solution
>>>> really ended up making things a lot more clearer than trying us use
>>>> wiki's,
>>>> jiras, text documents and emails.
>>>>
>>>> For multiple users on the same issue, the only really good idea they
>>>> have is
>>>> for subtasks (
>>>>
>>>> https://confluence.atlassian.com/display/JIRA/How+do+I+assign+issues+to+multiple+users )
>>>> and I'd even suggested in the past that we could use a plugin like the
>>>> "Create subtask on transition"
>>>>
>>>> (https://studio.plugins.atlassian.com/wiki/display/CSOT/Jira+Create+Subtask+for+transition )
>>>> which would automatically create a subtask on each issue when it hits
>>>> resolved that essentially was "Test SAK-XXXXX". When this was resolved
>>>> then
>>>> both the parent and subtask were closed.
>>>>
>>>> However in the past this was basically dismissed because, well we've got
>>>> hundreds of issues in the resolved state and nobody was reviewing them
>>>> anyway (no QA) so what's the point? It just creates more useless tickets
>>>> that won't be reviewed.
>>>>
>>>> I think the tester can watch it, the tester can comment "I'm testing
>>>> it",
>>>> but that should be about it. If we had people doing full time testing
>>>> and
>>>> actually caught up (where all of these issues were closed) then that's
>>>> fine.
>>>>
>>>> Is there a reason we can't just RENAME the current status from saying
>>>> "Resolved" to "Resolved - Ready for Testing" and "Closed" to "Closed -
>>>> Issue
>>>> Verified". What would another state get us?
>>>>
>>>> On Mon, Mar 26, 2012 at 1:35 PM, Noah Botimer <botimer at umich.edu> wrote:
>>>>>
>>>>> I don't disagree. The emails are handy, and why I often Watch issues
>>>>> I've
>>>>> touched. Maybe that can be an automatic part of marking something
>>>>> Resolved
>>>>> (getting added as a watcher)?
>>>>>
>>>>> I was also reminded of something last week -- some people get tons of
>>>>> JIRA
>>>>> emails and they often go to a folder or just overlooked. In that mode,
>>>>> a
>>>>> direct contact is the only effective way forward, and it is probably a
>>>>> good
>>>>> practice anyway if there are questions.
>>>>>
>>>>> I think you're right about status + fix version, but I am interested in
>>>>> "who has the ball", which leaving the developer assigned doesn't seem
>>>>> to
>>>>> indicate.
>>>>>
>>>>> Thanks,
>>>>> -Noah
>>>>>
>>>>> On Mar 26, 2012, at 1:21 PM, Bryan Holladay wrote:
>>>>>
>>>>> I'm not a big fan of re-assigning jira's for testing. When you do
>>>>> that, the person who was originally assigned won't be notified of any
>>>>> comments/etc unless they physically clicked "watch" in the ticket.
>>>>> This is usually the person you want fixing any bugs found.
>>>>>
>>>>> how about:
>>>>>
>>>>> Resolved + Fix version means it's ready for testing
>>>>> Closed + fix version means it's tested and confirmed
>>>>>
>>>>> -Bryan
>>>>>
>>>>> On Mon, Mar 26, 2012 at 1:11 PM, Noah Botimer <botimer at umich.edu>
>>>>> wrote:
>>>>>
>>>>> Is this something we can handle with assignment of the issue? Any
>>>>> mechanism is going to require some diligence to be useful -- they're
>>>>> all
>>>>> roughly equivalent there. But, I think the "who is the contact for this
>>>>> issue?" question is pretty important. If you are going to commit to
>>>>> testing
>>>>> it (claim it), assigning to yourself is a good way to do that.
>>>>>
>>>>>
>>>>> I am just a little hesitant to add more interim states -- our workflow
>>>>> and
>>>>> reports are complicated enough as it is. If it's a matter of access,
>>>>> our few
>>>>> known QA folks should definitely be able to assign/unassign issues
>>>>> across
>>>>> the board.
>>>>>
>>>>>
>>>>> This is just a thought -- I won't be hurt if we end up with another
>>>>> status. I just think Resolved + Unassigned is a good way to say "this
>>>>> needs
>>>>> testing" and Resolved + Assigned is a good way to say "this is being
>>>>> tested". It may alternatively make sense to set up a mail group/account
>>>>> like
>>>>> KERNEL TEAM for QA, and make it the default assignee for newly resolved
>>>>> issues (for an activity email push for those interested and easier
>>>>> reports
>>>>> of items needing to be tested).
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Noah
>>>>>
>>>>>
>>>>> On Mar 26, 2012, at 12:53 PM, Aaron Zeckoski wrote:
>>>>>
>>>>>
>>>>> It isn't too much trouble. I could probably get to that later today.
>>>>>
>>>>> -AZ
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 26, 2012 at 12:48 PM, Diego del Blanco Orobitg
>>>>>
>>>>> <diego.delblanco at samoo.es> wrote:
>>>>>
>>>>> Ok, Simple way at this moment: add a comment with the "testing" message
>>>>> in
>>>>> the Jira.
>>>>>
>>>>> Anyway, maybe "testing" intermediate status between Resolved-fixed and
>>>>> Verified can be useful but as you say if doing it it's not too much
>>>>> trouble.
>>>>>
>>>>>
>>>>> Diego.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----Mensaje original-----
>>>>>
>>>>> De: azeckoski at gmail.com [mailto:azeckoski at gmail.com] En nombre de Aaron
>>>>> Zeckoski
>>>>>
>>>>> Enviado el: lunes, 26 de marzo de 2012 13:39
>>>>>
>>>>> Para: Diego del Blanco Orobitg
>>>>>
>>>>> CC: Miguel Carro Pellicer; Rob.Egan at marist.edu;
>>>>> cle-release-team at collab.sakaiproject.org
>>>>>
>>>>> Asunto: Re: [cle-release-team] [Building Sakai] [WG: Sakai QA] Sakai
>>>>> CLE
>>>>> 2.9.0 release status
>>>>>
>>>>>
>>>>> I would encourage you ro think of a process that uses jira itself as a
>>>>> way
>>>>> to know if an issue is being tested. This may be an adjustment in the
>>>>> JIRA
>>>>> workflow but I can make that without too much trouble. The problem with
>>>>> the
>>>>> google docs thing is that it doesn't scale up very well and it requires
>>>>> insider knowledge so it is a barrier to community participation.
>>>>>
>>>>>
>>>>> Also, in the current workflow, there is a "Verified" status which is
>>>>> what
>>>>> issues should be set to when testing is complete. So anything not
>>>>> verified
>>>>> yet will be marked as "resolved - fixed" and once testing is complete
>>>>> it is
>>>>> marked as "verified".
>>>>>
>>>>>
>>>>> -AZ
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 26, 2012 at 6:58 AM, Diego del Blanco Orobitg
>>>>> <diego.delblanco at samoo.es> wrote:
>>>>>
>>>>> Dear Rob:
>>>>>
>>>>>
>>>>> We are going to begin to test the jira issues as Aaron said. To
>>>>> coordinate
>>>>> with you and don't duplicate work, we can create a shared document (in
>>>>> google docs or in a confluence page...), where each time one of us
>>>>> selects a
>>>>> jira to test, we need to add it to the document, so it's easy to check
>>>>> if an
>>>>> issue is "in use" at that moment. There is only need to search in the
>>>>> document. If it's not, then it's free.
>>>>>
>>>>>
>>>>> And in that way we have the full list of jiras tested by each one too.
>>>>>
>>>>>
>>>>> Aaron do you think this is good way or do you recommend other way as
>>>>> can
>>>>> be to put comments directly on Jira's indicating that we are working on
>>>>> this
>>>>> issue?
>>>>>
>>>>>
>>>>> I think in the shared document will be quicker to search "free issues"
>>>>> to
>>>>> work with.
>>>>>
>>>>>
>>>>> Other question... Once tested a Jira if everything is ok.. do we need
>>>>> to
>>>>> modify something in the Jira (add a comment or change state) or only
>>>>> communicate it to the team?
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> Diego.
>>>>>
>>>>>
>>>>>
>>>>> -----Mensaje original-----
>>>>>
>>>>> De: azeckoski at gmail.com [mailto:azeckoski at gmail.com] En nombre de
>>>>>
>>>>> Aaron Zeckoski Enviado el: viernes, 23 de marzo de 2012 22:36
>>>>>
>>>>> Para: Miguel Carro Pellicer; Rob.Egan at marist.edu; Diego del Blanco
>>>>>
>>>>> Orobitg
>>>>>
>>>>> CC: Sam Ottenhoff; cle-release-team at collab.sakaiproject.org
>>>>>
>>>>> Asunto: Re: [Building Sakai] [WG: Sakai QA] Sakai CLE 2.9.0 release
>>>>>
>>>>> status
>>>>>
>>>>>
>>>>> Samoo guys,
>>>>>
>>>>> Here is the filter of items to QA in order for 2.9. There are quite a
>>>>> few
>>>>> (over 600) and there will be more coming in the following weeks.
>>>>>
>>>>> Just start cranking through them and you can give a status update at
>>>>> the
>>>>> meeting next week.
>>>>>
>>>>>
>>>>> https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&req
>>>>>
>>>>> uestId=13243
>>>>>
>>>>>
>>>>> I am copying in Rob from Marist since he does some QA as well and you
>>>>> guys
>>>>> might want to coordinate some.
>>>>>
>>>>>
>>>>> -AZ
>>>>>
>>>>>
>>>>>
>>>>> El 22/03/2012 14:07, Aaron Zeckoski escribió:
>>>>>
>>>>>
>>>>> Yes, they are always at 10am US NY time on thursdays.
>>>>>
>>>>> I will double check with the group and send you something later today
>>>>> (maybe a JIRA filter with a series of tickets in it).
>>>>>
>>>>> -AZ
>>>>>
>>>>>
>>>>> On Thu, Mar 22, 2012 at 9:03 AM, Miguel Carro Pellicer
>>>>> <miguel.carro at samoo.es> wrote:
>>>>>
>>>>>
>>>>> Got it, i think we can help in those activities.
>>>>>
>>>>>
>>>>> Assign me some Jiras and we will verify/validate each ticket in the
>>>>> affected QA servers. Additionaly, we can propose a solution if the Jira
>>>>> requires a developer intervention.
>>>>>
>>>>>
>>>>> Usually the meetings are at the same hour? We're in GMT+1
>>>>>
>>>>>
>>>>> Regards, Miguel.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> cle-release-team mailing list
>>>>> cle-release-team at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team
>>>>>
>>>>
>>>> _______________________________________________
>>>> cle-release-team mailing list
>>>> cle-release-team at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> cle-release-team mailing list
>>>> cle-release-team at collab.sakaiproject.org
>>>> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team
>>>>
>>>
>>>
>>>
>>> --
>>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>>
>>
>>
>
>
>
> --
> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
More information about the cle-release-team
mailing list