[cle-release-team] [Building Sakai] [WG: Sakai QA] Sakai CLE 2.9.0 release status

Mon Mar 26 16:40:57 PDT 2012

Gotcha, good idea.
:-)
-AZ


On Mon, Mar 26, 2012 at 7:39 PM, Steve Swinsburg
<steve.swinsburg at gmail.com> wrote:
> Yeah, I know, I thought we were talking about the testing part. I just just showing the whole workflow.
>
> S
>
>
> On 27/03/2012, at 10:35 AM, Aaron Zeckoski wrote:
>
>> Steve,
>> Pretty much already the case (except the testing part). Check the
>> workflow here (or in jira itself):
>> https://confluence.sakaiproject.org/display/MGT/Sakai+Jira+Guidelines
>>
>> -AZ
>>
>>
>> On Mon, Mar 26, 2012 at 7:15 PM, Steve Swinsburg
>> <steve.swinsburg at gmail.com> wrote:
>>> Oops, I forgot a status of ours. We have Testing in between Ready for Test
>>> and Closed.
>>>
>>> Anyway, I am -100 the idea of closing tickets as an indicator they are
>>> tested. It was what we used to do in Jira 3 and worked ok, but in Jira 4,
>>> closing the tickets means they need to be reopened before you can edit them,
>>> ie to change the merge flag. This just disrupts everything and sends out new
>>> notifications. Closing should mean its done, all work including QA is done,
>>> all merges are done, move on. I know you aren't suggesting this Matthew,
>>> just wanted to put it out there as an issue.
>>>
>>> I think a new field that indicates that an item is being tested is fine. But
>>> you are also going to need one to indicate testing is complete.
>>>
>>> So the workflow could be:
>>>
>>> Ticket created - Open/Unresolved
>>> Developer works on ticket - In progress (not always used, doesn't matter too
>>> much)
>>> Developer resolves ticket - Resolved
>>> QA person takes ticket - Testing
>>>
>>> If issues in testing, reopen.
>>>
>>> If no issues from testing, Verified/Testing complete
>>> Merges are done and issue edited to set appropriate values on branch merge
>>> status fields
>>> Issue closed.
>>>
>>>
>>> cheers,
>>> S
>>>
>>> On 27/03/2012, at 10:06 AM, Matthew Jones wrote:
>>>
>>> Well, as far as merging, the merging is indicated by the Merge status
>>> dropdown which mirrors the statuses of the the other fields including Merge,
>>> Resolved (Ready for Testing?), Closed (Tested).
>>>
>>> I believed that the ideal workflow involved closing the original ticket
>>> (which means it was verified) before the merge could happen. The branch
>>> manager would then set the merge drop down to Resolved. Then QA would test
>>> the branch, and set the Merge flag to Closed after they verified a branch.
>>> But we never had enough QA lately to test the initial issue, let alone
>>> whether or not the merge worked. And Sam is the only one really doing branch
>>> management.
>>>
>>> So we'd basically just merge as soon as that flag was set, if the issue was
>>> really a bug.
>>>
>>> I'm not against this new status, I just feel like it would be used as much
>>> as the "In Progress" status is. I can definitely see the value if there are
>>> multiple QA testers and they don't want other people duplicating effort.
>>> This rarely happens when fixing bugs though.
>>>
>>> -Matthew
>>>
>>> On Mon, Mar 26, 2012 at 6:27 PM, Aaron Zeckoski <azeckoski at unicon.net>
>>> wrote:
>>>>
>>>> We have an extra step of merging into branches at the moment which
>>>> means we (Sakai) have something between Testing complete and Closed so
>>>> that adds a bit more complexity and my preference is to take baby
>>>> steps and not overcomplicate things.
>>>>
>>>> For now, I think we should just add an extra step in between the
>>>> current "Resolved" and "Tested" statuses which indicates it is being
>>>> tested. I don't think we should reassign it because that has some
>>>> negative consequences in terns of reopening and it seems overly
>>>> complex to create a subtask every time a ticket is being verified.
>>>>
>>>> If anyone is hugely against trying this for now then please let me
>>>> know. Otherwise I will proceed with this first thing tomorrow.
>>>>
>>>> This stuff can all be changed later of course.
>>>>
>>>> -AZ
>>>>
>>>>
>>>> On Mon, Mar 26, 2012 at 6:13 PM, Steve Swinsburg
>>>> <steve.swinsburg at gmail.com> wrote:
>>>>> At ANU we renamed some of the existing statuses and added a couple more.
>>>>>
>>>>> So our workflow is:
>>>>>
>>>>> Open/Unresolved
>>>>> In Progress
>>>>> Resolved
>>>>> Staged for Test (you could eliminate this one, it just signals that an
>>>>> issue
>>>>> is on the testing server)
>>>>> Ready for Test
>>>>> Closed
>>>>>
>>>>> In regards to reassigning tickets when working on them, I think that is
>>>>> important so that people know what is happening. If more work needs to
>>>>> be
>>>>> done, just reassign back to the original developer who worked on it.
>>>>>
>>>>> Alternatively we could start using the Project Roles more, and have a
>>>>> defined set of QA people, then adjust the notification scheme so that
>>>>> those
>>>>> people get notified at the appropriate time.
>>>>>
>>>>> cheers,
>>>>> Steve
>>>>>
>>>>> On 27/03/2012, at 4:51 AM, Matthew Jones wrote:
>>>>>
>>>>> Yea Jira never really had any good way of dealing with this (identify
>>>>> who is
>>>>> testing it), and we've talked about it in the past. At Michigan we ended
>>>>> up
>>>>> using a completely separate system (Pivotal Tracker) to manage the QA
>>>>> process because Jira alone just was not doing it. I believe that
>>>>> solution
>>>>> really ended up making things a lot more clearer than trying us use
>>>>> wiki's,
>>>>> jiras, text documents and emails.
>>>>>
>>>>> For multiple users on the same issue, the only really good idea they
>>>>> have is
>>>>> for subtasks (
>>>>>
>>>>> https://confluence.atlassian.com/display/JIRA/How+do+I+assign+issues+to+multiple+users )
>>>>> and I'd even suggested in the past that we could use a plugin like the
>>>>> "Create subtask on transition"
>>>>>
>>>>> (https://studio.plugins.atlassian.com/wiki/display/CSOT/Jira+Create+Subtask+for+transition )
>>>>> which would automatically create a subtask on each issue when it hits
>>>>> resolved that essentially was "Test SAK-XXXXX". When this was resolved
>>>>> then
>>>>> both the parent and subtask were closed.
>>>>>
>>>>> However in the past this was basically dismissed because, well we've got
>>>>> hundreds of issues in the resolved state and nobody was reviewing them
>>>>> anyway (no QA) so what's the point? It just creates more useless tickets
>>>>> that won't be reviewed.
>>>>>
>>>>> I think the tester can watch it, the tester can comment "I'm testing
>>>>> it",
>>>>> but that should be about it. If we had people doing full time testing
>>>>> and
>>>>> actually caught up (where all of these issues were closed) then that's
>>>>> fine.
>>>>>
>>>>> Is there a reason we can't just RENAME the current status from saying
>>>>> "Resolved" to "Resolved - Ready for Testing" and "Closed" to "Closed -
>>>>> Issue
>>>>> Verified". What would another state get us?
>>>>>
>>>>> On Mon, Mar 26, 2012 at 1:35 PM, Noah Botimer <botimer at umich.edu> wrote:
>>>>>>
>>>>>> I don't disagree. The emails are handy, and why I often Watch issues
>>>>>> I've
>>>>>> touched. Maybe that can be an automatic part of marking something
>>>>>> Resolved
>>>>>> (getting added as a watcher)?
>>>>>>
>>>>>> I was also reminded of something last week -- some people get tons of
>>>>>> JIRA
>>>>>> emails and they often go to a folder or just overlooked. In that mode,
>>>>>> a
>>>>>> direct contact is the only effective way forward, and it is probably a
>>>>>> good
>>>>>> practice anyway if there are questions.
>>>>>>
>>>>>> I think you're right about status + fix version, but I am interested in
>>>>>> "who has the ball", which leaving the developer assigned doesn't seem
>>>>>> to
>>>>>> indicate.
>>>>>>
>>>>>> Thanks,
>>>>>> -Noah
>>>>>>
>>>>>> On Mar 26, 2012, at 1:21 PM, Bryan Holladay wrote:
>>>>>>
>>>>>> I'm not a big fan of re-assigning jira's for testing.  When you do
>>>>>> that, the person who was originally assigned won't be notified of any
>>>>>> comments/etc unless they physically clicked "watch" in the ticket.
>>>>>> This is usually the person you want fixing any bugs found.
>>>>>>
>>>>>> how about:
>>>>>>
>>>>>> Resolved + Fix version means it's ready for testing
>>>>>> Closed + fix version means it's tested and confirmed
>>>>>>
>>>>>> -Bryan
>>>>>>
>>>>>> On Mon, Mar 26, 2012 at 1:11 PM, Noah Botimer <botimer at umich.edu>
>>>>>> wrote:
>>>>>>
>>>>>> Is this something we can handle with assignment of the issue? Any
>>>>>> mechanism is going to require some diligence to be useful -- they're
>>>>>> all
>>>>>> roughly equivalent there. But, I think the "who is the contact for this
>>>>>> issue?" question is pretty important. If you are going to commit to
>>>>>> testing
>>>>>> it (claim it), assigning to yourself is a good way to do that.
>>>>>>
>>>>>>
>>>>>> I am just a little hesitant to add more interim states -- our workflow
>>>>>> and
>>>>>> reports are complicated enough as it is. If it's a matter of access,
>>>>>> our few
>>>>>> known QA folks should definitely be able to assign/unassign issues
>>>>>> across
>>>>>> the board.
>>>>>>
>>>>>>
>>>>>> This is just a thought -- I won't be hurt if we end up with another
>>>>>> status. I just think Resolved + Unassigned is a good way to say "this
>>>>>> needs
>>>>>> testing" and Resolved + Assigned is a good way to say "this is being
>>>>>> tested". It may alternatively make sense to set up a mail group/account
>>>>>> like
>>>>>> KERNEL TEAM for QA, and make it the default assignee for newly resolved
>>>>>> issues (for an activity email push for those interested and easier
>>>>>> reports
>>>>>> of items needing to be tested).
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -Noah
>>>>>>
>>>>>>
>>>>>> On Mar 26, 2012, at 12:53 PM, Aaron Zeckoski wrote:
>>>>>>
>>>>>>
>>>>>> It isn't too much trouble. I could probably get to that later today.
>>>>>>
>>>>>> -AZ
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 26, 2012 at 12:48 PM, Diego del Blanco Orobitg
>>>>>>
>>>>>> <diego.delblanco at samoo.es> wrote:
>>>>>>
>>>>>> Ok, Simple way at this moment: add a comment with the "testing" message
>>>>>> in
>>>>>> the Jira.
>>>>>>
>>>>>> Anyway, maybe "testing" intermediate status between Resolved-fixed and
>>>>>> Verified can be useful but as you say if doing it it's not too much
>>>>>> trouble.
>>>>>>
>>>>>>
>>>>>> Diego.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Mensaje original-----
>>>>>>
>>>>>> De: azeckoski at gmail.com [mailto:azeckoski at gmail.com] En nombre de Aaron
>>>>>> Zeckoski
>>>>>>
>>>>>> Enviado el: lunes, 26 de marzo de 2012 13:39
>>>>>>
>>>>>> Para: Diego del Blanco Orobitg
>>>>>>
>>>>>> CC: Miguel Carro Pellicer; Rob.Egan at marist.edu;
>>>>>> cle-release-team at collab.sakaiproject.org
>>>>>>
>>>>>> Asunto: Re: [cle-release-team] [Building Sakai] [WG: Sakai QA] Sakai
>>>>>> CLE
>>>>>> 2.9.0 release status
>>>>>>
>>>>>>
>>>>>> I would encourage you ro think of a process that uses jira itself as a
>>>>>> way
>>>>>> to know if an issue is being tested. This may be an adjustment in the
>>>>>> JIRA
>>>>>> workflow but I can make that without too much trouble. The problem with
>>>>>> the
>>>>>> google docs thing is that it doesn't scale up very well and it requires
>>>>>> insider knowledge so it is a barrier to community participation.
>>>>>>
>>>>>>
>>>>>> Also, in the current workflow, there is a "Verified" status which is
>>>>>> what
>>>>>> issues should be set to when testing is complete. So anything not
>>>>>> verified
>>>>>> yet will be marked as "resolved - fixed" and once testing is complete
>>>>>> it is
>>>>>> marked as "verified".
>>>>>>
>>>>>>
>>>>>> -AZ
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 26, 2012 at 6:58 AM, Diego del Blanco Orobitg
>>>>>> <diego.delblanco at samoo.es> wrote:
>>>>>>
>>>>>> Dear Rob:
>>>>>>
>>>>>>
>>>>>> We are going to begin to test the jira issues as Aaron said. To
>>>>>> coordinate
>>>>>> with you and don't duplicate work, we can create a shared document (in
>>>>>> google docs or in a confluence page...), where each time one of us
>>>>>> selects a
>>>>>> jira to test, we need to add it to the document, so it's easy to check
>>>>>> if an
>>>>>> issue is "in use" at that moment. There is only need to search in the
>>>>>> document. If it's not, then it's free.
>>>>>>
>>>>>>
>>>>>> And in that way we have the full list of jiras tested by each one too.
>>>>>>
>>>>>>
>>>>>> Aaron do you think this is good way or do you recommend other way as
>>>>>> can
>>>>>> be to put comments directly on Jira's indicating that we are working on
>>>>>> this
>>>>>> issue?
>>>>>>
>>>>>>
>>>>>> I think in the shared document will be quicker to search "free issues"
>>>>>> to
>>>>>> work with.
>>>>>>
>>>>>>
>>>>>> Other question... Once tested a Jira if everything is ok.. do we need
>>>>>> to
>>>>>> modify something in the Jira (add a comment or change state) or only
>>>>>> communicate it to the team?
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> Diego.
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Mensaje original-----
>>>>>>
>>>>>> De: azeckoski at gmail.com [mailto:azeckoski at gmail.com] En nombre de
>>>>>>
>>>>>> Aaron Zeckoski Enviado el: viernes, 23 de marzo de 2012 22:36
>>>>>>
>>>>>> Para: Miguel Carro Pellicer; Rob.Egan at marist.edu; Diego del Blanco
>>>>>>
>>>>>> Orobitg
>>>>>>
>>>>>> CC: Sam Ottenhoff; cle-release-team at collab.sakaiproject.org
>>>>>>
>>>>>> Asunto: Re: [Building Sakai] [WG: Sakai QA] Sakai CLE 2.9.0 release
>>>>>>
>>>>>> status
>>>>>>
>>>>>>
>>>>>> Samoo guys,
>>>>>>
>>>>>> Here is the filter of items to QA in order for 2.9. There are quite a
>>>>>> few
>>>>>> (over 600) and there will be more coming in the following weeks.
>>>>>>
>>>>>> Just start cranking through them and you can give a status update at
>>>>>> the
>>>>>> meeting next week.
>>>>>>
>>>>>>
>>>>>> https://jira.sakaiproject.org/secure/IssueNavigator.jspa?mode=hide&req
>>>>>>
>>>>>> uestId=13243
>>>>>>
>>>>>>
>>>>>> I am copying in Rob from Marist since he does some QA as well and you
>>>>>> guys
>>>>>> might want to coordinate some.
>>>>>>
>>>>>>
>>>>>> -AZ
>>>>>>
>>>>>>
>>>>>>
>>>>>> El 22/03/2012 14:07, Aaron Zeckoski escribió:
>>>>>>
>>>>>>
>>>>>> Yes, they are always at 10am US NY time on thursdays.
>>>>>>
>>>>>> I will double check with the group and send you something later today
>>>>>> (maybe a JIRA filter with a series of tickets in it).
>>>>>>
>>>>>> -AZ
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 22, 2012 at 9:03 AM, Miguel Carro Pellicer
>>>>>> <miguel.carro at samoo.es> wrote:
>>>>>>
>>>>>>
>>>>>> Got it, i think we can help in those activities.
>>>>>>
>>>>>>
>>>>>> Assign me some Jiras and we will verify/validate each ticket in the
>>>>>> affected QA servers. Additionaly, we can propose a solution if the Jira
>>>>>> requires a developer intervention.
>>>>>>
>>>>>>
>>>>>> Usually the meetings are at the same hour? We're in GMT+1
>>>>>>
>>>>>>
>>>>>> Regards, Miguel.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> cle-release-team mailing list
>>>>>> cle-release-team at collab.sakaiproject.org
>>>>>> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> cle-release-team mailing list
>>>>> cle-release-team at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> cle-release-team mailing list
>>>>> cle-release-team at collab.sakaiproject.org
>>>>> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>>>
>>>
>>>
>>
>>
>>
>> --
>> Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile
>
> _______________________________________________
> cle-release-team mailing list
> cle-release-team at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/cle-release-team


-- 
Aaron Zeckoski - Software Architect - http://tinyurl.com/azprofile