[Building Sakai] Sakai OAE Usability Testing Package 1

Mon Aug 22 17:14:33 PDT 2011

Hi folks,

I want to reassure you that "the usability of things that are important at Berkeley" wasn't meant to be a statement that is in any way divisive or indicate that we are trying to push the product in a direction that inhibits overall coherence. It was just an honest statement, based on the fact that I don't believe certain things that are important to us right now may be as important to other institutions at the moment. 

As a UX practitioner, when I plan a usability test I definitely define tasks that help me understand key issues I'm working on. For example, categories are quite important to us as we feel like this is going to be an important navigational method for our users. [1] However, I am not even sure if any other institutions are using categories so I wouldn't necessarily expect them to be interested in spending precious time with users testing it. 

We thought about adding tasks because seemed like the tasks outlined wouldn't take very long (we often have hour-long usability test sessions). If we started the sessions with the tasks in the protocol, I'm sure it wouldn't change the results to add a few more at the end. As you can see from the instructions it is quite a bit of work & coordination to get users together for testing sessions, so adding tasks would help us limit our coordination overhead and make better use of our time with users. We were hoping, however, that at Berkeley we could add tasks that would both inform the managed project and help answer questions we have locally. 

As a final note, I wasn't aware that there were changes coming soon in categories & profile or I likely would not have suggested testing them (unless of course we could get input to the managed project before folks start work on these changes). I know we have the 90 day plan [2], but I'm not aware of a way to tell which work is coming next--is there a place to find that info? It may also be helpful to have a discussion with Nico of the places in the system that he and Sam view as "unfinished" and not worthy of doing any testing.

Cheers,
Allison

[1] Here is a screenshot, which you can also check out live at calcentral.berkeley.edu, of our categories in case folks are interested in how we've worked certain issues around not having enough hierarchy:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen shot 2011-08-22 at 4.34.33 PM.png
Type: image/png
Size: 136377 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/sakai-dev/attachments/20110822/ccbe8668/attachment.png 
-------------- next part --------------

[2] https://confluence.sakaiproject.org/display/3AK/Sakai+OAE+v1.1+90-day+plan

On Aug 22, 2011, at 6:44 AM, Clay Fenlason wrote:

> Nico and I are working on getting some Amazon instances up for
> usability testing, including pre-populated data IIUC, and I hope we'll
> get that out to you soon.
> 
> A key part of the goal in my mind is to support evidence-based design
> decisions, as opposed to merely trying to compromise between several
> institutional opinions. To that end I think it would be more important
> to start by finding a common set of tests we can all run. For similar
> reasons, I'm leery of formulations like "the usability of things that
> are important to Berkeley." I don't want to read too much into that
> phrase, but if we accept it uncritically we could find ourselves
> sliding back into a process of negotiating opinions and a
> 'contrib/core' mentality that wouldn't help us achieve overall
> coherence. It would at least be premature, as all our design issues
> are fairly general at this stage.
> 
> Do we have the beginning of a group of people who could work to evolve
> and refine the test suite?
> 
> ~Clay
> 
> On Fri, Aug 19, 2011 at 3:39 PM, Allison Bloodworth
> <abloodworth at berkeley.edu> wrote:
>> Hi Anne-Sophie,
>> Thanks much for all the incredible work you've put into this! We are looking
>> forward to starting testing soon.
>> 
>> Rachel & I are going over the protocol in detail now and it would be very
>> helpful to actually run through it on a test server. However, we don't want
>> to start Berkeley's countdown clock on having our server for 3 weeks quite
>> yet--is there a server that is set up now that we could use to run through
>> the test?
>> In our current context (as we have no courses in Sakai OAE) it would
>> actually make more sense to test this on a group, e.g. a "Calculus Research
>> Project group." This would likely also allow us to obtain information about
>> how users in groups function. Do you think it would be problematic to make
>> that change?
>> Additionally, there are a few other things we'd like to add to the test to
>> investigate the usability of other things that are important to us at
>> Berkeley (e.g. categories, profile). We are hoping information from some
>> additional tasks we add would still be helpful to the managed project?
>> Thanks again!
>> Allison
>> On Aug 18, 2011, at 5:59 AM, Clay Fenlason wrote:
>> 
>> I'm very happy to see this, Anne-Sophie, thank you. I know it's early
>> days, and there will be any number of ways people will want to improve
>> this practice as time goes on, but just setting the precedent seems to
>> me a significant step, and not only for the released product. We don't
>> have many UX professionals on staff (I think most Sakai institutions
>> are in the same boat), and if the Sakai community helps us learn how
>> to do this sort of work well it will be yet another advantage for our
>> participation.
>> 
>> ~Clay
>> 
>> On Wed, Aug 17, 2011 at 4:28 PM, Anne-Sophie De Baets
>> <asd38 at caret.cam.ac.uk> wrote:
>> 
>> Hi everyone,
>> 
>> The Sakai OAE project team strongly believes that regular
>> 
>> usability testing of our product across various institutions is
>> 
>> essential in identifying problems, missing areas and improving
>> 
>> overall usability. It is also a great way for the institutions to
>> 
>> provide feedback and start exposing some of their staff and
>> 
>> students to Sakai OAE.
>> 
>> Therefore, Usability Test packages will be sent out from time
>> 
>> to time, aiming to investigate specific areas of Sakai OAE. These
>> 
>> packages will contain everything necessary for each institution
>> 
>> to conduct the usability test themselves. These materials include
>> 
>> a manual with tips and hints for people that have never conducted
>> 
>> usability testing, the actual usability test guide and all other materials
>> 
>> necessary. Each institution willing to participate will also receive
>> 
>> its own (temporary) OAE URL that can be used for testing. All user
>> 
>> accounts, data and materials necessary to run the usability test
>> 
>> will be available on this.
>> 
>> I'm proud to announce that the first Sakai OAE Usability
>> 
>> Testing package is now available at [1]. Its goal is to
>> 
>> investigate various management activities around content
>> 
>> and courses/groups.
>> 
>> We would like to encourage all institutions to participate in this.
>> 
>> Please don't hesitate to get in touch with us if you would have
>> 
>> any feedback or follow-up questions.
>> 
>> [1] https://confluence.sakaiproject.org/display/3AK/Usability+Testing
>> 
>> 
>> Have fun,
>> 
>> Anne-Sophie De Baets
>> 
>> 
>> _______________________________________________
>> 
>> sakai-dev mailing list
>> 
>> sakai-dev at collab.sakaiproject.org
>> 
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> 
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>> 
>> _______________________________________________
>> sakai-dev mailing list
>> sakai-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
>> 
>> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org
>> with a subject of "unsubscribe"
>> 
>> Allison Bloodworth
>> Senior User Experience Designer
>> Educational Technology Services
>> University of California, Berkeley
>> abloodworth at berkeley.edu
>> 510-289-6627
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> sakai-ui-dev mailing list
>> sakai-ui-dev at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-ui-dev
>> 
>> 
> _______________________________________________
> sakai-dev mailing list
> sakai-dev at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
> 
> TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"

Allison Bloodworth
Senior User Experience Designer
Educational Technology Services
University of California, Berkeley
abloodworth at berkeley.edu
510-289-6627