[Building Sakai] Sakai OAE Usability Testing Package 1

[Anne-Sophie De Baets]

Hi Allison,

Thanks a lot for your feedback. Some comments inline:

> Rachel & I are going over the protocol in detail now and it would be very
> helpful to actually run through it on a test server. However, we don't
> want to start Berkeley's countdown clock on having our server for 3 weeks
> quite yet--is there a server that is set up now that we could use to run
> through the test?

That makes sense. You can experiment with the 3akai.sakaiproject.org
server, using the same accounts and data as described in the manual,
which have been created on there as well. However, the server is not
running the latest code, so bugs might still exist. We'll reset the
server to its initial state from time to time, meaning that you will
lose all of the changes you make to the test accounts and data.

> In our current context (as we have no courses in Sakai OAE) it would
> actually make more sense to test this on a group, e.g. a "Calculus
> Research Project group." This would likely also allow us to obtain
> information about how users in groups function. Do you think it would be
> problematic to make that change?

In general, it would be good if we could keep the testing scenario
as close to the original test as possible, just to make it easier
to analyse results coming from different institutions. However, in
this case, I think the change you'd like to make should be fine. You'll
still be testing the same functionality and interface elements, as
you're basically just changing the name and type.

It's probably worth keeping in mind that the server will already have
the Maths course pre-created for you. This means that you'll have
to create the Mathematics Research Project group yourself and modify
the content of the messages present in the system. It is important
to keep the content of the group and messages as close as possible
to the originals.

> Additionally, there are a few other things we'd like to add to the test to
> investigate the usability of other things that are important to us at
> Berkeley (e.g. categories, profile). We are hoping information from some
> additional tasks we add would still be helpful to the managed project?

I think it's great that you're keen to test other things as well, but
for this first package I would encourage institutions to run the test
as closely as possible to the original without changing or expanding
the scope and goals. We're still trying to figure out how we can
best analyse and summarise results coming from different institutions,
so for now it would be more helpful if all participating institutions
conducted the same focused test.

Another reason why it might not be the best time to test categories and
profiles is that both of these are currently undergoing design changes.
It feels more useful to create a usability package that tests these
changes, rather than the current solution which will disappear soon.

However, it's really valuable to find out what you would like to test
as well. The goal is to send out small focused usability test packages
at regular times, and that information could help us in deciding what
to test next. Tomorrow, I will create a Confluence page where everyone
can list what they would like to test in the next usability package. I'll
send out a note once that page is up.

Best wishes,
Anne-Sophie