[Using Sakai] Large Scale Quizzing

[Sean DeMonner]

Yes, the issues at U-M were related to the use of the EVALSYS tool,  
locally referred to as Teaching Questionnaires (TQ). There was a  
Sakai-wide conference call this past Monday (5/11) to review some of  
the issues and to discuss approaches for remediation:

	http://confluence.sakaiproject.org/confluence/x/ogU2Aw

Unfortunately, there we many contributing factors to the problems, so  
a "nutshell" description is probably not very useful. That said, very  
high TQ response rates (peaking at 160+ per minute), high background  
load (9000+ concurrent users), and several instances of inefficient  
code resulted in load-related outages.

There will be more discussion of these events and what is being done  
to address the problems at the Boston conference.

SMD.



On May 13, 2009, at 12:29 PM, Luke Fernandez wrote:

> Just for clarification was the recent wired campus article about the
> eval challenges at Michigan associated with the use of the evaluation
> system tool?  Or was it something else?
>
> http://chronicle.com/wiredcampus/article/3734/umich-online-teaching- 
> evaluation-system-fails
>
> Is there a way in a technical nutshell to describe what the challenge
> was? (e.g. load 'cuz of an unoptimized query, etc.?)
>
> Luke
>
> On Wed, May 13, 2009 at 9:56 AM, Ricci, Margaret P  
> <mricci at indiana.edu> wrote:
>> On a related note, Stephen, since you seem to be using T&Q, how  
>> are you finding it to behave in large sites?  We are considering  
>> it at Indiana, but we are concerned about how well it scales.  The  
>> rumors are that when we tried it before, it didn't scale well.  It  
>> seems like you folks would be a good test since you are using it  
>> on such a large scale.
>>
>> Thanks,
>> Maggie
>>
>>
>> Margaret Ricci
>> Instructional Technology Consultant and Developer
>> Teaching and Learning Technologies Centers
>> 812-855-8355
>> mricci at indiana.edu
>>
>>
>>
>> -----Original Message-----
>> From: sakai-user-bounces at collab.sakaiproject.org [mailto:sakai- 
>> user-bounces at collab.sakaiproject.org] On Behalf Of Stephen Marquard
>> Sent: Wednesday, May 13, 2009 11:47 AM
>> To: Sakai Dev; Sakai; Dave McPherson
>> Subject: Re: [Using Sakai] Large Scale Quizzing
>>
>> We have successfully used the contrib Evaluation System to deliver  
>> a survey to a site with 27000 users in it (4000+ responses).  
>> EvalSys will not associate a response with a username though in  
>> the export as its designed for anonymous surveys (unless you tweak  
>> the code to do so).
>>
>> We recently also had T&Q in a similarly sized site, but had  
>> problems exporting the results until we dropped the site  
>> membership of the site (then the export worked fine).
>>
>> Regards
>> Stephen
>>
>>>>> Dave McPherson <dmcphers at vt.edu> 05/13/09 5:33 PM >>>
>> Hi,
>>
>> We are considering using Sakai to deploy  a large scale survey to our
>> faculty on VT's campus.  This would be around 3,500 respondents to a
>> single question quiz. We would need to know who answered each quiz  
>> and
>> then be able to export the answers to an external application.  I'm
>> wondering if anyone has done anything like this in the past or if  
>> they
>> know of any limitations in Sakai that would make this a bad idea.
>>
>> Thanks,
>> Dave McPherson
>> --
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> David McPherson              Virginia Tech
>> dmcphers at vt.edu        Learning Technologies
>> 540 231-8993                     Manager, OCS
>> fax 540 231-5922               1200A Torgersen Hall
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>>
>>
>> _______________________________________________
>> sakai-user mailing list
>> sakai-user at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-user
>>
>> TO UNSUBSCRIBE: send email to sakai-user- 
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>> _______________________________________________
>> sakai-user mailing list
>> sakai-user at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/sakai-user
>>
>> TO UNSUBSCRIBE: send email to sakai-user- 
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>>
> _______________________________________________
> sakai-user mailing list
> sakai-user at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/sakai-user
>
> TO UNSUBSCRIBE: send email to sakai-user- 
> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>

SMD.


==========================================================
Sean DeMonner, IT Senior Project Manager, CTools Implementation Group
Digital Media Commons @ The Duderstadt Center, U-M      (734) 615-9765




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-user/attachments/20090513/eea8fac1/attachment.html