[Contrib: Evaluation System] Evaluation Strategy for 3akai

[Sean DeMonner]

Some related thoughts from U-M:

- We'd really love to use EvalSys for ad-hoc surveys of the type  
Stephen describes. We're still wrestling with getting our "official"  
course evals working the way we want however, so that will like have  
to wait another semester or two.

- Having an ad hoc evaluation capability would also allow us to  
support the affiliated-but-not-fully-integrated Flint and Dearborn  
campuses (in addition to Ann Arbor) because we wouldn't have to feed  
evaluation order info from our PeopleSoft SIS (which currenlty only  
knows about Ann Arbor classes). This would be a big win.

- We've done *a lot* of communications work trying to broaden our  
community's thinking beyond the notion that evals are about the  
"institution checking on the faculty." We heavily emphasize that both  
formative (midterm) and summative (finals) feedback is used by  
faculty to improve teaching, not just to penalize poor instruction.  
That said, many faculty remain skeptical and the "QC" hierarchy does  
tend to mirror administrative hierarchy (e.g. departments typically  
specify questions and receive the collected data in addition to  
instructors).

- We have explicitly tried to avoid coercive techniques to drive  
participation (like making the release of grades contingent upon  
responding to evals). Because evals are a highly politicized issue,  
we have a faculty steering committee that is working to make policy  
recommendations to the Provost on this and other topics.

SMD.





On Mar 11, 2009, at 3:35 PM, Stephen Marquard wrote:

> The deeper we get into assessment vs course evaluations, the more  
> different they look.
>
> If Sakai had "survey technology" then there'd be a stronger  
> overlap. Currently the evaluation tool has one of the strongest  
> feature sets for generic surveys (i.e. it's better at surveys than  
> either T&Q or Mneme, for example through support for blocked  
> questions).
>
> The fact that it's a contrib tool and the code structure that Steve  
> G alludes already make it an independent tool which is integrated  
> (and in different ways at different institutions).
>
> We haven't got into hierarchy much yet, but our Course Evaluation  
> hierarchy use wouldn't  be too different to other academic  
> hierarchy use (e.g. specific access for heads of academic  
> departments).
>
> The characterisation of the "US use case" and "Quality Control" is  
> probably a little narrow. Our use case is largely self-service  
> within a broad institutional policy, tending towards mid-term  
> course evaluations which are more formative than summative.
>
> So from our point of view, the evaluations tool is well positioned,  
> and its integration with Sakai is appealing and of value for all  
> the campus stakeholders in the process. There is also some demand  
> for deeper integration along the lines of the "condition service",  
> e.g. make release of gradebook items or certain resources  
> conditional on completing a course evaluation.
>
> An interesting question is whether there might be a bigger audience  
> for it outside current Sakai-using institutions, if it was easy to  
> use with a "mini Sakai" or something.
>
> Regards
> Stephen
>
>>>> John Norman <john at caret.cam.ac.uk> 03/11/09 8:33 PM >>>
>
> On 11 Mar 2009, at 18:08, Steven Githens wrote:
>> [...]
>>
>> Again, the Evaluation system is an interesting use case since it was
>> designed to be decoupled/standalone and not necessarily depend on
>> the concept of 'sites'.
>>
>> -Steve
>
> I find it interesting for a similar reason. In fact I would question
> whether we add value by including it in Sakai at all. It seems that
> for the US use case - which I am tempted to describe as the
> institution checking on the faculty - the 'independence' of a separate
> solution has merits. The thing that is shared between a course
> management system and a course evaluation system is the roster (staff
> and student, including sections) the course name and potentially some
> aspects of instructor hierarchy within the course (e.g. professor vs
> TA). The hierarchy of Quality Control (who can edit which questions,
> who can see the responses for which courses) seems to be an
> independent hierarchy. Bulk mailing has some overlap with certain
> Sakai features and aspects of questionnaire delivery seem to have high
> potential overlap with Sakai assessment technology, but that has not
> been exploited.
>
> So all in all, I'm strongly tempted to look at it as a stand alone
> system and look at integrating it as such.
>
> John
> _______________________________________________
> evaluation mailing list
> evaluation at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>
> TO UNSUBSCRIBE: send email to evaluation- 
> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
> _______________________________________________
> evaluation mailing list
> evaluation at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>
> TO UNSUBSCRIBE: send email to evaluation- 
> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
>
>

SMD.


==========================================================
Sean DeMonner, IT Senior Project Manager, CTools Implementation Group
Digital Media Commons @ The Duderstadt Center, U-M      (734) 615-9765




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/evaluation/attachments/20090311/acd1c033/attachment.html