[Contrib: Evaluation System] Question > How to push evaluation to all courses via Admin?
Sean DeMonner
demonner at umich.edu
Fri Jul 17 13:40:13 PDT 2009
Faust,
Some of this information is captured in the panel presentation that
was delivered in Boston last week. Ellen B has uploaded the slides to
the Confluence Home page for the project:
http://confluence.sakaiproject.org/display/EVALSYS/Home
And here's the direct link to the slides:
http://confluence.sakaiproject.org/download/attachments/35029038/
UsingSakai4TeachingEvaluations.ppt
At U-M we have seen no significant difference in the ratings on the 4
University-wide, 5 point Likert scale questions we use for all evals.
This has been true across several pilots and full-scale deployments.
We have however seen a dip in response rates vs. traditional paper
(which historically saw response rates of 75%-80% depending on who
you ask).
Here's a chart showing the last 4 mid-term pilots (which typically
see lower response rates), and end-of-term results:

Fall 08 was the first full-scale end-of-term evaluation period and we
enjoyed what we felt were pretty good response rates (73% of students
participated, submitting 61% of possible evaluations). However, those
rates took a big dip in Winter 09 (due in part to technical
problems), and our Spring rates were lower still. Understandably,
academic leadership is quite concerned, and we are in fact meeting on
Monday to discuss options for boosting rates. Up til now we have
avoided carrots and sticks per the direction of the Provost's
Teaching Questionnaires task force which helped define our approach a
couple years ago. It's not clear what, if any changes we will make
for future terms, but I'm sure the following will be discussed:
- Exhorting/pressuring/rewarding faculty to dedicate class time and/
or petition their students for responses. This is the primary tactic
we've taken (the exhorting piece) in the past since our research
shows that faculty requests to their classes have significant
positive effect on response rates.
- Extending the response windows from one week to 10 or more days
- Increasing the frequency of reminder emails to non-responders (we
currently send one reminder every other day if a student has one or
more unanswered evals)
- Making the evaluation response a class "assignment" that carries
"extra credit" value (we have evidence that this boost rates
significantly since students always go for the points, but it raises
lots of difficult conflict questions and faculty for the most part
are opposed on philosophical grounds).
- Withholding/delaying grades until evaluations are submitted
- Using a raffle or other random reward delivery to motivate
students ("free iPod for 5 lucky responders!")
- Giving students who respond to all evals some sort of preferential
treatment administratively (e.g. early registration)
Other schools are grappling with the response rate issue and there
was a fair amount of related discussion at the conference. Going
forward, I'm happy to report out on future EVALSYS conference calls
what U-M is going to do differently (if anything).
SMD.
On Jul 17, 2009, at 3:48 PM, Faust Gorham wrote:
> Hey Josh,
>
> Who I currently the school with the most experience in the
> evaluation space? I would like several people here at Merced to
> discuss the pedagogy and changes in submission rates, scores, etc,
> that have occurred due to the switch from paper to online format?
>
> Regards,
> Faust
>
> Sent from my iPhone
>
> On Jul 17, 2009, at 12:29 PM, Josh Baron <Josh.Baron at marist.edu>
> wrote:
>
>>
>> We in the final stages of our pilot of the evaluation system with
>> plans to move into production in the Fall semester.
>>
>> We have changed the tool permissions in Realms as per the
>> instructions in Confluence and deployed to Evaluation Tool to the
>> Administrator Workspace as well as user's My Workspaces. Things
>> are working well but what we're finding is that when you go to
>> assign a course evaluation using an "admin" account to course
>> sites that only course sites in which that "admin" is an
>> instructor are showing up.
>>
>> This is presenting a problem as we would like to have one of our
>> system administrators assign course evaluations to 60-80 courses
>> per semester but many of these are course sites in which the the
>> system admin is not in as an instructor. We could manually add
>> this person to each course site that we need to evaluate but that
>> is manual and time consuming.
>>
>> Is there a way to set things up so that system administrators see
>> all courses when they go to assign an evaluation to a course site?
>>
>> Thanks, Josh
>>
>> -----------------------------
>> Joshua Baron
>> Director, Academic Technology and eLearning
>> Marist College
>> Poughkeepsie, New York 12601
>> (845) 575-3623 (work)
>> Twitter: JoshBaron
>> _______________________________________________
>> evaluation mailing list
>> evaluation at collab.sakaiproject.org
>> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>>
>> TO UNSUBSCRIBE: send email to evaluation-
>> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
> _______________________________________________
> evaluation mailing list
> evaluation at collab.sakaiproject.org
> http://collab.sakaiproject.org/mailman/listinfo/evaluation
>
> TO UNSUBSCRIBE: send email to evaluation-
> unsubscribe at collab.sakaiproject.org with a subject of "unsubscribe"
SMD.
==========================================================
Sean DeMonner, IT Senior Project Manager, CTools Implementation Group
Digital Media Commons @ The Duderstadt Center, U-M (734) 615-9765
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/evaluation/attachments/20090717/9f72501a/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pastedGraphic.tiff
Type: image/tiff
Size: 162082 bytes
Desc: not available
Url : http://collab.sakaiproject.org/pipermail/evaluation/attachments/20090717/9f72501a/attachment-0001.tiff
More information about the evaluation
mailing list