[Contrib: Evaluation System] Requirements and Questions

[Aaron Zeckoski]

It seems like every institution has a different set of requirements. I
still hope some day that someone will pick it up and say "hey! this
happens to do everything we need!".

To clarify on some of the notes below, when I say there is no
interface, I mean the backend can handle it but there is no frontend
yet. There are a lot of things like this since I can adjust the
backend much faster than I can create UI.

On Thu, Apr 30, 2009 at 9:20 PM, Clay Fenlason
<clay.fenlason at et.gatech.edu> wrote:
> OK, thanks Aaron.  I've been sent a clearer list of requirements from
> the unit that handles these things, so I can lay out more of our
> current process and ask which portions you'd guess might be
> problematic:
>
> Survey Set–Up
>
> * Core questions – same for every survey.
ready either via a standard template or tags

> * Departmental optional questions – department can choose some extra
> questions from our bank of about 200 questions (i.e. they do not get
> to create their own) to add to the surveys for some courses – they can
> have up to three different sets of these.
partially there, this works using tags but this is currently a manual
(import or db driven) process, it would be best if there were
interfaces to allow assigning tags and creating the expert items
Not sure what the 3 different sets part means

> * Instructor optional questions – instructor can choose up to three
> extra questions from our question bank (i.e. they do not get to create
> their own) to add to each of his/her surveys.
partially there, we still need an interface to allow the instructors
to select them, currently the choosing has to be done via manual DB
intervention

> * Comments – there are currently comments after each question and at
> the end.  In any survey format, we will have some comment blocks.
depending on what you need exactly, there are a few comment options

> Survey Administration
>
> * Faculty can access a report that tells them the response rate for
> their surveys in real time.
done if they are owners, there is no interface yet for someone who
does not own the survey

> Reports
>
> A.  For Students
>
> * The core question data is released to students for all courses for
> which there are at least 5 responses and a response rate of at least
> 30%.  Currently, we release these courses to student government, and
> they then link to that data from their site.
sorta done, students can be allowed to see results, this is done via
number of responses only though

> * Faculty can also access this student data.
faculty can be allowed to access the results as well

> B.  For Faculty
>
> * Five days after grades are submitted faculty can access their survey
> reports.  We set this date in the system so it is flexible.
done, use the viewable date

> * Faculty get a full report for all questions (core, departmental,
> instructor) and all comments by course.
done, the results include all items and comments and can be limited to
a single course or to a mix of courses for an evaluation

> C.  For Administrators
>
> * Seven days into the subsequent semester, administrative reports are available.
we don't currently have any limitation on when admins can view results

> * A School Chair can see the core questions and departmental optional
> questions for any course that was taught in that unit.
part of the hierarchy feature request

> * A Dean can see core questions for any course taught in that College.
part of the hierarchy feature request

> * There are other specialized reports that allow for reports across
> either time (multiple terms, single instructor) or across individuals
> (multiple courses, single term), etc.
this has been traditionally done outside of the system by the
institutions using the tool in the past, that said, I would like to
see this kind of functionality at some point, it just has not been a
priority for anyone yet

Hope that helps
-AZ

-- 
Aaron Zeckoski (aaronz at vt.edu)
Senior Research Engineer - CARET - Cambridge University
[http://bugs.sakaiproject.org/confluence/display/~aaronz/]
Sakai Fellow - [http://aaronz-sakai.blogspot.com/]