[Building Sakai] T&Q capacity to avoid duplicate target items from pool

Tue Apr 10 23:13:55 PDT 2012

Hi,

I don't know if this https://jira.sakaiproject.org/browse/SAM-1212 is related with what you're discussing. We are about to finish the fix but it would be great if anyone could help us with testing.

Thanks a lot!!

David
________________________________________
De: sakai-dev-bounces at collab.sakaiproject.org [sakai-dev-bounces at collab.sakaiproject.org] En nombre de Paul Dagnall [pdagnall1 at udayton.edu]
Enviado el: miércoles, 11 de abril de 2012 3:03
Para: Keli Sato Amann
CC: sakai-dev at collab.sakaiproject.org
Asunto: Re: [Building Sakai] T&Q capacity to avoid duplicate target items from pool

Our users have done things similar to this. It basically was accomplished by creating large amounts of sub-pools without any redundancy and each "question" was actually a random draw "part" that selects a single question from the sub-pool.

This isn't exactly easy on the user but it's an uncommon use case.

Paul Dagnall
Application Developer & Administrator
University of Dayton

On Tue, Apr 10, 2012 at 6:39 PM, Keli Sato Amann <kamann at stanford.edu<mailto:kamann at stanford.edu>> wrote:
Hi David,
We want to make sure we understand your question, so let me see if I can rephrase it so that the developers can answer:

1) As a very simple example, let's say you want to make sure someone understands the words for 8 colors. Each color has 3 possible questions, for instance
The grass is ___ (a color)
Frogs are stereotypically ___ (a color)
Emeralds are  ___ (a color)

Let's say instead of a thousand questions, you have exactly 24 questions in your pool: 3 each for each of 8 colors. If in an 8 question test you only wanted 1 question per color, you couldn't put them all in one pool. You would need to assign a different pool to each of the 8 colors, then assign one pool per question.

 But in real life, you have hundreds of words, and you only get to quiz people on 8 of those. So you want to know is it feasible to add a feature that allows Samigo to detect the right answer to a question or add the metatag "green" to the question so that when drawing questions, you never draw more than one question with the same answer?

2) As to your second question, I am not sure if you are asking whether you can have multiple correct answers for Fill in the Blanks (yes, you can) or if multicorrect would make the detection of repeat answers (detected directly or via meta tag) more difficult. I think you mean the latter. I suspect it's only slightly more difficult than addressing #1 above, but #1 might be very difficult.

Our developers don't have the capacity to develop this feature request for you, but they could probably advise you on your approach.

Keli Amann
User Experience Specialist
Academic Computing Services, Stanford University

---------- Forwarded message ----------
From: David Minugh < David.Minugh at english.su.se<mailto:David.Minugh at english.su.se> >
Date: Fri, Apr 6, 2012 at 5:01 AM
Subject: [Building Sakai] T&Q capacity to avoid duplicate target items from pool
To: " sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org> " < sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org> >

Apologies in advance for the length of this query, and uncertainty about whether it’s the developer or user forum this should be directed to, but it has to do with whether Sakai can handle pool questions of this type:

We are implementing a traditional-type English-only vocabulary test drawing on a database. The test consists of several sections, but for simplicity’s sake consider the section where each item is a sentence with a one-word slot where the student provides the answer. Each student gets a random selection of items (i.e., no two students get the same test). The items are drawn from frequency bands, with e.g. 8 items randomly selected per 1000-word band (with cognates, names and other less suitable items eliminated, the actual band pool is 200±100 items).

As we expand the pool, we would like to have multiple questions for each target word, and the problem then arises of whether Sakai’s T&Q can be configured to prevent Sakai from selecting several questions with the same answer. In a fully random sample of 8 items from a 200-word base, with 3 questions per word, the overlap for e.g. the 8 th item ought to be =14/600, if the first 7 contain no overlaps. This suggests that for a 400-person test, a dozen or so will get two questions on the same word, which doesn’t seem acceptable, and wouldn’t be in a normal written test. A cascading system with several thousand sub-pools seems less reasonable than a check that prevents several questions with identical answers, but does Sakai have such a capacity?

A secondary problem arises from the nature of the fill-in questions: we use Nation’s pattern of giving the students the first two letters of the word, plus a rough definition or synonym, to limit the options. This works fine for most words in English ( It was of no AV__ [ help ] would target avail , but not use ), but expecially for many words with prefixes such as in - or re -, there can be multiple acceptable answers ( It was her IN___ destiny [ unavoidable ] must accept at least inevitable|ineluctable|inescapable , and this too needs to be handled.

The same problem arises if we consider the multiple-choice section (except the program then needs to match the answer words, not the actual answer a, b, c, d, e).

The attractiveness of the individualized test is its asynchronic capacity: we don’t have the lab capacity to administer a lock-down test to 400 students at a time.

Has anyone solved this already? Or do we need to rethink our test format?

All the best,

David C. Minugh E-mail: David.Minugh at english.su.se<mailto:David.Minugh at english.su.se>

Director of Studies Tel: (+46) 8 16 36 11<tel:%28%2B46%29%208%2016%2036%2011>

English Department Cell phone: (+46) 70 - 23 14 777<tel:%28%2B46%29%2070%20-%2023%2014%20777>

Stockholm University Office: E 877, Frescati

_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"
_______________________________________________
sakai-dev mailing list
sakai-dev at collab.sakaiproject.org<mailto:sakai-dev at collab.sakaiproject.org>
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev

TO UNSUBSCRIBE: send email to sakai-dev-unsubscribe at collab.sakaiproject.org<mailto:sakai-dev-unsubscribe at collab.sakaiproject.org> with a subject of "unsubscribe"