[Using Sakai] T&Q capacity to avoid duplicate target items from Q pool

[David Minugh]

Apologies in advance for the length of this query and the cross-posting to sakai-dev., but it has to do with whether Sakai can handle pool questions of this type:

We are implementing a traditional-type English-only vocabulary test drawing on a database. The test consists of several sections, but for simplicity's sake consider the section where each item is a sentence with a one-word slot where the student provides the answer.  Each student gets a random selection of items (i.e., no two students get the same test). The items are drawn from frequency bands, with e.g. 8 items randomly selected per 1000-word band (with cognates, names and other less suitable items eliminated, the actual band pool is 200±100 items).

As we expand the pool, we would like to have multiple questions for each target word, and the problem then arises of whether Sakai's T&Q can be configured to prevent Sakai from selecting several questions with the same answer. In a fully random sample of 8 items from a 200-word base, with 3 questions per word, the overlap for e.g. the 8th item ought to be =14/600, if the first 7 contain no overlaps. This suggests that for a 400-person test, a dozen or so will get two questions on the same word, which doesn't seem acceptable, and wouldn't be in a normal written test. A cascading system with several thousand sub-pools seems less reasonable than a check that prevents several questions with identical answers, but does Sakai have such a capacity?

A secondary problem arises from the nature of the fill-in questions: we use Nation's pattern of giving the students the first two letters of the word, plus a rough definition or synonym, to limit the options. This works fine for most words in English (It was of no AV__ [help] would target avail, but not use), but expecially for many words with prefixes such as in- or re-, there can be multiple acceptable answers (It was her IN___ destiny [unavoidable] must accept at least inevitable|ineluctable|inescapable, and this too needs to be handled.

The same problem arises if we consider the multiple-choice section (except the program then needs to match the answer words, not the actual answer a, b, c, d, e).

The attractiveness of the individualized test is its asynchronic capacity: we don't have the lab capacity to administer a lock-down test to 400 students at a time.

Has anyone solved this already? Or do we need to rethink our test format?

All the best,


David C. Minugh                                                                  E-mail: David.Minugh at english.su.se
Director of Studies                                                               Tel: (+46) 8 16 36 11
English Department                                                                    Cell phone: (+46) 70 - 23 14 777
Stockholm University                                                            Office: E 877, Frescati

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/sakai-user/attachments/20120406/b007fe9e/attachment.html