MCQs are a popular choice in some disciplines as a way to deliver Core assessments or deliver formative assessments, as they swiftly check understanding without an undue burden on marking, as they can be automatically marked. The introduction of Inspera as the University Group’s digital assessment platform offers more diversity in automatically marked assessments including gap filling, true/false, multiple response, calculations, and matching/pairing questions as well as traditional MCQs.
These question types can be used both to check understanding and to feedback on correct and incorrect answers but do rely on having well written questions that comply with some simple conventions. Below are 6 guidelines for developing good automatically marked questions along with a link to more in-depth discussions and exemplars.
- Good questions (or stems) need to be constructively aligned, relating to module content and the learning outcomes.
- Good questions are clear in what they are asking for and do not contain extraneous information.
- Give clear instructions so that students understand the question types and what they are required to do. You may wish to create a short preview quiz for them.
- Alternative answers (or distractors) should be plausible (4 or 5, including the answer, is generally OK). Distractors should be constructed in the same manner and be the same length as the correct answer.
- Negative phrasing or double negatives are difficult for some students to rationalise, and care needs to be taken to ensure what is being asked for is clear. This will make your questions more accessible for students with dyslexia. If you must use negative language, make the negative words bold.
- Mix up the order of the correct answers, people often subconsciously put them either 2nd or 3rd in the list. If you are using Inspera you can randomise the order that potential answers appear to candidates.
Inspera allows different options for answers. Answer options can be text, images, formulas, or even audio clips. This allows you to construct more diverse and complex automatically marked questions.
For more detailed information on constructing good multiple choice questions see this advice from Vanderbilt University The advice on question stems and writing effective alternatives also applies to true/false questions, gap-fill or gap-match questions, and other matching/pairing questions.
Explaining question set-ups to candidates
Bothell (2001) suggested distinguishing and explaining the question types to students in the introduction, for example.
“Questions 1 – 10 are multiple-choice questions designed to assess your ability to remember or recall basic and foundational pieces of knowledge related to this course. Please read each question carefully before reading the answer options. When you have a clear idea of the question, find your answer and mark your selection.
Questions 11 – 20 are multiple-choice questions designed to assess your ability to think critically about the subject. Please read each question carefully before reading the answer options. Be aware that some questions may seem to have more than one right answer, but you are to look for the one that makes the most sense and is the most correct. When you have a clear idea of the question, find your answer and mark your selection.”
When creating your Question Sets Inspera allows you to use Sections to group questions by topic or type and add in sign-posting information as described above. See guidance on creating Questions and Question Sets in Inspera
Creating higher level learning outcomes
Automatically marked questions are often considered to be only suitable for testing understanding at a basic level, this is not the case. The range of automatically marked question types in Inspera provides greater opportunities for creating questions that test higher order learning. These questions at a higher level do require discriminating judgement and can evaluate a situation or a problem using Blooms Taxonomy. For example, calculation or numerical simulation questions can test a candidate’s ability to apply mathematical theories to a new problem.
To test higher level thinking, questions should have multiple concepts to synthesise and relate to the situation and problem and answers should be based around existing protocols/frameworks/practice; there should be an established best, agreed or most important answer.
Bothell summarised this as “The key to preparing memory-plus application questions is to place the concept in a life situation or context that requires the student to first recall the facts and then apply or transfer the application of those facts into a situation.” (Bothell 2001)
Inspera allows you to add extra information and context as a ‘stimulus’ for a question or question set. This can be a useful way of introducing background information for more complex automatically marked questions.
For more detailed information on creating higher level thinking MCQs see these suggestions from the Learning Guild
Further reading
Bone, E and Prosser, M: Multiple Choice Questions: an Introductory Guide Really useful guide from the University of Melbourne
Morrison, S and Walsh Free, K Writing higher level thinking quizzes
Piontek, M, E Best practices for designing and grading exams Guide from the University of Michigan with advice and a checklist for constructing multiple choice exams.
Inspera guide to automatically marked questions
Example Question Types in Inspera- demo test
Creating questions and question sets in Inspera
References
Bothell, T., W. (2001). 14 rules for writing multiple-choice questions. Brigham Young University.https://pdf4pro.com/view/handouts-14-rules-for-writing-multiple-choice-questions-5a304e.html
