Some uses for multiple choice tasks:
- Compulsory activity: students must achieve a certain score to be eligible to take the exam. Students can also create multiple choice tasks themselves as a learning activity.
- In teaching: students are tested on learning objectives along the way, to give an indication of how they are doing, and to engage and make the teaching more interactive.
- Final assessment/exam: as a final assessment, multiple choice exams are used primarily in combination with other assessment methods.
What can you test with multiple choice tasks?
Multiple choice tasks as an assessment method can cover a wide range and test students on large portions of the curriculum in a relatively short amount of time. There must be a clear connection between established learning objectives and the multiple choice tasks.
A tip is to create questions based on the cognitive domains you want to test. A commonly used model in this context is Bloom's taxonomy, which divides knowledge or cognitive skills into six hierarchical levels.
Knowledge based on remembering and understanding is located at the lower level of the taxonomy. This includes the ability to reproduce and summarize information. The ability to apply and analyze is placed in the middle layer of the knowledge hierarchy. Synthesis and evaluation can be categorized as high-level knowledge, while the knowledge to create original material ranks at the top of Bloom’s Taxonomy.
With multiple choice tasks, you can measure skills at multiple levels. You can measure the ability to memorize and recall the curriculum, as well as the ability to understand and apply knowledge. The ability to problem-solve can also be tested with multiple choice tasks. An example of this is presenting students with a text excerpt that contains a clear problem. The students must then answer questions about it.
At a higher cognitive level, multiple choice questions can test the ability to analyze an issue, compare and apply knowledge in new situations. Interactive question types, as well as images and videos, can contribute to creating a multiple choice test that is both challenging, has a high level of difficulty, and is educational. Moreover, providing students with constructive feedback will lay a solid foundation for good formative assessments that support learning.
Therefore, the tip is to formulate questions and construct multiple choice tasks in a way that allows you to test the skills you desire.
How to construct good multiple choice questions?
KU Leuven (2022) provides guidelines and advice for creating multiple choice questions in Formulating multiple choice questions - Education (kuleuven.be). The advice can be used as a checklist in the work with multiple choice tests, but it is by no means exhaustive. We present a selective selection of tips on how to construct good questions and answer alternatives
The question:
- Formulate a clear question that contains a precise problem with little room for misunderstanding. Students should not need the answer options to understand the question
- Answer the multiple choice question yourself (or test it on a colleague). This is to check that the above point is met. Make adjustments if you cannot answer the question yourself or if you hesitate at an answer option.
- Questions should be independent of each other. Each question should address one problem and not depend on the solutions to previous tasks.
Answer alternatives
- Formulate the correct answer before formulating the alternatives. The options that are not the correct answer are called distractors.
- You can find the best distractors in the mistakes students most often make in the subject.
- Answer alternatives should not overlap. Overlapping answers can imply that several answers are correct and create confusion among students.
- Use a logical order in presenting the answer options. This minimizes the likelihood of consistently placing the correct answer in one spot. Repeating patterns among answer alternatives should be avoided as it could indicate the right answer.
- The distractors must be plausible. Avoid answer alternatives that are obviously right or wrong. This reduces the chance of students guessing their way to the correct answer.
See the full guide here: Formulating multiple choice questions - Education (kuleuven.be)
Choice of platform
When creating multiple choice questions, there are several platforms to choose from. As a teacher, Inspera, Canvas, and Mentimeter are relevant platforms. The platform you should choose will depend on the goal of the multiple choice test.
Multiple choice test as an exam form: Inspera
When multiple choice questions are used as a form of assessment, the exam platform Inspera is used. It can be time-consuming because every question must be added manually. However, it is most time-consuming the first time you create a multiple choice exam, as the same exam can be used for the following years. For example, you can add 10 new questions and choose the "random selection" function.
If there are exam candidates with both nynorsk and bokmål as their primary language, the multiple-choice test must be made in both written languages. This is a requirement from The Language Council of Norway jf Forskrift om målform i eksamensoppgåver. Because this must be done manually, the requirement can be bypassed if the exam task is considered "particularly extensive."
UiO has created informative guidelines for importing and exporting tasks in Inspera and Canvas: Import and export tasks (in Norwegian).
Multiple-choice test as a mandatory/optional activity: Canvas
.jpg){kind=logo}
Canvas is more user-friendly and less time-consuming than Inspera and can be used to create voluntary multiple-choice tests for students to use for self-learning.
If you want to create a multiple-choice test, you can follow these steps:
- Go to the course room in Canvas.
- Find the "Quizzes" function in the left menu.
- Click "New Quizzes." This gives you the opportunity to create a new quiz.
- If you want a multiple-choice test, select "Multiple Choice."
- Click "Save and Publish" when the test is complete. This will make it available to students.
Cell (Center for Experiential Legal Learning) has developed a user manual for How to create tests in Canvas (in Norwegian). The user manual provides simple guidance on setting up, building, and publishing tests.
Multiple-choice task as an activation in teaching: Mentimeter
If you want to engage students with a more interactive teaching approach, Mentimeter is a good choice. The platform can be used in teaching sessions. Mentimeter has created an informative instructional video: Create an Interactive Mentimeter quiz
Video: Mentimeter
Dilemmas
The choice you make is up to you. However, it may be helpful to understand the arguments for and against before forming an opinion.
Total number of questions
The total number of questions depends on the type of questions and their complexity. A higher number of tasks can result in a wider range of grades compared to a smaller number of tasks. The time a student needs for each question will depend on the difficulty level of the task. SVEXFAC estimated that students needed 1 - 1.5 minutes per question. With one hour available, their total number of questions was therefore 50.
Number of answer options
We recommend having three answer options. More than two distractors can seem redundant and less plausible. However, some use more than three options to reduce the chance of guessing.
Scoring - with or without negative points?
There are divided opinions on the use of negative points in multiple-choice tests. It is argued that negative points prevent guessing from being rewarded. At the same time, others believe that this can result in blank answers. Some students do not take the risk of receiving negative points and choose to leave the answer blank. The decision to have multiple-choice tasks without negative points can be made to avoid the risk of students performing worse than their actual level suggests. A compromise is to give low negative points, e.g. -0.25 points where a correct answer gives 1 point.
Adapting for students
Multiple-choice exams can be challenging for students with reading difficulties and dyslexia. These are students who often take longer to read and understand questions and answer options. Norwegian educational institutions are required by law to have universal design. This means that assessment forms and teaching should be adapted to the abilities and needs of the students.
How to adapt for students needs:
- Short questions and few answer options per task. This is suitable because there is less text to read and understand.
- Extended exam time. The student must apply for special examination arrangements.
- Digital aids such as speech synthesis, where tasks and answer options are read aloud.
Literature and sources
This guide is knowledge-based: it is based on research literature, guides from other higher education institutions and on the experiences of lecturers and administration at UiO. Here you will find the sources on which this guide is based.
Academic papers etc.
Bloom, B. S. (1956). Taxonomy of educational objectives; The classification of educational goals, by a committee of college and university examiners. New York: McKay.
Haladyna, T. M. & Downing, Steven M. (1993). How many options is enough for a multiplechoice test item? (Validity Studies). Educational and Psychological Measurement, 53(4), 999-1009.
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education, 15(3), 309-34
Hecke, V, T. (2015). Initial correction versus negative marking in multiple choice examinations. Teaching Mathematics and its Applications: An International Journal of the IMA, 34(3), 171–178. https://doi.org/10.1093/teamat/hrv011
Holt, A. (2006) An analysis of negative marking in multiple-choice assessment. The School of Computing and Mathematical Sciences. The University of Waikato, Hamilton, New Zealand.
Lesage, E., Valcke, M. & Sabbe, E. (2013). Scoring methods for multiple choice assessment in higher education – Is it still a matter of number right scoring or negative marking?. Studies in Educational Evaluation, 39(3), 188-193.
Nwadinigwe, P, I. & Naibi, L. (2013). The Number of Options in a Multiple-Choice Test Item and the Psychometric Characteristics. Journal of Education and Practice, 28(4), 189-197.
O’Neill, D, L. (2021). Multiple choice-spørgsmål i undervisningen. Dansk Universitetspædagogisk Tidsskrift, 16(30). 70-83. file:///C:/Users/Kaja/Downloads/ONeill2021.pdf
Olsen, S, T. (2009). Flervalgsoppgaver og digitale læringsplattformer: En kvalitativ studie av forberedelse til den første delen av skriftlig eksamen i Biologi 2. [Masteroppgave, Norges teknisk-naturvitenskapelige universitet].
Sirnes, S. (2005). Flervalgsoppgaver - konstruksjon og analyse. Bergen: Fagbokforl.
Woodford, K. & Bancroft, P. (u.å.). Multiple choice questions not considered harmful, School of Software Engineering and Data Communications. Queensland University of Technology.
User manuals/Guides
CELL. (u.å.). 7. Brukermanual: Hvordan utforme tester i Canvas? Hentet 19.desember 2022 fra https://www.jus.uio.no/cell/ressurser/verktoy/brukermanual-krim1500.pdf
Horgen, S, A. (2017, 29.mai). Hvordan lage gode tester i Canvas [Video]. Youtube. https://www.youtube.com/watch?v=kYSULmSqMGU
Ku Leuven (2022). Formulating multiple choice questions. Hentet fra Formulating multiple choice questions - Education (kuleuven.be)
NTNU. (2014). Flervalgsoppgaver. Undervisningsveileder fra Norges teknisk naturvitenskaplige universitet.
Olaussen, J. E. (16.august 2018). REAL- undervisningsseminar. Hentet 19.desember 2022 fra https://www.mn.uio.no/kurt/aktuelt/arrangementer/presentasjoner/2018/inkluderende-undervisning-real-uio-plenum-aug2018.pdf
Universitetet i Oslo. For ansatte. Importere og eksportere oppgaver. Hentet 19.desember 2022 fra https://www.uio.no/for-ansatte/arbeidsstotte/sta/eksamen/digital-eksamen/veiledninger/inspera/importere-og-eksportere-oppgavesett.html#toc4
Rules/regulations
Dysleksi Norge. (2021). Rettigheter høyere utdanning. Dysleksinorge.no. https://dysleksinorge.no/rettigheter-hoyere-utdanning/
Universitetet i Oslo. Se hva du har fått innvilget. Hentet 16.desember 2022 fra https://www.uio.no/studier/eksamen/tilrettelegging/se-hva-du-har-fatt-innvilget/
Universitetet i Oslo. (u.å.). Eksamen. Uio.no. https://www.uio.no/studier/emner/sv/sv/SVEXFAC03/#exam
Conversations
In connection with this report, 3 interviews have been conducted with employees at UiO.
- Eivind Balsvik, course convenor for SVexfac, conversation on multiple choice exam SVEXFAC 22 September 2022.
- Eirik Finne, studies adviser at IFIKK, a conversation on the use of multiple choice questions in Inspera 15 November 2022.
- Francesca Jensenius (ISV), conversation on multiple choice exam in STV4021 30 November 2022.