It’s good to regularly review the advantages and disadvantages of the most commonly used test questions and the test banks that now frequently provide them.
- Quick and easy to score, by hand or electronically
- Can be written so that they test a wide range of higher-order thinking skills
- Can cover lots of content areas on a single exam and still be answered in a class period
- Often test literacy skills: “if the student reads the question carefully, the answer is easy to recognize even if the student knows little about the subject” (p. 194)
- Provide unprepared students the opportunity to guess, and with guesses that are right, they get credit for things they don’t know
- Expose students to misinformation that can influence subsequent thinking about the content
- Take time and skill to construct (especially good questions)
- Considered to be “one of the most unreliable forms of assessment” (p. 195)
- Often written so that most of the statement is true save one small, often trivial bit of information that then makes the whole statement untrue
- Encourage guessing, and reward for correct guesses
- Quick and easy to grade
- Quick and easy to write
- Encourage students to memorize terms and details, so that their understanding of the content remains superficial
- Offer students an opportunity to demonstrate knowledge, skills, and abilities in a variety of ways
- Can be used to develop student writing skills, particularly the ability to formulate arguments supported with reasoning and evidence
- Require extensive time to grade
- Encourage use of subjective criteria when assessing answers
- If used in class, necessitate quick composition without time for planning or revision, which can result in poor-quality writing
Questions provided by test banks
- Save instructors the time and energy involved in writing test questions
- Use the terms and methods that are used in the book
- Rarely involve analysis, synthesis, application, or evaluation (cross-discipline research documents that approximately 85 percent of the questions in test banks test recall)
- Limit the scope of the exam to text content; if used extensively, may lead students to conclude that the material covered in class is unimportant and irrelevant
We tend to think that these are the only test question options, but there are some interesting variations. The article that promoted this review proposes one: Start with a question, and revise it until it can be answered with one word or a short phrase. Do not list any answer options for that single question, but attach to the exam an alphabetized list of answers. Students select answers from that list. Some of the answers provided may be used more than once, some may not be used, and there are more answers listed than questions. It’s a ratcheted-up version of matching. The approach makes the test more challenging and decreases the chance of getting an answer correct by guessing.
Remember, students do need to be introduced to any new or altered question format before they encounter it on an exam.
Editor’s note: The list of advantages and disadvantages comes in part from the article referenced here. It also cites research evidence relevant to some of these advantages and disadvantages.
Reference: McAllister, D., and Guidice, R.M. (2012). This is only a test: A machine-graded improvement to the multiple-choice and true-false examination. Teaching in Higher Education, 17 (2), 193-207.
Reprinted from The Teaching Professor, 28.3 (2014): 8. © Magna Publications. All rights reserved.
Tagged with assessing student learning, designing test questions, grading strategies, multiple-choice tests, test questions
Charles Champlin (2006), a journalist for Time and Life magazines, describes his experience of taking essay tests as a student at Harvard:
“The worst were the essay questions (which seemed only distantly related to whatever you’d read or heard in lectures). They made a statement and then simply said, ‘Discuss.’ O terrifying word, ‘Discuss.’ Nothing so simple as tossing in a few facts retained from all-night cramming. It was meaning that was sought – which was, as I’d already begun to appreciate, the way it should be. But it was a strained step up from the exams I’d known before, when memory, regurgitated, would get you around almost any corner.”
Champlin’s reminiscence reveals some of the strengths and dangers associated with essay questions. They are a wonderful way to test higher-level learning, but they require careful construction to maximize their assessment effectiveness.
I. Strengths Associated with Essay Examinations
Among the strengths of essay examinations, faculty who use them find they are a valuable means to measure higher-order learning and a wonderful way, when scored properly, to further student learning. Given these strengths, essay tests require careful preparation and scoring.
1. Essay Questions Test Higher-Level Learning Objectives
Unlike objective test items that are ideally suited for testing students’ broad knowledge of course content in a relatively short amount of time, essay questions are best suited for testing higher-level learning. By nature, they require longer time for students to think, organize and compose their answers.
In the table below, appropriate testing strategies are associated with Bloom’s hierarchy of learning. The action verbs under each domain illustrate the kinds of activities that a test item might assess. Use the verbs when constructing your essay questions so that students know what you expect as they write. While essay questions can assess all the cognitive domains, most educators suggest that due to the time required to answer them, essay questions should not be used if the same material can be assessed through a multiple-choice or objective item. Reserve your use of essay questions for testing higher-level learning that requires students to synthesize or evaluate information.
2. Essay Questions When Scored Properly Can Further Learning
Teachers score essay exams by either the holistic approach or the analytic approach.
The holistic approach involves the teacher reading all the responses to a given essay question and assigning a grade based on the overall quality of the response. Some teachers use a holistic approach by ranking students’ answers into groups of best answers, average answers and poor answers and subdividing the groups to assign grades.
Holistic scoring works best for essay questions that are open-ended and can produce a variety of acceptable answers.
Analytic scoring involves reading the essays for the essential parts of an ideal answer. In this case, you will need to make a list of the major elements that students should include in an answer. You will grade the essays based on how well students’ answers match the components of the model answer.
Whichever method, holistic or analytic, that you use to score the exam, you should write comments on the students’ papers to enhance their learning. Your comments will help students write better essays for future classes and reinforce what students know and need to learn. Your comments are also a good reminder for yourself if students come to you with questions about their grades.
II. Dangers to Consider When Giving and Grading Essay Examinations
1. Establish limits within the essay question
The example of Charles Champlin’s experience at Harvard where his teachers gave a statement and then simply said, ‘Discuss,’ shows a danger in using essay questions. Instructors should build limits into questions in order to save needless writing due to vague questions: “With some essay questions, students can feel like they have an infinite supply of lead to write a response on an indefinite number of pages about whatever they feel happy to write about. This can happen when the essay question is vague or open to numerous interpretations. Remember that effective essay questions provide students with an indication of the types of thinking and content to use in responding to the essay question” (Reiner, 2002).
Another good way to prevent students from spending excessive time on essays is to give them testing instructions on how long they should spend on test items. McKeachie (2002) gives the following advice: “As a rule of thumb I allow about 1 minute per item for multiple-choice or fill-in-the-blank items, 2 minutes per short-answer question requiring more than a sentence answer, 10 to 15 minutes for a limited essay question, and half-hour to an hour for a broader question requiring more than a page or two to answer.”
2. Remember that essays require more time to score
While essay exams are quicker to prepare than multiple-choice exams, essay exams take much longer to score. You should plan sufficient time for scoring the essays to prevent finding yourself crunched to report final grades.
3. Avoid scoring prejudices
Essay exams are subject to scoring prejudices. Reading all of an individual’s essays at the same time can cause either a positive or a negative bias on the part of the reader. If a student’s first essay is strong, the examiner might read the student’s remaining essays with a predisposition that they are also going to be strong. The reverse is also true. To prevent this scoring prejudice, educators suggest reading all the answers to a single essay question at one time.
Champlin, C. (2006). A life in writing: the story of an American journalist. Syracuse: Syracuse University.
McKeachie, W. (2002). McKeachie’s teaching tips (11th. ed.) New York: Houghton Mifflin.
Reiner, C., Bothell, T., Sudweeks, R., & Wood, B. (2002). Preparing effective essay questions. (http://testing.byu.edu/info/handbooks/WritingEffectiveEssayQuestions.pdf).