Chapter Appendix B
Developing a Test Item Bank
A multiple choice item stem may take one of several basic forms:
- A direct question followed by several possible answers.
- An incomplete sentence followed by several possible phrases that complete the sentence.
- A stated problem based on an accompanying graph, diagram, or other artwork followed by the correct response and the distractors.
The learner may be asked to select the one correct choice or completion, the one choice that is an incorrect answer or completion, or the one choice that is the best answer option presented in the test item.
Beginning test writers find it easier to write items in the question form. In general, the form with the options as answers to a question is preferable to the form that uses an incomplete statement as the stem. It is more easily phrased and is more natural for the learner to read. Less likely to contain ambiguities, it usually results in more similarity between the options and gives fewer clues to the correct response.
When multiple choice questions are used, three or four alternatives are generally provided. It is usually difficult to construct more than four convincing responses; that is, responses which appear to be correct to a person who has not mastered the subject matter. Learners are not supposed to guess the correct option; they should select an alternative only if they know it is correct. An effective means of diverting the learner from the correct response is to use common learner errors as distractors. For example, if writing a question on the conversion of degrees Celsius to degrees Fahrenheit, providing alternatives derived by using incorrect formulas would be logical, since using the wrong formula is a common learner error.
Items intended to measure the rote level of learning should have only one correct alternative; all other alternatives should be clearly incorrect. When items are to measure achievement at a higher level of learning, some or all of the alternatives should be acceptable responses—but one should be clearly better than the others. In either case, the instructions given should direct the learner to select the best alternative.
To use multiple choice questions, consider the following guidelines for construction of effective test items:
- Make each item independent of every other item in the test. Do not permit one question to reveal, or depend on, the correct answer to another question.
- Design questions that call for essential knowledge rather than for abstract background knowledge or unimportant facts.
- State each question in language appropriate to the learners.
- Include sketches, diagrams, or pictures when they can present a situation more vividly than words. They generally speed the testing process, add interest, and help to avoid reading difficulties and technical language.
- When a negative is used, emphasize the negative word or phrase by underlining, bold facing, italicizing, or printing in a different color.
- Avoid questions containing double negatives, which invariably cause confusion.
- Avoid trick questions, unimportant details, ambiguities, and leading questions that confuse and antagonize the learner. If attention to detail is an objective, detailed construction of alternatives is preferable to trick questions.
Stems
When developing the stem of a multiple choice item, the following general principles should be utilized. [Figure B-3]
- The stem should clearly present the central problem or idea. The function of the stem is to set the stage for the alternatives that follow.
- The stem should contain only material relevant to its solution, unless the selection of what is relevant is part of the problem.
- The stem should be worded in such a way that it does not give away the correct response. Avoid the use of determiners, such as clue words or phrases.
- Put everything that pertains to all alternatives in the stem of the item. This helps to avoid repetitious alternatives and saves time.
- Generally avoid using “a” or “an” at the end of the stem. They may give away the correct choice. Every alternative should grammatically fit with the stem of the item.
Alternatives
The alternatives in a multiple choice test item are as important as the stem. They should be formulated with care; simply being incorrect should not be the only criterion for the distracting alternatives.
Popular distractors are:
- An incorrect response related to the situation and which sounds convincing.
- A common misconception.
- A statement which is true, but which does not satisfy the requirements of the problem.
- A statement that is either too broad or too narrow for the requirements of the problem.
Research of instructor-made tests reveals that, in general, correct alternatives are longer than incorrect ones. When alternatives are numbers, they should generally be listed in ascending or descending order of magnitude or length.
Matching
A matching test item consists of two lists, which may include a combination of words, terms, illustrations, phrases, or sentences. The learner matches alternatives in one list with related alternatives in a second list.
In reality, a matching exercise is a collection of related multiple choice items. In a given period of time, more samples of a learner’s knowledge usually can be measured with matching rather than multiple choice items. The matching item is particularly good for measuring a learner’s ability to recognize relationships and to make associations between terms, parts, words, phrases, clauses, or symbols listed in one column with related items in another column. Matching reduces the probability of guessing correct responses, especially if alternatives may be used more than once. The testing time can also be used more efficiently.