By Kevin Roberts

When I was young, I found speaking in public to be, well, petrifying. That is not much of an exaggeration, for I would literally freeze up at times when called upon to speak. It took a significant amount of effort and experimentation throughout my teens and 20s to overcome this propensity. One small insight I found was to focus on a single individual rather than the group.

Some people respond to certification testing the way I responded to public speaking in my youth. They may feel something between simple foreboding and outright dread. However, just as groups of people are made up of individuals, tests are made of individual questions. By focusing on the individual parts, we may lose some of our fear of the whole test.

Plus, if you know how the questions are written, you may better understand how to answer them.

This article looks at the construction of a test question. Subsequent articles will look at how to avoid overthinking the question.

By way of introduction, a test question has three components:

• The question.
• The correct answer.
• Three incorrect answers.

First: The Question

A question is a string of words that should be structured so that the thought of the author is reproduced as a thought in the reader.

To achieve this: It must be unambiguous (no words with more than one meaning). It must follow grammar (it uses the proper words) and syntax (the words appear in the correct order). It must avoid regional idiosyncrasies. And, of course, it must be accurate (it must conform to reality).

This is true of any written communication. Simple, right? If you have ever read a well-written question, the key to its understandability is found in its simplicity. The challenge is that producing a simple question is anything but simple. The more words that are used, the more likely a test taker will overthink the question and end up in a whirlwind of what ifs.

To quote a famous French mathematician, Blaise Pascal, “I would have written a shorter letter, but I did not have the time.”

Or to quote an American novelist, Wallace Stegner, “Hard writing makes easy reading.”

So how do you do it?

You organize a committee.

This is the first opportunity to see if one committee member can create a question that reproduces his idea in the minds of the other committee members. If any other committee member asks, “What?”, you have not created a good question. And this is only the first consideration.

As the process proceeds, edits are suggested to eliminate the “What?” response. This is hard work because once the creator of the question has explained himself, the other committee members can’t “unknow” his explanation. They must try to continually put themselves in that “unknowing” position of the Test Taker who sees the question on the test for the first time. He doesn’t receive that explanation that was shared with the committee.

Next, the clearest, simplest, most elegant wording will not make a good question if, when complete, it does not coincide with reality. This may be due to regionalism, misinformation, or single manufacturer idiosyncrasies. This is another reason why a diverse committee is essential. If every member of the committee is from the same neighborhood and works on the same brand rig this may detract from the test’s effectiveness. 

Why multiple-choice questions?

To facilitate the scoring of a test, we are stuck with multiple choice questions. It is too much to expect a test taker to produce an essay answer and too much for the certification organization to provide the manpower to evaluate an essay answer. 

Why Technician A and Technician B questions?

Then there is the Technician A and Technician B type of question. Most technicians that I have spoken to really dislike these questions. But we use them for a reason. As we stated above, the test is a filter. By using the format of Technician A and Technician B, we make that filter finer. It requires the Test Taker to know two items for each question rather than one. Does that make the test more difficult? Of course. But the goal of the Certification Commission is neither to make the test easy nor difficult. It is to make the test effective. By doing so we accomplish several things at once. We enhance the credibility of the test and therefore the credibility of the test taker that passes the test. Plus, we enhance the credibility of the certifying body. Finally, we enhance the credibility of the industry.  

Second, the Correct Answer

This may sound obvious, but there can be controversies in the committee room over this. Further, once a correct answer is agreed upon, setting up the wording (grammar and syntax) can be a challenge. The purpose of the wording, as with the question, is to prevent the Test Taker from struggling with what the answer means. He should rapidly understand that, and he should quickly understand whether or not he knows it to be the correct answer.

Third, the Incorrect Answers

This can actually be one of the biggest challenges of writing a test question. The goal is to have three answers that are plausible to the uninformed Test Taker, but not to the informed Test Taker. If the question asks, “What is the measure of power in an electric circuit?” Giving four answers as follows, A) Watt, B) An exclamation point, C) North Dakota, or D) A banana, will not make the question valuable. The three incorrect answers are not plausible. This is why proper incorrect answers are often called “distractors”.

If the options are A) Watt, B) Volt, C) Amp, or D) Ohm, we now have proper distractors. I often use this question in class to gauge how informed the students are.

The responsibility of writing an easily understandable question with one correct answer and three incorrect but somewhat plausible answers is not for the faint of heart, nor for the ego driven committee member. Do you think you could contribute meaningfully to the process? Continue to hone your technical skills and your language skills, achieve Master Level EVTCC status and join us for our Validation Conference.

Next time: Preparation to take a test.