Using Students’ Questions to Check for Understanding

You’d like a quick way to check for understanding without a formal test.

Something formative. Different. And maybe a little fun.

As it turns out, Art Graesser and Brent Olde at the University of Memphis have discovered a great way to tell whether a person really understands something:

Listen to the kinds of questions they ask when facing a “broken” (or faulty) example.

In a clever study, the researchers had more than a hundred students think out loud as they attempted to troubleshoot six common household devices and appliances: a cylinder lock, an electronic bell, a car temperature gauge, a clutch, a toaster, and a dishwasher.

The published their paper, “How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down,” in the Journal of Educational Psychology.

Graesser and Olde found that the questions the students asked as they thought about how the device works could be used to check for understanding. The quality of the questions the students asked to diagnose each broken device indicated how deeply they understood it.

In the study, Graesser and Olde showed the students illustrations of the parts of each device. Also, for each device they provided a short description of a situation in which it had broken down. For example, for the lock it might say “the key turns but the bolt does not move.”

The students were then asked to write their thoughts as they tried to figure out what might be wrong. They did this two ways:

1. For half of the devices they were asked to just write whatever came to mind. Here students generated mostly ideas about what might be wrong.
2. For the remaining devices they were instead asked to write down as many questions as they could think of.

After the students had seen all the devices, the researchers gave them a test to see how well they really understood the workings of each. This was their gold standard. A summative assessment to calibrate their simple “formative” check for understanding.

The researchers found that there was no difference between the freeform ideas the students generated about the breakdowns.

But, there was a difference between the kinds of questions that were asked by the students who scored well on the comprehension test and those who scored poorly.

The high scorers asked questions that focused on the possible causes of the breakdown. For example, “Is the bar that fits under the cam broken?”

The lower scoring students asked questions that were less likely to identify faults. And their questions were more likely to be shallow or irrelevant, for example, “What kind of lock is it?”.

In general the students who had a greater depth of technical knowledge about the devices tended to ask questions that started with:

• Why…
• Why not…
• How…
• What if…
• What if not…

Graesser and Olde saw this as a sign that these students were thinking deeply about how the components of the devices work together. Other research shows similar results, and teaching students to ask open-ended questions that uncover deep explanations like these is a key piece of our study skills curriculum.

The Graesser and Olde study suggests a quick way to check for understanding with your students.

First, prompt your students to ask a few questions about the lesson. These could be about the material in general. Or, you could have them “troubleshoot” a faulty example that you provide.

Then, listen to the quality of their questions.

Do they reveal some depth of thought about the principles you’ve been covering? Or, are they more surface oriented, with little real substance?

Image Credit: Editor B