Is it worth it to have your husband try his luck as a handyman first, or might you just go ahead and call a service professional instead? Is it safe to let your (impatient) wife get her hands on the lawnmower or would it be better to call those landscaping guys your neighbor uses? But then, how can you tell if the so called experts even know what they’re doing?
It may not be necessary to have others take a written test to check for understanding. And more importantly, you don’t necessarily have to be an expert yourself in order to tell. Paying attention to others when they talk can give you important clues about how much they know.
So, what do you listen for to check for understanding?
Art Graesser and Brent Olde at the University of Memphis have discovered a way to tell whether a person really understands something: do they ask good questions?
In clever study, the researchers had more than a hundred college students think out loud as they attempted to troubleshoot six common household devices and appliances: a cylinder lock, an electronic bell, a car temperature gauge, a clutch, a toaster, and yes, a dishwasher. The published their paper, How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down, in the Journal of Educational Psychology.
Graesser and Olde found that the questions the students asked as they thought about how the device works could be used to check for understanding. The quality of the questions the students asked to diagnose each broken device indicated how well they understood it.
In the study, Graesser and Olde showed the students illustrations of the parts of each device. Also, for each device they provided a short description of situation in which it had broken down. For example, for the lock it might say “the key turns but the bolt does not move.”
The students were then asked to write their thoughts as they tried to figure out what might be wrong and looked for a way to repair it. They did this two ways:
- For half of the devices they were asked to just write whatever came to mind. Here students generated mostly ideas about what might be wrong.
- For the remaining devices they were instead asked to write down as many questions as they could think of.
After the students had seen all the devices, the researchers gave them a test to see how well they really understood the workings of each. This was their gold standard to check for understanding.
The researchers found that there was no difference between the freeform ideas the students generated about the breakdowns. But, there was a difference between the kinds of questions that were asked by the students who scored well on the comprehension test and those who scored poorly.
The high scorers asked questions that focused on the possible causes of the breakdown. For example, “Is the bar that fits under the cam broken?”
The lower scoring students asked questions that were less likely to identify faults—and were more likely to be shallow or irrelevant, for example, “What kind of lock is it?”).
In general the students who had a greater depth of technical knowledge about the devices tended to ask questions that started with “why,” “why not,” “how,” “what if,” and “what if not”. Graesser and Olde saw this as a sign that these students were thinking deeply about how the components of the devices work together.
The finding that questions were a better check for understanding than freeform ideas about what might be wrong suggests that asking someone else to just tell you what they think the problem is might not be the best way to go.
Instead, a better strategy would be to prompt them to ask questions. You might say “what are the questions I should be asking myself when I see a problem like this?” Because questions reveal a person’s deep knowledge, prompting them this way could also help you learn more from what they know. If they really do know what they’re talking about, that is.
Image Credit: Editor B
Graesser, A., & Olde, B. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. Journal of Educational Psychology, 95 (3), 524-536 DOI: 10.1037/0022-06184.108.40.2064