What is Cognitive Task Analysis?

Cognitive Task Analysis for Digging up Mental Gems Cognitive Task Analysis helps you unpack the thought processes of experts, so you can teach them to others.

Have you ever needed to train novices to perform well on a real-world task or job that’s complex and poorly understood?

Some of your experienced people do it well. But you don’t have a clear understanding of how they do it.

You can observe the experts’ behavior. But that only gets you so far, especially when the tasks are complex. To really understand, you need to know what’s going on inside their heads.

You need to figure out what they know and how they think. You need access to their interpretations, their goals, the ways they frame problems and decisions, as well as the thought processes they employ to work through them.

Cognitive Task Analysis (CTA) is a family of psychological research methods for uncovering and representing what people know and how they think. CTA extends traditional task analysis to tap into the mental processes that underlie observable behavior, and reveal the cognitive skills and strategies needed to effectively tackle challenging situations.

At Global Cognition, we primarily apply cognitive task analysis to inform instruction focused on higher order thinking skills. However, CTA has also been used to design human-computer interfaces and other technological systems. There are numerous ways that cognitive task analysis can help boost performance in complex work settings. For example, CTA has provided significant input into our cultural competence modeling efforts.

What does a cognitive task analysis look like?

At this point in time, it really depends on the researchers conducting the analysis. As I mentioned above, CTA represents a family of methods. And it’s not a small family. In an overview chapter (pdf) on cognitive task analysis, Richard Clark of the University of Southern California and his colleagues noted that there are currently over 100 types of CTA methods in use.

Regardless of the specific technique, CTA typically consists of several broad phases that the family holds in common:

Background preparation – getting familiar with the domain and population of interest. Reading through any existing manuals, doctrine, and holding informal discussions are common ways to start getting up to speed in the problem area.

Elicitation of knowledge – using one or more specific techniques to draw out the tacit knowledge and thought processes of experts. More on this below.

Analysis of qualitative data – sifting through the mass of data, usually in the form of transcripts of the experts’ verbal reports. Identifying decisions, cues, goals, strategies, concepts, and other elements of thought.

Knowledge representation – assembling those thought elements into a readily digestible format for understanding and communication. Usually, this means creating tables, charts, or diagrams that clearly represent the experts’ knowledge.

Design & develop applications – creating instruction, decision aids, or other applications using the newly constructed model of the experts’ knowledge as a starting point for ideation and design.

Knowledge Elicitation

To go a bit deeper into CTA methods, let’s take a look at one specific approach for eliciting knowledge, the critical decision method. First described by Gary Klein and colleagues at Klein Associates, this is an interview technique for eliciting critical incidents to unpack the intuitive decision making of experts.

We’ll focus on this knowledge elicitation method because it distinguishes cognitive task analysis from other psychological research methods. Unlike the focus of many self-report survey methods, no one asks you to rate your opinions in a critical incident interview. And, this approach doesn’t lead you to assess your own personality, skills, or competence.

Instead, the idea of the critical decision method is to get experienced professionals to describe some of the toughest challenges they faced. By using carefully crafted probes, the CTA interviewer teases out how these people assessed situations and made decisions in critical moments of their experience.

So, how does the critical decision method work?

First, the CTA interviewer guides the subject-matter expert to identify a relevant incident. Next, the participant tells their whole story, without interruption. They are asked to recount the events in their entirety.

Then, the interviewer combs back over key points in the story several times to get the subject-matter expert to elaborate them. As they visit and revisit narrow slices of the experience, the interviewer and participant establish the critical decisions that were made, where understanding changed, and other turning points during the episode. The interviewer asks targeted questions to uncover the factors and cues noticed, goals adopted, and strategies used to resolve the incident.

Once the original story is fully fleshed out, the interviewer might ask hypothetical, “what if” questions to get a sense of anticipated outcomes that were never realized and trade-offs that were made.

Unlike the last monotonous phone survey you took, critical incident interviewers are not asking questions in lock-step fashion. Instead, they keep some flexibility to probe on aspects of a lived experience that make the most sense in the moment.

This semi-structured approach makes the critical decision method a fairly difficult interview to conduct. It takes some study and practice to learn how to execute it well. It’s also common for two interviewers to conduct the session together.

In the end, the interview produces a rich description of a critical incident and the cognition used to tackle it. In addition to observable details about the situation and behaviors, the cognitive task analysis interviewer dredges up information about the tacit knowledge, goal structures, and judgment and decision processes underlying the readily observable actions taken.

Cognitive Task Analysis for Instructional Design

These critical incidents, along with the decision points, cues, and strategies provide experience-based information that can be taught to novices.

One CTA, for example, uncovered the initial cues that experienced paramedics used to identify heart attack victims before they were fully symptomatic. These cues included changes in skin tone, the temperature of their skin, the state of their breathing and the patient’s mental state. Once they were identified, instructional materials were developed to teach new paramedics to recognize these subtle cues.

In addition to informing learning objectives, critical incidents provide a solid basis for creating informed, realistic scenario-based exercises. Along with providing a highly engaging instructional method, such exercises can expose learners to a variety of the tough situations they may encounter, and clue them in to the difficult decisions they’ll have to make. Further, they enable learners to practice the higher order thinking skills they’ll need to successfully work through challenging problems.

In all, using an expert’s knowledge, skills, procedures, and methods, locked inside incidents they have experienced, to design training materials can yield better decision-making processes for everyone.