I am interested in talking with anyone who is using Cognitive Task Analysis (CTA) or Naturalistic Decision Making (NDM) methods and paradigms to inform their customer interviews. I have been reading “Working Minds: A Practitioner’s Guide to Cognitive Task Analysis” by Gary Klein et. al. and I had an epiphany that these techniques would be directly applicable to a startup developing analytics or collaboration software. CTA/NDM techniques appear to be in active use for the analysis of military, health, fire and safety, and other problem domains where experts need to make life and death decisions in ambiguous situations against short deadlines.
Background on Gary Klein
I have been a fan of Gary Klein ever since I read a 1998 Science News article “Seeing Through Expert Eyes” that described his “naturalistic decision making” model:
…one of many examples of decision-making expertise collected over the past 20 years by psychologist Gary Klein. […] Klein is helping to develop a research perspective—known as naturalistic decision making, or NDM—for unraveling how people become bona fide experts in performing complex, real-life tasks.
“In many dynamic, uncertain, and fast-paced environments, there is no single right way to make decisions,” Klein says. “Experts learn to perceive things that are invisible to novices, such as the characteristics of a typical situation. They make high-quality decisions under extreme time pressure. When difficulties arise, experts find opportunities for improvising solutions.”
Klein has written several excellent books on decision making and expertise
- Sources of Power: How People Make Decisions (1998)
- The Power of Intuition: How to Use Your Gut Feelings to Make Better Decisions at Work (2002)
- Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making (2011)
In “Sources of Power” he outlines a “recognition primed decision making” model for how experts manage uncertainty against tight time and resource constraints–situations analogous to many challenges bootstrapping entrepreneurs face. I quoted him in my “Limits of I’ll Know It When I See It” talk on expertise driven companies.
Key Elements of Cognitive Task Analysis
Klein divides his time between MacroCognition and ARA, the latter firm has a page on cognitive task analysis at that breaks it into three phases (Working Minds goes into much more detail in 300+ pages):
- Knowledge Elicitation is the process of extracting information, through in- depth interviews and observations, about cognitive events, structures, or models. Often the people who provide this information are subject matter experts (SMEs) People who have demonstrated high levels of skill and knowledge in the domain of interest.
- Analysis is a process of structuring data inspecting, selecting, simplifying, abstracting, and transforming information, developing explanations, and extracting meaning. CTA practitioners use a range of quantitative and qualitative analyses in handling data.
- Knowledge Representation is the process of displaying data and depicting relationships, explanations, and the meaning derived from data analysis. This step is integral for enabling other people sponsors, customers, system designers who understand the results of the CTA.
Applying Cognitive Task Analysis to Market Exploration
Working Minds highlights “Concept Maps” (see for example http://cmap.ihmc.us/publications/researchpapers/theorycmaps/theoryunderlyingconceptmaps.htm ) as a powerful way to summarize interviews with experts and allow multiple experts to compare notes. I would welcome any insights on using either Cognitive Task Analysis or Concept Maps as of part of the customer interview process. My current approach also involves two people taking notes but using a wiki for hyperlinking connections instead of a graphical representation and collecting verbatim quotes instead of trying to code interview answers.
For More On Gary Klein’s Insights
- Associating, Pattern Matching, and Sensemaking
- Video and Slides From “The Limits of I’ll Know It When I See It”
- Articles, Ideas, and Books that have Changed My Life as an Entrepreneur
- “The PreMortem – anticipating a plan’s weaknesses” by Shawn Callahan explains Klein’s premortem technique very nicely
Pingback: SKMurphy, Inc. Scientific Method Entrepreneurship for Market Exploration