The Phoenix Checklist for Framing a Problem and Its Solution

In “Thinkertoys,” Michael Michalko (@MichaelMichalko) describes “The Phoenix Checklist” as a rich set of questions developed by the CIA designed to help agents look at a challenge from many different angles.

The Phoenix Checklist for Framing a Problem and Its Solution

Michael Michalko introduces the “Phoenix Checklist” in Chapter 14 (entitled “Phoenix”) of his book ThinkerToys. It’s a book well worth reading for renewing your creative juices; it offers a rich set of techniques to pick and choose from for creative problem framing and solving. Here is his introduction to the Phoenix Checklist:

Phoenix is a checklist of questions developed by the Central Intelligence Agency to encourage agents to look at a challenge from many different angles. […] Use the Phoenix checklist as a base on which to build your own personal checklist of questions.

Blueprint

Write your challenge. Isolate the challenge you want to think about and commit yourself to an answer, if not the answer, by a certain date.

Ask the questions. Use the Phoenix checklist to dissect the challenge in as many different ways as you can.

Record your answers. Information requests, solution, and ideas for evaluation and analysis.

Michael Michalko in “ThinkerToys” Chapter 14 “Phoenix”

Framing the Problem

Following is excerpted from “ThinkerToys” by Michael Michalko. The first level bold bullets are his; the second level bullets are my related comments and observations.

Why is it necessary to solve the problem?
- What is the problem you are trying to solve?
- What deadlines for a solution does the situation impose on you?
- What can you do to mitigate or ameliorate the problem in the absence of a complete solution?
What benefits will you receive by solving the problem?
- What costs or other negative outcomes will you incur if you cannot solve the problem? Avoiding them is one benefit to consider.
- What will solving this problem enable?
What is the unknown?
- In addition to identifying the initial set of unknowns, keep track of new unknowns you add as your understanding of the problem grows.
What is it you don’t yet understand?
- Writing down what you need to determine can help work toward a solution.
- Treat these “known unknowns” as variables and assign them a range (e.g., 10th, 50th, 90th percentile) if your uncertainty will resolve into a number. If the unknown will resolve as true or false, estimate its probability of being true. This approach to known unknowns can help you bound your ignorance. Forcing yourself to provide ranges or probabilities enables you to update your state of information as you learn more.
- What makes this harder is your current understanding of critical factors may turn out to be wrong. You don’t realize your lack of knowledge because your incorrect understanding masks it.
- Understanding what you don’t understand is complicated by “unknown unknowns.” While you can treat known unknowns as variables in your model of the situation, unknown unknowns are, by definition, omitted. Reminding yourself periodically that “it’s more complicated than that” applies to any understanding can keep you alert to missing information.
What is the information you have?
- How certain are you of each piece of information? Could false or incomplete information be influencing your ability to frame or describe the problem accurately?
What isn’t the problem?
- There may be issues that appear related but are not, upon closer examination, part of the problem.
- There may be aspects of the situation that you can tolerate–setting these aside enables you to narrow the focus of your efforts to severe pain or critical need.
Is the information sufficient? Or is it insufficient? Or redundant? Or contradictory?
- Available information is rarely sufficient and is often contradictory. The critical question is, what can you do with the information you have?
- What is the expected value of perfect information? How does that compare with the cost of gathering it?
Should you draw a diagram of the problem? A figure?
- It never hurts to draw a diagram. The act of drawing a diagram seems to access a different portion of the brain and can unlock insights that a verbal description will fail to capture.
- Asking for a drawing also helps when you are trying to learn more about the problem from others: ask them to sketch their picture of the problem.
- Drawing a picture of the problem when you are explaining it to someone else will help clarify your thinking. Hand them the pen and ask them to mark it up.
Where are the boundaries of the problem?
- Problems will have financial, social, and knowledge implications.
- What boundaries does the problem impose on a possible solution?
- Self-imposed constraints on a solution that are not part of the problem is a common mistake. One source of creative solutions is to remove these self-imposed boundaries that were not valid constraints upon closer examination.
Can you separate the various parts of the problem? Can you write them down? What are the relationships of the parts of the problem? What are the constants of the problem?
- Decomposition or ‘divide and conquer” can help, but we are often confronted with a complex problem that is a problem precisely because of the interactions.
- If an aspect of the problem is managing behavior, then clearly map out the incentives each person is facing.
Have you seen this problem before?
- If you find yourself solving the same problem or variations on it, you probably have a higher level process problem or have yet to find a root cause corrective action.
- Document the basis for your diagnosis and see if these same symptoms could be the result of another root cause or collection of causes.
Have you seen this problem in a slightly different form? Do you know a related problem?
- In addition to your reviewing your own experience, it’s always a good idea to research how others have solved this problem or related ones and what they tried that failed. As Frank H. Westheimer observed, “A month in the laboratory can often save an hour in the library.”
- Building a model of the problem can makes the search for similar problems easier.
Try to think of a familiar problem having the same or a similar unknown
- We do this naturally: there is a real risk is seeing a similarity that isn’t really there. To help guard against this you should be explicit about why the familiar problem “rhymes” with the one you need to solve now.
Suppose you find a problem related to yours that has already been solved. Can you use it? Can you use its method?
- Put another way: look for related problems, even from other contexts, and how they were solved.
- A “picnic in the graveyard “can help you find solutions that were tried and failed but may now be viable.
Can you restate your problem? How many different ways can you restate it? More general? More specific? Can the rules be changed?
- Drawing analogies to other problems can also be very helpful in getting oriented. Even a poor analogy can provide a starting point for refinement.
What are the best, worst, and most probable cases you can imagine?
- In decision analysis, 10%, 90%, and 50% probabilities are often substituted for best, worst, and average.

The Plan / Defining the Solution

Following is excerpted from “ThinkerToys” by Michael Michalko. The first level bold bullets are his; the second level bullets are my related comments and observations.

Can you solve the whole problem? Part of the problem?
- While “small wins” are useful, beware of sub-optimizing or solving the parts that are easily tractable.
- Always consider that you are part of the problem in particular when you are in a hole stop digging.
What would you like the resolution to be? Can you picture it?
- One way to solve a problem is to live with it. Make sure that you are working on a critical problem before investing effort in solving it.
How much of the unknown can you determine?
- When answering this, always be clear on the cost and the time frame to measure or bound an unknown. It’s often the case that a small effort can reduce uncertainty to an acceptable level. It may be more prudent to allocate the cost and time needed to achieve higher accuracy to reducing other uncertainties.
Can you derive something useful from the information you have?
- Crafting hypotheses (“taking guesses”) is one way to proceed with limited information.
- What simple and relatively safe, low cost, and short-term actions can you take to refine your understanding of the problem.
Have you used all the information?
- Normally only a few pieces of information are critical, don’t obsess about developing a detailed model when a simple one is sufficient.
- Consider the implications for the information you have. If a key piece of information were wrong, how would you determine that?
- What key piece of missing information would change your mind?
Have you taken into account all essential notions in the problem?
- It’s easy to become fixated on one aspect. Don’t focus only on your areas of expertise. You risk acting like the the drunk who lost his keys trying to unlock his car but searches for them under an outdoor lamp in another part of the parking lot because the light is better there. Many problems will require you to master new concepts and skills.
- If you wanted to make the problem worse what would you do? This can give some useful hints for actions to avoid.
Can you separate the steps in the problem-solving process? Can you determine the correctness of each step?
- Working backward from the solution is another check on your process.
- Problems solved by a flash of insight may be suddenly seen in a new light.
- Keep a pad and paper and something write with by your bed in case you get an insight in the middle of the night.
What creative thinking techniques can you use to generate ideas? How many different techniques?
- Michael Michalko offers a wealth of techniques in “ThinkerToys.”
- Pursue “ridiculous ones” as a thought experiment if you are all out of good ideas. Scott Adams calls this “writing the bad version” (see “Six Tips For Writing An Email“).
Can you see the result? How many different kinds of results can you see?
- Pursue multiple approaches in parallel.
How many different ways have you tried to solve the problem?
- Keep track of what have already tried. It can keep you from wandering in circles, it allows a team to take a divide and conquer approach with different members trying different methods, and it enables you to brief newcomers or outside experts to bring them up to speed on your progress.
- Keep a detailed log of what you have already tried. It can keep you from wandering in circles and it allows you to explore variations on earlier attempts by varying your approach slightly each time.
- A shared log allows a team to take a divide and conquer approach with different members trying different methods.
- A log enables you to brief newcomers or outside experts to bring them up to speed on your progress.
What have others done?
- The trick here is in picking the right others. An excellent place to start is with people or firms who are similar. But don’t stop there. Look farther afield at other industries and professions: you may find practices that can be re-purposed to yield creative breakthroughs.
- It’s also helpful to look for implicit solutions that others have adopted, traditions, or practices that prevent the problem from occurring in the first place. One way to look for these is to ask, “who should have this problem but does not?”
Can you intuit the solution? Can you check the result?
- Intuition can provide an excellent starting point for a solution. I am reminded of a quote by Ingmar Bergman reported in “Ingmar Bergman Confides in Students” (New York Times, May 7, 1981): “I make all my decisions on intuition. I throw a spear into the darkness. That is intuition. Then I must send an army into the darkness to find the spear. That is intellect.”
- Unless you have considerable experience with the system that the problem is embedded in, your intuition may prove faulty. Jay W. Forrester provides two warnings in Urban Dynamics (1969) regarding obvious or intuitive solutions. The first, sometimes referred to as Forrester’s Law, is “In complicated situations efforts to improve things often tend to make them worse, sometimes much worse, on occasion calamitous, because the obvious thing to do is often dead wrong.” His second is more blunt, “Any intuitive alteration of a complex system will cause it to become worse off.”
- Intuition can make a positive contribution but is better deployed to suggest starting points or trial solutions than a final and complete solution.
What should be done? How should it be done?
- Pay as much attention to goals as tasks so you don’t lose sight of your objective.
- “How should it be done” is a question for routine recurring challenges that are amenable to a standard operating procedure. Be careful of focusing too much on how when addressing a strategic problem.
Where should it be done?
- Where is the pain worst?
- Where is the culture most receptive?
- Where is the need greatest?
- Where can we judge results the most rapidly?
When should it be done?
- Synchronize this effort with other processes in your firm unless this is the dominant problem facing your team or organization. Then it becomes a priority interrupt.
Who should do it?
- Who has relevant expertise and experience?
- Who is most familiar with problem?
- Who is most familiar with the skills or techniques the solution or plan relies on?
What do you need to do at this time?
- Is this a critical business issue that represents an interrupt to normal practice?
- Is this a migration effort that needs sustained focus over a long period of time?
- Is this a maintenance task that needs to be addressed but not necessarily urgently?
- What problem will you promote when you solve this problem? Should you work on that problem first?
Who will be responsible for what?
- Define your goals, the roles needed to meet them, and finally, the process or procedures that should be followed. Here is another phrasing for this rule of thumb: attack what first, then who, then when and where.
- Consider who can act as a backup for each person with primary responsibility for a task. Define clear triggers, for example, deadlines or performance thresholds, when the primary responsibility should be reconsidered.
Can you use this problem to solve some other problem?
- It never hurts if your solution acts as a two-bird stone, solving two–or more–problems at once. But if this is a critical problem, don’t let a desire for generality delay implementation where it matters most.
- This a better question for an after-action once the plan has been successfully executed. Otherwise, you risk including additional constraints from related problems before you are certain that your basic solution is satisfactory.
What is the unique set of qualities that makes this problem what it is and none other?
- Invert this for additional insights: what categories is this product a member of?
- Always look at the people involved to make each situation unique. What more can you learn about their specific needs and goals.
What milestones can best mark your progress?
- What accomplishments would reduce your risk of not finding a solution?
- How can you measure your distance from a viable solution? (This is a different but equally useful question from, “how much progress have you made?”)
How will you know when you are successful?
- This is a key question. For most entrepreneurs a “good enough” solution arrived at quickly and at low cost should be the target.
- For entrepreneurs, success means balancing a number of trade-offs that change as the business grows, faces new competitors, enters new markets, or operates under new economic conditions. As Thomas Sowell observed, “There are no solutions, there are only trade-offs; and you try to get the best trade-off you can get, that’s all you can hope for.” As conditions change, you need to re-examine your earlier trade-offs.

M	T	W	T	F	S	S
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

The Phoenix Checklist for Framing a Problem and Its Solution