Sunday, August 13, 2006
PONDERING THE LIMITS OF THE INTRACTABLE: ON " WICKED PROBLEMS "
For the purposes of promoting clear thinking, Dave Schuler recently had a very informative post "Theseus’s Clew: strategies, meta-strategies, and “wicked problems” "; if you wish to look at the dynamics of conflict based scenarios with a clear ( some might say " glittering") eye, then you should read Dave's post in full.
But for the more slothful of my readers, an excerpt from Dave on the nature of " wicked problems":
"Even more unfortunately there are many real-world problems that have neither engineering solutions like the first class or negotiated solutions like the second. These are the difficult problems and, in some cases, these have been called “wicked problems”.
There are many reasons that a problem may be a wicked problem:
* the problem may be ill-defined
* the stakeholders in the problem may have dramatically different world views and framewor for understanding the problem
* the problem may have no stopping rules
* the problem may be unique and previous experience may not be applicable
Or, in many cases, the very act of selecting an approach to solving the problem permanently forecloses other alternatives. It is impossible to arrive at an iterative solution to the problem.
Consider, for example, the mythological Greek hero Theseus. Theseus navigated through the Minotaur’s maze with a clew, a ball of yarn. The clew gave him the ability to trace back to his starting point. Without it he’d have wandered the maze forever.
That’s the key to any iterative solution: you’re able to return to some point of departure and try another way. But when the initial choice precludes returning to the starting point, i.e. the decision has consequences, you can’t just try another way. When you’ve chosen the second branch, the only way out of the maze was through the first branch, and the first branch is no longer accessible to you, you’re stuck. There may no longer be a solution."
Dave offered some excellent strategies for dealing with "wicked problems", all of which I find to be both useful and generally correct. But as he asked me for some feedback, I have to say that there is more to the story here and some nuances to "wicked problems" that Dave did not include in his concisely written post.
First of all, not all "wicked problems" were created equally wicked. We must differentiate between those problems that are intractable from those that are merely hard or prohibitively expensive. The latter involves a significant degree of human value choice while the former is effectively beyond any direct solution within our present power to efface. Many cutting edge scientific questions are temporarily intractable until, say for example, computing power increases by a given order of magnitude. Some philosophical or religious questions, perhaps dealing with the nature of God or the afterlife, are intractable in a permanent sense.
"Wicked problems" dealing with conflict in a complex social system are not usually intractable, though we often use that word to describe very difficult to solve conflicts in places like the Middle East or Northern Ireland. What we really mean in such cases is that the problems are exceedingly complicated as well as deeply rooted in terms of psychological and emotional investment for those involved. We often describe the second aspect as being "self-destructive" or " irrational" but in economic terms it is not irrational behavior if you place overriding value on minimizing your opponent's gains ( though it may indeed be self-destructive to execute such a strategy at all costs) even if that value-set is a product of your own skewed perception of events.
How to deal with such " wicked problems"? Here are several options to consider for non-intractable but difficult scenarios:
For the purposes of promoting clear thinking, Dave Schuler recently had a very informative post "Theseus’s Clew: strategies, meta-strategies, and “wicked problems” "; if you wish to look at the dynamics of conflict based scenarios with a clear ( some might say " glittering") eye, then you should read Dave's post in full.
But for the more slothful of my readers, an excerpt from Dave on the nature of " wicked problems":
"Even more unfortunately there are many real-world problems that have neither engineering solutions like the first class or negotiated solutions like the second. These are the difficult problems and, in some cases, these have been called “wicked problems”.
There are many reasons that a problem may be a wicked problem:
* the problem may be ill-defined
* the stakeholders in the problem may have dramatically different world views and framewor for understanding the problem
* the problem may have no stopping rules
* the problem may be unique and previous experience may not be applicable
Or, in many cases, the very act of selecting an approach to solving the problem permanently forecloses other alternatives. It is impossible to arrive at an iterative solution to the problem.
Consider, for example, the mythological Greek hero Theseus. Theseus navigated through the Minotaur’s maze with a clew, a ball of yarn. The clew gave him the ability to trace back to his starting point. Without it he’d have wandered the maze forever.
That’s the key to any iterative solution: you’re able to return to some point of departure and try another way. But when the initial choice precludes returning to the starting point, i.e. the decision has consequences, you can’t just try another way. When you’ve chosen the second branch, the only way out of the maze was through the first branch, and the first branch is no longer accessible to you, you’re stuck. There may no longer be a solution."
Dave offered some excellent strategies for dealing with "wicked problems", all of which I find to be both useful and generally correct. But as he asked me for some feedback, I have to say that there is more to the story here and some nuances to "wicked problems" that Dave did not include in his concisely written post.
First of all, not all "wicked problems" were created equally wicked. We must differentiate between those problems that are intractable from those that are merely hard or prohibitively expensive. The latter involves a significant degree of human value choice while the former is effectively beyond any direct solution within our present power to efface. Many cutting edge scientific questions are temporarily intractable until, say for example, computing power increases by a given order of magnitude. Some philosophical or religious questions, perhaps dealing with the nature of God or the afterlife, are intractable in a permanent sense.
"Wicked problems" dealing with conflict in a complex social system are not usually intractable, though we often use that word to describe very difficult to solve conflicts in places like the Middle East or Northern Ireland. What we really mean in such cases is that the problems are exceedingly complicated as well as deeply rooted in terms of psychological and emotional investment for those involved. We often describe the second aspect as being "self-destructive" or " irrational" but in economic terms it is not irrational behavior if you place overriding value on minimizing your opponent's gains ( though it may indeed be self-destructive to execute such a strategy at all costs) even if that value-set is a product of your own skewed perception of events.
How to deal with such " wicked problems"? Here are several options to consider for non-intractable but difficult scenarios:
- Avoidance: The costs to benefits ratio of becoming deeply engaged in solving "wicked problems" are often unfavorable, even should you be successful. Unlike with the Middle East, the United States has never, for many reasons, invested much prestige or resources in remediating the legacy of 700 years of "Irish troubles" in Northern Ireland. Arguably, without any significant harm to our national interests whatsoever.
- New Eyes: If you must get involved, force yourself analyze the problem from a wide range of perspectives, the more non-traditional the better. Hone in on those perspectives that yield options with the greatest systemic effects even if those effects do not " solve" the wicked problem per se.
- The Gordian Knot: Every social system-based "wicked problem" represents a dynamic that is difficult to solve in part due to the rule-sets under which the participants are operating or the rule -set interacting with a unique environment. You can consider swallowing hard and just cutting through the entire mess by rejecting the entire paradigm in a single bold stroke ( a " Big Bang" system perturbation) that renders the problem irrelevant. This is of course, a risky move that invites replacing old problems with new and possibly worse ones.
- Rule set Reset: Somewhat akin to the Gordian knot, old paradigms are replaced with new ones because the advantages of so doing create a new consensus behind them. This is difficult but not impossible to pull-off. The Westphalian system of state sovereignty was a rule-set reset that reduced the geopolitical incentives for pursuing religious-dynastic warfare existing during the medieval world. The combined changes of nuclear weapons, the UN, Bretton Woods, and Cold War bipolarity was a partial rule-set reset from the diplomatic norms of the prewar era.
For "wicked problems" that are not intractable, wickedness is often in the eye of the beholder.
ADDENDUM:
More to come later tonight.
¶ 5:00 PM
Comments:
<< Home
Thanks, Mark. And your comments are extremely good and relevant. One of these days I'll need to post on problem patterns and the decision process for identifying what sort of problem you're dealing with.
So far much of the commentary on my post has been from one of two tacks. The more insightful immediately proposed some variant of the “Gordian Knot” approach. As with the original story such a solution requires real hegemony.
The less insightful just insisted that all problems had engineering solutions, demonstrating that they didn't understand the problem.
Remarkable how this rather abstract discussion provides a window into understanding the real-world approaches taken to solving certain problems, notably Iraq and Israel.
So far much of the commentary on my post has been from one of two tacks. The more insightful immediately proposed some variant of the “Gordian Knot” approach. As with the original story such a solution requires real hegemony.
The less insightful just insisted that all problems had engineering solutions, demonstrating that they didn't understand the problem.
Remarkable how this rather abstract discussion provides a window into understanding the real-world approaches taken to solving certain problems, notably Iraq and Israel.
I believe that there are three other ways of dealing with wicked but not intractable problems that should be noted: attenuation, decomposition, and accretion.
It may be that a wicked problem is not a single problem; indeed, in my experience, wicked problems are more often caused by the interactions between smaller inter-dependant problems than by a single problem being too large to solve.
In such cases, you can often work towards a solution by choosing critical subproblems and working to solve them. This will not solve the central problem, and in fact may make the central problem apparently worse. At the same time, though, the complexity of the total problem set goes down: the problem has attenuated somewhat by changes in its component parts. For example, the Middle East political solution in the 1970s was such a wicked problem. While the land-for-peace solution was ultimately incomplete, the problem's complexity was significantly reduced by the attempt: Egypt and Jordan essentially made peace with Israel. In the process, the Palestinian problem was made nearly intractable, and the Hizb'allah were created. Yet the problem is arguably easier now than it was in 1978. The take down of Iraq's government certainly reduced the problem set, and it's possible that taking down either or both of Iran and Syria could solve the problem by making it possible for Israel to reach settlements with its neighbors that do not include killing off the Jews and destroying the state of Israel.
Similarly, if a wicked problem is caused more by the relationships between smaller issues than by the largest issues in the problem set, it may be possible to eliminate or reduce the wicked aspect of the problem by separating the subproblems from each other. For example, Saddam in 1991 tried to tie the invasion of Kuwait to the Palestinian problem, thus making the invasion of Kuwait much harder to reverse. The opposite approach, decomposing a large problem into smaller problems by removing the connections between them, can make the large problem more easily solvable, or at least can create a situation where the larger problem can be improved.
Finally, sometimes a complete rethink of the problem is necessary. Besides a "big bang" approach, there is another way of doing this: accretion. In other words, make the problem bigger by tying other problems into it. It is sometimes, though it seems paradoxical, easier to solve a larger problem than a smaller one. This may be a sub-class of "rule set reset" though.
It may be that a wicked problem is not a single problem; indeed, in my experience, wicked problems are more often caused by the interactions between smaller inter-dependant problems than by a single problem being too large to solve.
In such cases, you can often work towards a solution by choosing critical subproblems and working to solve them. This will not solve the central problem, and in fact may make the central problem apparently worse. At the same time, though, the complexity of the total problem set goes down: the problem has attenuated somewhat by changes in its component parts. For example, the Middle East political solution in the 1970s was such a wicked problem. While the land-for-peace solution was ultimately incomplete, the problem's complexity was significantly reduced by the attempt: Egypt and Jordan essentially made peace with Israel. In the process, the Palestinian problem was made nearly intractable, and the Hizb'allah were created. Yet the problem is arguably easier now than it was in 1978. The take down of Iraq's government certainly reduced the problem set, and it's possible that taking down either or both of Iran and Syria could solve the problem by making it possible for Israel to reach settlements with its neighbors that do not include killing off the Jews and destroying the state of Israel.
Similarly, if a wicked problem is caused more by the relationships between smaller issues than by the largest issues in the problem set, it may be possible to eliminate or reduce the wicked aspect of the problem by separating the subproblems from each other. For example, Saddam in 1991 tried to tie the invasion of Kuwait to the Palestinian problem, thus making the invasion of Kuwait much harder to reverse. The opposite approach, decomposing a large problem into smaller problems by removing the connections between them, can make the large problem more easily solvable, or at least can create a situation where the larger problem can be improved.
Finally, sometimes a complete rethink of the problem is necessary. Besides a "big bang" approach, there is another way of doing this: accretion. In other words, make the problem bigger by tying other problems into it. It is sometimes, though it seems paradoxical, easier to solve a larger problem than a smaller one. This may be a sub-class of "rule set reset" though.
Hi Dave & Jeff,
"The less insightful just insisted that all problems had engineering solutions, demonstrating that they didn't understand the problem"
That amused me -examples of self-referential behavior usually do
"Remarkable how this rather abstract discussion provides a window into understanding the real-world approaches taken to solving certain problems, notably Iraq and Israel."
Indeed. That brings me to Jeff's observations:
"in my experience, wicked problems are more often caused by the interactions between smaller inter-dependant problems than by a single problem being too large to solve."
Very, very true. Interrelated and interdependent problems are are bitch to solve - esp. when some obviously negative behavior with a systemic impact actually makes crude sense for individuals operating at the lower levels of Maslow's hierarchy.
"In such cases, you can often work towards a solution by choosing critical subproblems and working to solve them. This will not solve the central problem, and in fact may make the central problem apparently worse. At the same time, though, the complexity of the total problem set goes down"
Excellent observation.
I think that partly depends, however, on the observer's perspective. Zeroing in on just Iraq, I could argue the complexity has risen but the magnitude of threat has receded as Iraq is obviously not going to build nukes or invade KSA in the near future - that wicked problem was resolved and replaced with others.
"Finally, sometimes a complete rethink of the problem is necessary. Besides a "big bang" approach, there is another way of doing this: accretion. In other words, make the problem bigger by tying other problems into it. It is sometimes, though it seems paradoxical, easier to solve a larger problem than a smaller one. This may be a sub-class of "rule set reset" though. "
I lean in that direction -excellent point on changing scales.
Post a Comment
"The less insightful just insisted that all problems had engineering solutions, demonstrating that they didn't understand the problem"
That amused me -examples of self-referential behavior usually do
"Remarkable how this rather abstract discussion provides a window into understanding the real-world approaches taken to solving certain problems, notably Iraq and Israel."
Indeed. That brings me to Jeff's observations:
"in my experience, wicked problems are more often caused by the interactions between smaller inter-dependant problems than by a single problem being too large to solve."
Very, very true. Interrelated and interdependent problems are are bitch to solve - esp. when some obviously negative behavior with a systemic impact actually makes crude sense for individuals operating at the lower levels of Maslow's hierarchy.
"In such cases, you can often work towards a solution by choosing critical subproblems and working to solve them. This will not solve the central problem, and in fact may make the central problem apparently worse. At the same time, though, the complexity of the total problem set goes down"
Excellent observation.
I think that partly depends, however, on the observer's perspective. Zeroing in on just Iraq, I could argue the complexity has risen but the magnitude of threat has receded as Iraq is obviously not going to build nukes or invade KSA in the near future - that wicked problem was resolved and replaced with others.
"Finally, sometimes a complete rethink of the problem is necessary. Besides a "big bang" approach, there is another way of doing this: accretion. In other words, make the problem bigger by tying other problems into it. It is sometimes, though it seems paradoxical, easier to solve a larger problem than a smaller one. This may be a sub-class of "rule set reset" though. "
I lean in that direction -excellent point on changing scales.
<< Home
