the art & science of
FORECASTING
Philip Tetlock, a professor at the Wharton School of the University of Pennsylvania, has helped a generation of forecasters hone their prediction skills. At the university’s Good Judgment Project,
LUCKBOX LEANS IN WITH
PHIL TETLOCK sitory or roaring back,” key figures could work together—with the help of trusted third parties—to identify what they disagree about and how to meaningfully test those disagreements.
WHERE
is adversarial collaboration used, and where do you see the most potential for it? The idea of adversarial collaboration has deep roots in the sciences. A famous example is the adversarial collaboration between prominent psychologists Daniel Kahneman and Gary Klein. Kahneman and Klein each won acclaim by developing such seemingly contradictory schools of thought that each scholar appeared to be a threat to the legacy of the other. They could have chosen to hunker down and fire off rival polemics, but instead they got together to discuss why they had such different views and how those views could be reconciled. The result was a 2009 paper in which they concluded, “We agree on most of the issues that matter.” On the rest, they could disagree without being disagreeable. Superforecasters apply this concept regularly to collaborate and derive more accurate forecasts. The same concept can be used in other domains, from personal investment decisions to policymaking on critical issues of the day. With a debate like “is inflation tran-
14
Why hasn’t adversarial collaboration been adopted on a larger scale? Adversarial collaboration requires good faith and the ability to say, “I was wrong” in the face of evidence. It needs a neutral platform and clear tests of one’s beliefs. But in noisy public arenas, that doesn’t make good television. This is probably why the media often turn to the loudest talking heads who do make good television no matter how bad their track record may be. Sadly, those voices usually have zero interest in adversarial collaboration. In other areas also, people tend to find a confident “yes” or “no” more satisfying than “maybe” and “on the other hand.” Research shows people trust more confident financial advisers over those who are less confident, even when their track records are identical. People all too often equate confidence and competence and mistakenly see concessions in a debate as a sign of weakness. Would politicians be likely to embrace adversarial collaboration to reach policy decisions? Individual politicians can be adversaries and collaborators. Sen. Bob Dole had strong views about how the world should be yet regularly reached across the aisle to craft legislation. Ted Kennedy and Orrin Hatch were strong adversaries in debate but collaborated productively on many issues. So it can be done. It’s really up to the politicians to do it, and right now the emphasis seems to be more on “adversary” than “collaboration.”
Tetlock worked alongside his wife and research partner Barbara Mellers to form a team of forecasters so accurate that they outperformed intelligence analysts who had access to classified data. He and journalist Dan Gardner were coauthors of an influential New York Times bestseller, Superforecasting: The Art and Science of Prediction. For this article, Tetlock agreed to field questions about adversarial collaboration, a process that creates solid predictions by combining differing views.
What challenges stand in the way of more widespread adversarial collaboration? The biggest obstacle is the reluctance of most political leaders to take the risk and present their beliefs in a way that can be rigorously put to the test. The answer is for all of us to hold them accountable and recognize leaders who are willing to take that risk. A psychological mechanism is at work here. Studies show that if people feel they will be held accountable for their views, they tend to avoid cognitive pitfalls such as overconfidence and the failure to update beliefs in response to new evidence. But if people make predictions that can never be disproved, it poses no reputational risk, and, in turn, encourages overconfidence and even more extreme predictions. If we get serious about keeping score, with it will come accountability that is dismally lacking in public discourse today. What’s the most important thing to know about adversarial collaboration? Adversarial collaboration is about disagreeing without being disagreeable. It is about having the integrity to examine one’s beliefs in a set of clear tests. It is about making more accurate forecasts and getting a little wiser each time. Ultimately, it’s about improving public discourse.
Luckbox | January / February 2022
2201_TOPICS_Adversarial_Collab.indd 14
12/17/21 4:35 PM
