55 pages 1 hour read

Superforecasting: The Art and Science of Prediction

Nonfiction | Book | Adult | Published in 2015

A modern alternative to SparkNotes and CliffsNotes, SuperSummary offers high-quality Study Guides with detailed chapter summaries and analysis of major themes, characters, and more.

Themes

Hedgehogs and Foxes

In the sixth century BCE (before the Common Era), the Greek lyric poet Archilochus wrote, “The fox knows many things but the hedgehog knows one big thing” (69). Archilochus remained silent on whether hedgehogs or foxes fared better, but in the 1953 essay “The Hedgehog and the Fox,” 20th-century philosopher Isaiah Berlin preferred foxes. While hedgehogs regard life through a singular big idea, foxes draw upon a multiplicity of experiences to deliver their life’s work. A typical hedgehog from history might be the ancient Greek philosopher Plato, with his world of ideals, while a fox would be the 16th-century British playwright William Shakespeare. While Tetlock claims to have no preference between hedgehogs and foxes, he enjoys the metaphor because “it captured something deep in my data” when he found that big-idea experts were more like hedgehogs, while more open-minded equivocal thinkers with an interest in many topics were foxes (69).

Tetlock’s view is that the media prefers to feature hedgehogs rather than foxes. The former, with their controversial views and consistent opinions, become like beloved, if stereotypical, sitcom characters. Even if they keep spouting a “truth” that is inconsistent with current reality, their constancy of opinion has a lulling effect on the audience. Tetlock shows how “prototypic hedgehog” Larry Kudlow, a Republican political commentator and financial analyst, was able to recover from maintaining that his big idea (supply-side economics) was triumphing during George W. Bush’s presidency despite growing evidence to the contrary (69). According to Tetlock, Kudlow was able to peddle this delusion (perhaps even to himself) because supply-side economics was “the Big Idea he uses over and over when trying to figure out what will happen next” and is akin to green-tinted glasses that cause the wearer to perceive the world with a verdant hue (71). People with Kudlow’s predilection for seeing the world through the distorted lens of their main idea do not make good forecasters, regardless of political persuasion. This is because the more they know on a topic, the less they are willing to test their hypotheses against counterarguments and new information. However, while they would never make it as a GJP superforecaster, hedgehogs in the media often recover quickly from their misleading predictions given “the inverse correlation between fame and accuracy”; Tetlock’s research found that the “the more famous an expert was, the less accurate he was” (72). Hedgehogs’ less accurate predictions often make better stories because they omit the troubling, contradictory nuance of reality. Hedgehogs can be very persuasive, adding on logical-seeming reasons about why they are right without considering other perspectives. This panders to the human evolutionary fear of uncertainty and the System 1 thinking that favors snap judgments to quickly identify predators in the shadows.

Foxes, in contrast, with their multiple perspectives and ambivalence, are less compelling media personalities. Their stories are layered, complex, and often contradictory in a way that is unsatisfying and even unsettling to audiences. However, while “the aggregation of many perspectives is bad TV […] it’s good forecasting” (72). In his book, Tetlock aims to overcome the media’s negativity toward foxes by painting attractive portraits of these mavericks, who have overcome difficulties, have many interests, and remain youthfully curious and open-minded. For example, Tetlock describes fox-natured superforecaster Devyn Duffy as “a fantastic updater,” implying that his main intellectual attribute is versatility and growth (153). For example, Duffy became a superforecaster with the GJP after being laid off from his Pittsburgh factory job, and he adapted quickly. He exhibits Carol Dweck’s notion of the growth mindset—a characteristic that, the authors claim, is more typical of foxes than hedgehogs, chiefly because foxes believe intelligence is mutable rather than fixed. This gives them an advantage in making predictions about a changing world. However, foxes also need to be humble and aware that they may stumble into the illusion of unassailable expertise. Superforecaster Jean-Pierre Beugoms, for example, who is knowledgeable about military matters, knows that he must exercise extra caution on this topic so that he does not fall into hubris. To keep forecasting accurately, he must be open-minded about the world and his own limitations.

Throughout the book, the authors argue that as we learn to appreciate uncertainty and consider views that oppose our own, we will come to revere the fox as much as the hedgehog. During this journey, we will learn the value of complexity, curiosity, and humility rather than big personalities and easy answers. However, at the end, the authors acknowledge that hedgehogs have their role in being superquestioners, as their propensity for confidently tackling big ideas can set the agenda for superforecasters’ investigations. Thus, the dream team would be made up of both foxes and hedgehogs.

Forecasting: Between Science and Art

The book’s full title is Superforecasting: The Art and Science of Prediction, encompassing disciplines that are often opposed in popular culture. While science is associated with information-gathering and the quest for objectivity, art is considered to be more subjective and about finding a creative pathway through a subject. In the course of their study, the authors make a case for applying both scientific and artistic principles to forecasting.

The authors strive to make forecasting more of a science when they hold up evidence-based medical testing as a model for the discipline’s future. At the beginning of the 20th century, evidence-based medicine was seen as a novelty and inconvenience by physicians who valued their personal experiences above such systematic empirical data, but by the end of that same century, evidence-based medicine was something that people took for granted. According to Tetlock, much contemporary forecasting, with its propensity to use undefined terms such as “probably” and “might,” is about as objective as pre-20th-century medicine. While this verbal ambiguity often benefits institutions such as politicians and the media, it does nothing to improve forecasts; it is far more difficult to correct and learn from a verbal forecast that has gone awry as opposed to one that has been given a numerical probability. Numerical probability also identifies errors more starkly and thereby holds forecasters accountable. Giving number values would help correct mediocrity in the media and other public spheres, where commentators can make inaccurate forecasts and then get away with it. In contrast, the GJP works with solely numerical predictions, which makes forecasting a science from the outset.

Tetlock also seeks to make forecasting more scientific by applying guidelines. For example, he unequivocally prescribes taking the outside view of a problem, or putting “the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena” (153). Another mathematical principle, the regression to the mean, is also useful in good forecasting, as it prevents forecasters from giving undue credit to their own skill (as opposed to luck).

Still, an artist’s independence of thought and their ability to liberally abandon the rules when the situation requires it is also helpful to becoming a superforecaster. Tetlock quotes the celebrated writer George Orwell, whose sixth and final rule on writing was “break any of these rules sooner than saying anything outright barbarous” (172). Similarly, a good forecaster must dispense with the illusion of certainty in forecasting guidelines and instead use “nuanced judgment” (172). Arguably, it is this difficult dance between rigorous methodology and creative solutions that makes a superforecaster.

Forecasting and the Crucial Ingredient of Doubt

It is a truism to state that the future is unpredictable. Forecasters must nevertheless honor this fact in their approach to prediction as they consider as many angles as possible.

Nassim Taleb, who believes the future is full of black swan events that cannot be anticipated and that confound our expectations of what is possible, doubts the usefulness of forecasting altogether. Instead, he proposes that we plan for the unexpected—investors would then strategize based on the assumption that a good proportion of their investments will fail. While the authors acknowledge that life is full of anticipated events, they argue that the recurrence of these events in themselves means that we can go some way toward predicting them. Thus, instead of determining exactly where and when a terrorist attack will take place, forecasting can estimate based on various potential catalysts. Additionally, the GJP’s philosophy of obtaining multiple forecasts for the same event staves off the errors that result from human arrogance and the delusion that a single person can have the final word on a particular topic. Aggregating multiple superforecasters’ prediction scores offers a better chance of obtaining a reliable picture of future peril.

Tetlock argues that superforecasters can make space for Taleb’s black swans by preserving doubt in every step of their judgment process. Even before they compare their predictions to those of other team members, superforecasters foster ambivalence by considering both sides of the argument. Thus, when tackling such questions as whether Yasser Arafat died of polonium poisoning, superforecasters must set aside their gut convictions and ask themselves what it would take for the answer to be both yes and no. They subsequently follow the methods of contemporary scientists, who are only satisfied with their hypotheses after ruling out rival hypotheses. Such strategic doubt prevents superforecasters both from doggedly sticking to an initial idea as a hedgehog might and from being overly swayed by news stories.

Importantly, even after a question has closed and the results of a particular problem are in, superforecasters continue interrogating their process. Regardless of whether they have been wrong or right in their estimates, they comb back through their reasoning to look for potential errors. This enables them to maintain the growth mindset of the eternal student rather than the fixed mindset that is so inimical to good forecasting.

blurred text
blurred text
blurred text
Unlock Icon

Unlock all 55 pages of this Study Guide

Plus, gain access to 9,100+ more expert-written Study Guides.

Including features:

+ Mobile App
+ Printable PDF
+ Literary AI Tools