Think Again

THE YODA EFFECT: “YOU MUST UNLEARN WHAT YOU
HAVE LEARNED”
On my quest to find people who enjoy discovering they were wrong, a
trusted colleague told me I had to meet Jean-Pierre Beugoms. He’s in his
late forties, and he’s the sort of person who’s honest to a fault; he tells the
truth even if it hurts. When his son was a toddler, they were watching a
space documentary together, and Jean-Pierre casually mentioned that the
sun would one day turn into a red giant and engulf the Earth. His son was
not amused. Between tears, he cried, “But I love this planet!” Jean-Pierre
felt so terrible that he decided to bite his tongue instead of mentioning
threats that could prevent the Earth from even lasting that long.

---

Back in the 1990s, Jean-Pierre had a hobby of collecting the
predictions that pundits made on the news and scoring his own forecasts
against them. Eventually he started competing in forecasting tournaments—
international contests hosted by Good Judgment, where people try to predict
the future. It’s a daunting task; there’s an old saying that historians can’t
even predict the past. A typical tournament draws thousands of entrants
from around the world to anticipate big political, economic, and
technological events. The questions are time-bound, with measurable,
specific results. Will the current president of Iran still be in office in six
months? Which soccer team will win the next World Cup? In the following
year, will an individual or a company face criminal charges for an accident
involving a self-driving vehicle?
Participants don’t just answer yes or no; they have to give their odds.
It’s a systematic way of testing whether they know what they don’t know.
They get scored months later on accuracy and calibration—earning points
not just for giving the right answer, but also for having the right level of
conviction. The best forecasters have confidence in their predictions that
come true and doubt in their predictions that prove false.
On November 18, 2015, Jean-Pierre registered a prediction that
stunned his opponents. A day earlier, a new question had popped up in an
open forecasting tournament: in July 2016, who would win the U.S.
Republican presidential primary? The options were Jeb Bush, Ben Carson,
Ted Cruz, Carly Fiorina, Marco Rubio, Donald Trump, and none of the
above. With eight months to go before the Republican National Convention,
Trump was largely seen as a joke. His odds of becoming the Republican
nominee were only 6 percent according to Nate Silver, the celebrated
statistician behind the website FiveThirtyEight. When Jean-Pierre peered
into his crystal ball, though, he decided Trump had a 68 percent chance of
winning.
Jean-Pierre didn’t just excel in predicting the results of American
events. His Brexit forecasts hovered in the 50 percent range when most of
his competitors thought the referendum had little chance of passing. He
successfully predicted that the incumbent would lose a presidential election
in Senegal, even though the base rates of reelection were extremely high
and other forecasters were expecting a decisive win. And he had, in fact,
pegged Trump as the favorite long before pundits and pollsters even
considered him a viable contender. “It’s striking,” Jean-Pierre wrote early

---

on, back in 2015, that so many forecasters are “still in denial about his
chances.”
Based on his performance, Jean-Pierre might be the world’s best
election forecaster. His advantage: he thinks like a scientist. He’s
passionately dispassionate. At various points in his life, Jean-Pierre has
changed his political ideologies and religious beliefs.
*
 He doesn’t come
from a polling or statistics background; he’s a military historian, which
means he has no stake in the way things have always been done in
forecasting. The statisticians were attached to their views about how to
aggregate polls. Jean-Pierre paid more attention to factors that were hard to
measure and overlooked. For Trump, those included “Mastery at
manipulating the media; Name recognition; and A winning issue (i.e.,
immigration and ‘the wall’).”
Even if forecasting isn’t your hobby, there’s a lot to be learned from
studying how forecasters like Jean-Pierre form their opinions. My colleague
Phil Tetlock finds that forecasting skill is less a matter of what we know
than of how we think. When he and his collaborators studied a host of
factors that predict excellence in forecasting, grit and ambition didn’t rise to
the top. Neither did intelligence, which came in second. There was another
factor that had roughly triple the predictive power of brainpower.
The single most important driver of forecasters’ success was how often
they updated their beliefs. The best forecasters went through more
rethinking cycles. They had the confident humility to doubt their judgments
and the curiosity to discover new information that led them to revise their
predictions.
A key question here is how much rethinking is necessary. Although the
sweet spot will always vary from one person and situation to the next, the
averages can give us a clue. A few years into their tournaments, typical
competitors updated their predictions about twice per question. The
superforecasters updated their predictions more than four times per
question.
Think about how manageable that is. Better judgment doesn’t
necessarily require hundreds or even dozens of updates. Just a few more
efforts at rethinking can move the needle. It’s also worth noting, though,
how unusual that level of rethinking is. How many of us can even
remember the last time we admitted being wrong and revised our opinions
accordingly? As journalist Kathryn Schulz observes, “Although small

---

amounts of evidence are sufficient to make us draw conclusions, they are
seldom sufficient to make us revise them.”
That’s where the best forecasters excelled: they were eager to think
again. They saw their opinions more as hunches than as truths—as
possibilities to entertain rather than facts to embrace. They questioned ideas
before accepting them, and they were willing to keep questioning them
even after accepting them. They were constantly seeking new information
and better evidence—especially disconfirming evidence.
On Seinfeld, George Costanza famously said, “It’s not a lie if you
believe it.” I might add that it doesn’t become the truth just because you
believe it. It’s a sign of wisdom to avoid believing every thought that enters
your mind. It’s a mark of emotional intelligence to avoid internalizing every
feeling that enters your heart.
Ellis Rosen/The New Yorker Collection/The Cartoon Bank

---

Another of the world’s top forecasters is Kjirste Morrell. She’s
obviously bright—she has a doctorate from MIT in mechanical engineering
—but her academic and professional experience wasn’t exactly relevant to
predicting world events. Her background was in human hip joint
mechanics, designing better shoes, and building robotic wheelchairs. When
I asked Kjirste what made her so good at forecasting, she replied, “There’s
no benefit to me for being wrong for longer. It’s much better if I change my
beliefs sooner, and it’s a good feeling to have that sense of a discovery, that
surprise—I would think people would enjoy that.”
Kjirste hasn’t just figured out how to erase the pain of being wrong.
She’s transformed it into a source of pleasure. She landed there through a
form of classical conditioning, like when Pavlov’s dog learned to salivate at
the sound of a bell. If being wrong repeatedly leads us to the right answer,
the experience of being wrong itself can become joyful.
That doesn’t mean we’ll enjoy it every step of the way. One of Kjirste’s
biggest misses was her forecast for the 2016 U.S. presidential election,
where she bet on Hillary Clinton to beat Donald Trump. Since she wasn’t a
Trump supporter, the prospect of being wrong was painful—it was too
central to her identity. She knew a Trump presidency was possible, but she
didn’t want to think it was probable, so she couldn’t bring herself to
forecast it.
That was a common mistake in 2016. Countless experts, pollsters, and
pundits underestimated Trump—and Brexit—because they were too
emotionally invested in their past predictions and identities. If you want to
be a better forecaster today, it helps to let go of your commitment to the
opinions you held yesterday. Just wake up in the morning, snap your