The Art of Thinking Clearly: Better Thinking, Better Decisions

See also ‘Because’ Justification (ch. 52); Falsification of History (ch. 78); Hindsight Bias
(ch. 14); Fundamental Attribution Error (ch. 36)

---

98
SPEED DEMONS MAKE SAFE DRIVERS
Intention-To-Treat Error
You’ll find it hard to believe, but speed demons drive more safely than so-called
‘careful’ drivers. Why? Well, consider this: the distance from Miami to West Palm
Beach is around 75 miles. Drivers who cover the distance in an hour or less we’ll
categorise as ‘reckless drivers’ because they’re travelling at an average of 75
mph or more. All others we put into the group of careful drivers. Which group
experiences fewer accidents? Without a doubt, it is the ‘reckless drivers’. They all
completed the journey in less than an hour, so could not have been involved in
any accidents. This automatically puts all drivers who end up in accidents in the
slower drivers’ category. This example illustrates a treacherous fallacy, the so-
called 
intention-to-treat error
. Unfortunately, there is no catchier term for it.
This might sound to you like the 
survivorship bias
(chapter 1), but it’s different.
In the 
survivorship bias
you only see the survivors, not the failed projects or cars
involved in accidents. In the 
intention-to-treat error
, the failed projects or cars with
accidents show up prominently, just in the wrong category.
A banker showed me an interesting study recently. Its conclusion: companies
with debt on their balance sheets are significantly more profitable than firms with
no debt (equity only). The banker vehemently insisted that every company should
borrow at will, and, of course, his bank is the best place to do it. I examined the
study more closely. How could that be? Indeed, from 1,000 randomly selected
firms, those with large loans displayed higher returns not only on their equity but
also on their total capital. They were in every respect more successful than the
independently financed firms. Then the penny dropped: unprofitable companies
don’t get corporate loans. Thus, they form part of the ‘equity-only’ group. The firms
that make up this set have bigger cash cushions, stay afloat longer and, no matter
how sickly they are, remain part of the study. On the other side, firms that have
borrowed a lot go bankrupt more quickly. Once they cannot pay back the interest,
the bank takes over, and the companies are sold off – thus disappearing from the
sample. The ones that remain in the ‘debt group’ are relatively healthy, regardless
of how much debt they have amassed on their balance sheets.

---

If you’re thinking, ‘OK, got it’, watch out. The 
intention-to-treat error
is not easy
to recognise. A fictional example from medicine: a pharmaceutical company has
developed a new drug to fight heart disease. A study ‘proves’ that it significantly
reduces patients’ mortality rates. The data speaks for itself: among patients who
have taken the drug regularly, the five-year mortality rate is 15%. For those who
have swallowed placebo pills, it is about the same, indicating that the pill doesn’t
work. However – and this is crucial – the mortality rate of patients who have taken
the drug at irregular intervals is 30% – twice as high! A big difference between
regular and irregular intake. So, the pill is a complete success. Or is it?
Here’s the snag: the pill is probably not the decisive factor; rather, it is the
patients’ behaviour. Perhaps patients discontinued the pill following severe side
effects and so landed in the ‘irregular intake’ category. Maybe they were so ill that
there was no way to continue it on a regular basis. Either way, only relatively
healthy patients remain in the ‘regular’ group, which makes the drug look a lot
more effective than it really is. The really sick patients who, for this very reason,
couldn’t take the drug on a regular basis, ended up populating the ‘irregular
intake’ group.
In reputable studies, medical researchers evaluate the data of all patients
whom they originally intend to treat (hence the title); it doesn’t matter if they take