Python Programming for Biology: Bioinformatics and Beyond

Type I and II errors

Because  statistics  deals  with  predictions  and  probability,  some  mistakes  will  be  made.

Thus, it is important to consider how we can quantify the errors of predictions made on the

basis  of  statistical  analysis.  With  a  mathematical  formulation  based  on  testing  a  null

hypothesis (a random control), the analysis can lead to two types of errors:

A  Type  I  error  is  the  situation  where  the  null  hypothesis  has  wrongly  been  rejected

when it is really correct.

A Type II error is the situation when the null hypothesis has not been rejected when it

---

should have been, because it is really incorrect.

Alternatively,  we  might  say  that  these  errors  are  respectively  false  positive  and  false

negative as far as the alternative hypothesis is concerned. As an example of a Type I error,

a statistical analysis of a specific trial might lead us to believe that a drug has a beneficial

effect,  thus  favouring  the  alternative  hypothesis  and  rejecting  the  null  hypothesis,  when

actually there was no real benefit and the null hypothesis was true. The end result is that

further development of the drug would be needlessly undertaken. Conversely for a Type II

error, a new drug might have a worthwhile effect but statistics of a specific trial might lead

us  to  believe  that  it  does  not,  incorrectly  accepting  the  null  hypothesis,  meaning  that

further development of a promising drug might be stopped.

Naturally, both kinds of errors are potentially serious. In a given situation, with a given

sample size, you could specify in advance what Type I or Type II error you are willing to

tolerate.

1

However, by decreasing the probability of a Type I error you generally increase

the  probability  of  a  Type  II  error,  and  vice  versa.  For  example,  if  we  incorporate  a  new

medical  test  that  makes  it  less  likely  to  detect  a  spurious  effect  from  a  drug  we  will  get

fewer false positives, but we will tend to miss more marginal cases and have more false

negatives. Hence, it is commonplace to have a balance between Type I and Type II errors,

depending on things like financial cost and danger etc. Also, for a specified Type I error

level,  you  can  decrease  the  probability  of  a  Type  II  error  by  increasing  the  sample  size,

and  vice  versa.  This  is  because  having  more  data  values  means  we  get  more  precise

estimates  of  the  null  hypothesis  parameters,  so  for  a  fixed  Type  I  error  probability  the

Type II error is less affected by the variation due to the random data sampling.