Python Programming for Biology: Bioinformatics and Beyond

Bayesian analysis

Bayesian  analysis  is  a  very  powerful  and,  in  the  minds  of  some,  the  ‘proper’  way  to

generally  think  about  scientific  matters.  Scientific  philosophy  is  largely  based  upon

proving  or  disproving  hypotheses  using  experimental  evidence.  Thinking  in  general  and

abstract terms with the above example, and introducing the symbolic notation ‘|’ to mean

‘given’, we have:

Pr(Hypothesis

i

| Data) Pr(Data) = Pr(Data | Hypothesis

i

) Pr(Hypothesis

i

)

What  this  says  about  the  scientific  approach  may  not  be  immediately  clear  from  a

symbolic  representation,  but  a  key  aspect  here  is  that  in  science  we  compare  different

hypotheses,  hence  the  introduction  of  the  subscript  i,  to  label  one  hypothesis  among

others. Essentially what this says is that the interesting posterior quantity Pr(Hypothesis

i

|

Data) is only meaningful in comparison with other, competing hypotheses. The likelihood

of a given hypothesis generating the experimental data Pr(Data | Hypothesis

i

) is a measure

of how well the data fits the hypothesis. However, even if one hypothesis seems to fit the

experimental  data  very  well,  our  confidence  in  this  particular  hypothesis  is  naturally

diminished if there is a somewhat different hypothesis that also fits the experimental data

very  well.  Conversely  if  all  the  hypotheses  that  fit  the  data  are  very  similar  then  the

confidence of our answer increases, and we gain an awareness of the width of acceptable

---

solutions:  what  precision  is  meaningful  in  our  hypotheses.  Accordingly,  the  Bayesian

inferential  approach  is  more  objective  than  a  simple  deductive  approach  (where  if

something  fits  well  it  is  assumed  to  be  the  correct  answer),  and  it  has  an  inbuilt

mechanism to quantify the uncertainty associated with a hypothesis.

An  aspect  of  Bayesian  analysis  which  in  some  situations  may  not  seem  particularly

scientific  is  the  quantity  Pr(Hypothesis

i

)  that  represents  the  prior  information  about  the

hypothesis,  in  the  absence  of  any  experimental  evidence.  Indeed  the  ability  of  this

approach  to  work  well  can  often  depend  on  a  scientist’s  ability  to  come  up  with  a  good

estimate of the prior probability. We can always say that we have no prior information to

compare hypotheses in the initial instance, i.e. that the prior is the same for all hypotheses,

in which case our analysis effectively becomes a maximum likelihood approach. However,

it  is  often  possible  to  do  better  by  thinking  about  the  system  under  study,  which  is  a

general  principle  when  doing  mathematical  modelling.  Thinking  of  an  example  about

molecular 3D structure, we can use prior probabilities to say that some conformations are

more  likely  than  others,  considering  things  like  the  length  of  and  the  angle  between

chemical bonds. Effectively we are selecting hypotheses that fit what we generally know

about  molecular  structures,  disregarding  solutions  with  distorted  geometries.  You  could

argue  that  this  is  subjective  and  thus  biased,  to  find  solutions  that  fit  our  expectations.

However, in practice good and useful prior probabilities will generally derive from a well-

founded theory or other experimental observations, e.g. about how long different kinds of

chemical bonds are on average.

It  should  be  noted  that  the  quantity  of  Pr(Data)  is  the  same  for  all  hypotheses  and  so

calculating  its  value  doesn’t  help  in  determining  the  best  hypothesis.  Accordingly  it  is

often  ignored  and  set  to  a  value  of  1.  However,  if  an  accurate  value  of  Pr(Hypothesis

i

 |

Data)  is  sought  then  Pr(Data)  can  be  calculated  by  summing  the  likelihood  over  all

hypotheses: Σ

i

Pr(Data | Hypothesis

i

)Pr(Hypothesis

i

).