Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

 
Bayesian updating 
Equation (3.3) can be seen as a way of updating information. Our prior knowledge is 
updated with data. Then, as new data become available the posterior distribution is updated 
using Bayes’ theorem. Greenberg (2008) explains this process: let 
be the parameter (or a 
vector of parameters) of interest and 
be the first set of data available. We have, 
( | 
) ( 
| ) ( ) 
(3.4) 
Now, suppose a new data set 
is obtained and we want the posterior distribution given 
all the available data. Thus, 
( | 
) ( 
| ) ( ) ( 
| 
) ( 
| ) ( )
using Equation (3.1) 
( 
| 
) ( | 
)
because 
( | 
) ( 
| ) ( )
from Equation (3.4). 
If the data sets 
and 
are independent 
( 
| 
)
simplifies to 
( 
| )
. We, therefore, 
obtain 
( | 
) ( 
| ) ( | 
) 
(3.5) 

---

25 
From Equation (3.5) we can see that the posterior distribution in Equation (3.4) is now the 
prior distribution in Equation (3.5). Ntzoufras (2009) shows how Equation (3.5) can be 
generalized for a number of different data sets 
( | 
) ( 
| ) ( 
| ) ( ) 
∏
( 
| ) ( )
. 
Thus, as new information becomes available, the posterior distribution becomes the prior 
distribution for the next experiment.
 
Large samples 
It is important to examine how the posterior distribution behaves in large samples. When 
there are independent trials, the likelihood function is 
( | ) ∏
( 
| )
∏
( | 
)
. The log-likelihood function is then 
( | ) ( | ) 
∑ ( | 
)
̅( | )
where 
̅( | ) (
) ∑ ( | 
)
is the mean log-likelihood contribution (Greenberg, 2008). 
The posterior distribution can now be written as 
( | ) ( ) ( | ) 
( ) ( ̅( | )) 
(3.6) 
Now, from Equation (3.6), we see that the posterior distribution is proportional to the 
product of the prior distribution and an exponential term raised to the power 
times a 
number. Thus, for large 
, the exponential term dominates 
( )
which does not depend on 
. Therefore, the larger the sample size, the less role the prior distribution will play in the 
posterior distribution (Greenberg, 2008).