Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

 
3.3.6 Bayesian logistic regression 
 
Bayesian inference for the logistic regression model requires priors on the model 
parameters. Wilhelmsen 
et al
. (2009) and Ziemba (2005) both use normally distributed 
priors for the model parameters and represented as follows 
( 
) ( 
)
.
(3.16) 
The posterior distribution is proportional to the product of the prior distribution and 
likelihood, 
( | ) ( | ) ( )
. Therefore, from Equations (3.13) and (3.16), we have 
( | ) ∏
(
)
∏
√
( 
( 
)
)
(3.17) 
or 
( | )
∏
(
)
∏
√
( 
( )
)
∫
∏
(
)
∏
√
( 
( )
)
(3.18) 
when the normalising constant is included.
The form of this posterior distribution, Equation (3.17), suggests that the prior does not 
belong to a conjugate family. There is in fact no conjugate prior for the Bayesian logistic 
regression model. The normalising constant, the integral in the denominator (Equation 
(3.18)) cannot be calculated explicitly. In this situation simulation methods need to be used 
in order to obtain the posterior distributions of the parameters. Markov Chain Monte Carlo 
(MCMC) methods are used where a Markov chain is generated with a stationary 
distribution equal to the posterior distribution of the vector 
β
. 

---

42 
3.4 Monte Carlo Methods
Simulation has greatly improved on the scope of Bayesian inference. Markov Chain Monte 
Carlo (MCMC) methods allow for sampling from a non-standard distribution. Therefore, 
Bayesian inference can be done in a wide range of posterior distribution forms, for 
example Equation (3.18). The idea is to generate a Markov chain whose limiting 
(stationary) distribution is equal to the posterior distribution. This section will describe 
simulation techniques, provide an introduction to Markov chains and then explain the role 
and purpose of Markov Chain Monte Carlo.