Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

 
 
5.2 Limitations, Recommendations and Further Research 
A limitation of this study was that the same data set was used to obtain prior information. 
The prior information used, therefore, reflects a situation when there is “perfect prior 
information”. Although this is a limitation, in practice there will be experts with much 
experience in credit scoring who would be able to provide very good prior information. A 
way to perhaps solve this problem would be to change the structure of the “old” and “new” 
data. This could be done by removing variables from the “old” data. There would then be 
only prior information for the variables in the “old” data. The usefulness of this reduced 
prior information could be explored.
It is very difficult to obtain credit scoring data from financial institutions. However, if one 
could get hold of two data sets from a financial institution, one from its home economic 
location and one from a new economic location, a more realistic analysis could be done. If 
these data could be obtained, a better insight would be gained into the use of Bayesian 
logistic regression models in practice.
Only one method to estimate the missing values in the data set was used. There are a 
number of methods which are available for the estimation of missing values. Simply 
replacing the missing values by the overall mean for each variable and the EM 
(Expectation-Maximization) algorithm are other possible methods which could be 

---

106 
investigated. This latter method works by substituting values iteratively in conjunction 
with a model. A comparison of these different estimation methods in credit scoring is 
another area for further research.
This study considered normally distributed priors for the Bayesian models with 
informative priors. Priors with other distributions can also be considered, for example, the 
beta distribution. The Laplace prior is another area for future research. For Bayesian 
models with non-informative priors, an improper uniform prior was used in this study. 
There are other possible choices, namely the Jeffreys’ non-informative priors.
There appeared to be a minor issue with the convergence of the MCMC algorithms. From 
the trace plots, there was possibly some significant autocorrelation in the Markov chain. 
Methods to remove this correlation could also be considered - such as thinning. Thinning 
reduces the sample size of the generated Markov chain by only taking every 2nd or 3rd 
observation. The Geweke diagnostic also showed that the generated Markov chain for 
some of the variables had not converged. The analysis could be done again this time using 
a larger burn-in period for the generated Markov chains (to allow for more time for the 
chain to converge). This could result in better parameter estimates for the Bayesian 
models.
This study used a random-walk Metropolis-Hastings algorithm to sample from the 
posterior distributions. There are a number of algorithms available. The independence 
sampler is another method which could be considered. The use of a Gibbs sampler when 
auxiliary variables are used is another interesting model which could be investigated.
There are many different models which can be used to build a credit scoring model. One 
method which has shown some success in a Bayesian framework is Bayesian networks as 
shown by Biçer 
et al. 
(2010). An investigation into these networks for credit scoring would 
also provide interesting further topics of research.