Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

 

---

37 
 
Assessment of fit 
 
Dobson and Barnett (2008) state that one way of assessing the fit of a model is to compare 
it with a model with the maximum number of parameters. The model with the maximum 
number of parameters is called the saturated model and has the same number of parameters 
as covariate patterns (i.e. observations with the same values of all the variables). The 
saturated model tells us no more than the actual data and is often non-informative 
(Faraway, 2006). However, we can use the saturated model to compare prospective 
models. The difference between the log-likelihood for the full model and model under 
consideration gives the likelihood ratio statistic, known as the deviance 
( 
̂
) ( 
̂) 
. 
The deviance for the binomial model is now derived. This follows from Dobson and 
Barnett (2008). From Equation (3.13) the likelihood function is
( ) ∏
( 
)
(
)
(
) (
) 
. 
This in term means the log-likelihood function is 
( ) ∑
( 
)
(
)
(
) (
) 
.
 
(
3.14) 
From this we find the maximum likelihood estimate for 
. Now, differentiating and 
equating to zero we have
(
) 
(
)
which leads to the maximum likelihood estimate 

---

38 
̂
.
Now, the maximum value of the log-likelihood function Equation (3.14) is
( 
̂
) ∑
(
)
(
)
(
) (
) 
.
For any other model with number of parameters less than the number of covariate patterns, 
let 
̂
̂
denote the fitted values. Then, the log-likelihood evaluated at these values is 
( 
̂) ∑
(
̂
)
(
̂
)
(
̂
) (
) 
. 
Therefore, the deviance for the Binomial model is
[ ( 
̂
) ( 
̂)] 
∑
(
)
(
)
(
) (
) ∑
(
̂
)
(
̂
)
(
̂
) (
)
∑ 
(
)
(
̂
)
(
)
(
̂
)
(
)
(
̂
)
∑ 
( (
) (
̂
)) ( 
) (
) ( 
) (
̂
)
∑ 
( 
̂
) ( 
) (
) (
̂
)
∑ 
( 
̂
) ( 
) [ (
̂
)] ( ) 
This deviance has a chi-squared distribution with degrees of freedom equal to the number 
of covariate patterns less the number of parameters. The deviance can, therefore, be used 
in a hypothesis test to assess the fit of a model. However, when the outcome is binary, i.e. 
when 
takes on the values zero or one, this goodness-of-fit measure is no longer useful. 

---

39 
There is also a Hosmer-Lemeshow statistic which tries to overcome the problem of a 
goodness-of-fit statistic for binary data (Hosmer and Lemeshow, 2000). However, its use 
is still questionable.