Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

3.3.5 Logistic regression 
Ntzoufras (2009) explains that data encountered with a binary response are often modelled 
with logistic regression. Logistic regression is a special case of the Generalized Linear 
Models (GLMs). For credit scoring data, a response 
represents a default or “bad” 
score and a response 
represents no default or “good” score. Logistic regression 
makes use of the canonical link function, 
(
)
. The logistic regression model is given 
below 
( 
) (
)
∑
( )
for 
. 
is the 
element in the 
i
th row and 
j
th column of the model matrix 
. 
From this, the probability of default is given by 
( 
∑
)
( 
∑
)
.
Other link parameters are also possible to model binary response data, for example the 
probit and clog-log links.
The likelihood for the logistic regression model is 

---

36 
( | ) ∏
(
)
(3.13)
∏
(
( 
∑
)
( 
∑
)
)
(
( 
∑
)
(
∑
)
)
. 
Estimation of the parameters for logistic regression can be done using the IRWLS 
procedure.
Parameter interpretation 
 
The parameters in logistic regression have an interpretation in terms of odds and odds 
ratios. Odds is defined as the relative probability of success (
) compared to the 
probability of failure (
)
when the data is binomial (Ntzoufras, 2009). Thus,
and the logistic regression model can be rewritten as
(
)
, 
(
)
∑
( )
. 
Odds provides a number to multiply the probability of failure by in order to calculate the 
probability of success. 
can be interpreted as follows: a unit increase in 
with all the 
other 
’s held fixed increases the log-odds of success by 
or increases the odds of 
success by 
. This interpretation is a major advantage of logistic regression as no such 
simple interpretation exists for other link functions such as the probit.
In credit scoring, a success corresponds to a default or bad applicant. Thus, the log-odds of 
success is the log-odds of default in the context of credit scoring. Therefore, 
can be 
interpreted as follows: a unit increase in 
with all the other 
’s held fixed increases the 
log-odds of default by 
or increases the odds of default by 
. A positive value for 
thus increases the odds of default as 
increases, while a negative value for 
decreases 
the odds of default as 
increases.