Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

3.3.2 Maximum likelihood estimation 
A GLM has the basic structure 
( 
)
where 
( 
)
, 
( 
) 
and 
( 
) 
indicates an exponential family distribution. Since the 
are mutually independent, 
the likelihood of 
is 
( ) ∏
( 
)
Thus the log-likelihood of 
β
is given by 
( ) ∑
( 
)
∑ 
( 
)
( )
(
)
where 
is assumed to be the same for all 
i
. For practical purposes, it is reasonable to 
assume that 
( )
, where 
w
i
is a constant. Therefore, 
( ) ∑ 
( 
)
(
)

---

30 
Differentiating with respect to β
j
gives 
∑
( 
)
Using the chain rule,
and Equation (2.2), we have 
( 
)
( 
)
Hence,
( 
)
( 
)
, which leads to 
∑
[
( 
)
( 
)
( 
)
] 
∑
[
( 
)
]
Now, from Equation (2.3) and using the assumption 
( )
, we obtain
( 
)
( 
)
. Hence 
∑
( 
)
which implies that the equations to solve for 
are given by 
∑
( 
)
 
These are the equations that need to be solved for non-linear weighted least squares, if the 
weights 
( 
)
are known in advance and are independent of 
. In this case the least 
squares objective is, therefore 
∑
( 
)
( 
)
(3.12) 

---

31 
where, 
depends non-linearly on 
 
but the weights, 
( 
)
are fixed (Wood, 2006).
An iterative procedure is needed to solve the Equations (3.12). Let 
̂
denote the 
estimated parameter vector at the 
iteration. Also let 
 
be the vector with elements 
̂
and 
be the vector with elements 
( 
)
, where 
( )
is the 
inverse function of the link function. Define a diagonal matrix