Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

 
Independence sampler 
If the proposal 
( )
does not depend on 
, that is 
( ) ( )
for all 
then the 
acceptance probability is 
( ) (
( ) ( )
( ) ( )
)
. 
The independence sampler is very similar to the accept-reject method in Section 3.4.1. 
Like the accept-reject method, it is important that the proposal kernel, 
is close to the 
target 
to allow for efficient simulation. However, the independence sampler produces 
dependent samples. Also, if there is a constant 
such that 
( ) ( )
, then the 
expected acceptance rate is at least 
when the Markov chain is stationary. The proof is 
as follows: 
( ( 
)) ( (
( ) ( 
)
( 
) ( )
)) ∫ ∫ (
( ) ( )
( ) ( )
) ( ) ( )
∫ ∫
( ) ( )
( )
( ) ( )
( ) ( )
∫ ∫
( ) ( )
( ) ( )
( ) ( )
( )
( ) ( )
( ) ( )
∫ ∫
( ) ( )
( )
( ) ( )
( ) ( )
∫ ∫
( ) ( )
( )
( ) ( )
( ) ( )
∫ ∫
( ) ( )
( ) 
( ) ( )
( ) ( )
∫ ∫
( ) ( )
( )
( ) ( )
( ) ( )
∫ ∫
( ) ( )
( )
( )
( ) 
( )
( )

---

60 
If
( )
( )
( )
the last integral is
( ( ) ( )) for 
U
, 
V
independent with distribution 
p
(
x
).
Therefore, it is as likely that 
( ) ( )
as that 
( ) ( )
if 
U
, 
V
are 
independent and identically distributed. Thus,
( ( ))
 
Random walk sampler
 
The other common choice is to use the current simulated value to generate the next value. 
In that way, the neighbourhood of the current value of the Markov chain is explored. A 
proposal kernel which allows this is the symmetrical kernel 
( ) ( )
. This leads 
to acceptance probability 
( ) (
( )
( )
) 
A proposed value 
is then accepted with probability one if 
( ) ( )
. Thus, 
points 
that are more likely (according to 
) than the previous 
will always be accepted. 
However, we also accept points which are less likely with a certain probability. Thus, 
making use of the previous 
explores 
in a more local way.