Bayesian Logistic Regression Models for Credit Scoring by Gregg Webster

Continuous state space 
 
It is now assumed we have a stochastic process 
( 
)
with discrete time but a 
continuous state space 
and that all distributions have densities.
For Markov chains with continuous state space the transition probabilities
( 
| 
)
are always zero. Therefore, looking at specific points 
when defining transition probabilities, is not helpful. Thus, subsets
are considered. 
This leads to the following definition: 
 
Definition 3.2 
Let 
( 
)
be a stochastic process with continuous state space 
. If for all 
and all states 
we have

---

53 
( 
| 
) ( 
| 
) 
then we call the stochastic process a Markov chain with continuous state space (Robert and 
Casella, 2004).
We will always assume that we can determine the transition probabilities 
( 
| 
)
using a transition kernel 
by 
( 
| 
) ∫
( 
)
A transition kernel has the following properties 
-
( 
)
for all 
; 
-
∫
( 
)
It can be shown that the two-step transition probability is given by 
( 
| 
) ( 
| 
) 
∫
∫
( 
) ( 
)
Therefore, the two-step transition kernel is
( )
( 
) ∫
( 
) ( 
)
. 
We can generalize this to T-step transitions (multi-step transitions). 
( 
| 
)
∫
∫
∫
( 
) ( 
) ( 
)
. 
Hence, we have the T-step transition kernel 
( )
( 
) ∫
∫
( 
) ( 
) ( 
)
. 
Therefore, we have

---

54 
( 
| 
) ∫
( )
( 
)
. 
The Chapman-Kolmogorov lemma is then also true with a countable state space
( )
( 
) ∫
( )
( 
) 
( )
( 
)
The concept of an irreducible Markov chain (as discussed under discrete state spaces) is 
the same for the continuous state space. Thus, the definitions of recurrent and transient 
Markov chains, communication and aperiodic also apply to the continuous case.
The concept of a stationary distribution for a Markov chain with continuous state space is 
now discussed.
We assume that for a Markov chain 
( 
)
with transition kernel
, the 
distribution of 
has the density 
. Now, if 
( 
)
is the density of 
then 
( 
) ( 
| 
) 
( 
)
( 
) ( 
)
. 
Therefore, for the distribution of 
for any 
( 
) ∫
( 
)
∫
∫
( 
) ( 
)
. 
A density for 
is therefore given by 
( 
) ∫
( 
) ( 
)
. 
We then have the following definition: