Description

1 Definitions
1. Fig. 1.1.1 shows a Markovs chain with 5 states. Transition from one state to another happens over time. s0 and s4 are absorbing states.
p + q =1 (1.1.1)
1 1
Fig. 1.1.1
2. At time instant n,

P0(n)
P1(n)
p(n) =P2(n) (1.2.1)

P3(n) P4(n)
where P(in) are defined to be the stationary probabilities.
3. Pi| j is defined as the transition probability of going to state i from state j.
4. For a matrix A, let
Ax= λx. (1.4.1)
Then λ is a scalar defined to be the eigenvalue of A and x is the corresponding eigenvector.
2
2 Problems
1. Let
P(0n+1) = P(0n) (2.1.1)
P(1n+1) = pP(0n) + qP2(n) (2.1.2)
P(2n+1) = pP(1n) + qP3(n) (2.1.3)
P(3n+1) = pP(2n) + qP4(n) (2.1.4)
P(4n+1) = P(4n)
Find the matrix P such that p(n+1) =Pp(n)
2. Show that 1 is an eigen value of P.
3. Show that (2.1.5)
P(2n+1) = pP(1n) + qP3(n)
4. If P0 =1, PN =0 and (2.3.1)
Pi = pPi−1 + qPi+1 show that (2.4.1)
i
Pi N ,0≤ i ≤ N −1 (2.4.2)
5. Find the angle between the two lines 2x =3y =−z and 6x =−y =−4z.