2.1 A Markov chain has transition matrix 1 2 1(0.1 0.3 0.6 P = 20 3 0.3 0.2 0.5 3 0.4 0.6 with initial distribution a = (0.2,0.3,0.5). Find the following: (a) P(X,= 3|X6 = 2) (b) P(X9 = 2|X, = 2, X5 = 1,X7 = 3) %3D %3D (c) P(X, = 3IX, = 1)

a)

Please do the following question with handwritten working out

E5.2. Consider a Markov chain with state space S = {1,2,3, 4, 5, 6} and transition matrix 1 0 0 0 0 0 0 0 0 P = 0 0 0 1 0 0 0 0 0 0 0 1 0 (a) Draw the transition graph of this chain. (b) Show that state 1 is absorbing and conclude that it is positive re- current. (c) Show that states 2 and 3 are transient. (d) Verify that states 4, 5 and 6 are positive recurrent.

C)

For each of the following transition matrices, do the following: (1) Determine whether the Markov chain is irreducible. (2) Find a stationary distribution 7; is the stationary distribution unique? (3) Give the period of each state. (4) Without using any software package, find P200 approximately. (a) (1/4 1/4 1/2 P = | 1/3 2/3 1/2 1/2, (Ъ) (0.2 0.8 0.5 0.5 P = 0.3 0.7 1