Let P be the one-step transition probability matrix of a Markov chain that takes value from {0, 1, 2, 3, 4}. We have P = - 11 2011 0 2 0 0 0 1 0 o P3 1,4 。 P0,1 0 x 0 o limn→∞ PA 1,4 o limn→ 4 0 a. Specify the classes of the Markov chain. Classify each state i € {0, 1, 2, 3, 4} into either transient or recurrent state. 0 b. Let f; be the probability that starting at state i, the Markov chain will ever reenter state i. Calculate fi for all i = {0, 1, 2, 3, 4}. P₂ 4,3 4 c. Denote Pm as the (i, j)th entry of Pm where i, je {0, 1, 2, 3, 4}. Is each of the following quantities non-zero? Briefly explain why. 1 0 1 13

Let P be the one-step transition probability matrix of a Markov chain that takes value from {0, 1, 2, 3, 4}. We have P = - 11 2011 0 2 0 0 0 1 0 o P3 1,4 。 P0,1 0 x 0 o limn→∞ PA 1,4 o limn→ 4 0 a. Specify the classes of the Markov chain. Classify each state i € {0, 1, 2, 3, 4} into either transient or recurrent state. 0 b. Let f; be the probability that starting at state i, the Markov chain will ever reenter state i. Calculate fi for all i = {0, 1, 2, 3, 4}. P₂ 4,3 4 c. Denote Pm as the (i, j)th entry of Pm where i, je {0, 1, 2, 3, 4}. Is each of the following quantities non-zero? Briefly explain why. 1 0 1 13

Linear Algebra: A Modern Introduction

4th Edition

ISBN:9781285463247

Author:David Poole

Publisher:David Poole

Chapter3: Matrices

Section3.7: Applications

Problem 9EQ

See similar textbooks