In each step someone randomly selects precisely one number from sequence {1; 2; 3; 4}. The chosen number can be selected again in any of the subsequent steps. The current state of the Markov chain is the minimum of all the numbers selected up to and including the current step. 1. Write the initial distribution and the transition probability matrix for the Markov chain; 2. Draw the graph of the Markov chain; 3. Calculate the probability that after 2 steps the system will be in a state with a number greater than 2.

In each step someone randomly selects precisely one number from sequence {1; 2; 3; 4}. The chosen number can be selected again in any of the subsequent steps. The current state of the Markov chain is the minimum of all the numbers selected up to and including the current step. 1. Write the initial distribution and the transition probability matrix for the Markov chain; 2. Draw the graph of the Markov chain; 3. Calculate the probability that after 2 steps the system will be in a state with a number greater than 2.

Linear Algebra: A Modern Introduction

4th Edition

ISBN:9781285463247

Author:David Poole

Publisher:David Poole

Chapter3: Matrices

Section3.7: Applications

Problem 12EQ: 12. Robots have been programmed to traverse the maze shown in Figure 3.28 and at each junction...

See similar textbooks