Consider a Markov chain with two possible states, S = {0, 1}. In particular, suppose that the transition matrix is given by Show that ?=[1−∝ ∝] ? 1−? ? 1 ? ∝ (1−∝−?)? ∝ −∝ ?=∝+?[?∝]+ ∝+? [−? ?]

Explain how you can determine the steady state matrix X of an absorbing Markov chain by inspection.

12. Robots have been programmed to traverse the maze shown in Figure 3.28 and at each junction randomly choose which way to go. Figure 3.28 (a) Construct the transition matrix for the Markov chain that models this situation. (b) Suppose we start with 15 robots at each junction. Find the steady state distribution of robots. (Assume that it takes each robot the same amount of time to travel between two adjacent junctions.)

In Exercises 1-4, let P=[0.50.30.50.7] be the transition matrix for a Markov chain with two states. Let X0=[0.50.5]be the initial state vector for the population. What proportion of the state 2 population will be in state 2 after two steps?