Problem 1PP: Consider the Markov chain on {1, 2, 3, 4} with transition matrix... Problem 1E: In Exercises 1-6, consider a Markov chain with state space with {1, 2,, n} and the given transition... Problem 2E: In Exercises 1-6, consider a Markov chain with state space with {1, 2,, n} and the given transition... Problem 3E: In Exercises 1-6, consider a Markov chain with state space with {1, 2,, n} and the given transition... Problem 4E: In Exercises 1-6, consider a Markov chain with state space with {1, 2,, n} and the given transition... Problem 5E: In Exercises 1-6, consider a Markov chain with state space with {1, 2,, n} and the given transition... Problem 6E: In Exercises 1-6, consider a Markov chain with state space with {1, 2,, n} and the given transition... Problem 7E: In Exercises 7-10, consider a simple random walk on the given directed graph. Identify the... Problem 8E: In Exercises 7-10, consider a simple random walk on the given directed graph. Identify the... Problem 9E: In Exercises 7-10, consider a simple random walk on the given directed graph. Identify the... Problem 10E: In Exercises 7-10: consider a simple random walk on the given directed graph. Identify... Problem 11E: Reorder the states in the Markov chain in Exercise 1 to produce a transition matrix in canonical... Problem 12E: Reorder the states in the Markov chain in Exercise 2 to produce a transition matrix in canonical... Problem 13E: Reorder the states in the Markov chain in Exercise 3 to produce a transition matrix in canonical... Problem 14E Problem 15E Problem 16E Problem 17E: Find the transition matrix for the Markov chain in Exercise 9 and reorder the states to produce a... Problem 18E: Find the transition matrix for the Markov chain in Exercise 10 and reorder the states to produce a... Problem 19E: Consider the mouse in the following maze from Section 10.1, Exercise 19. a. Identify the... Problem 20E: Consider the mouse in the following maze from Section 10.1, Exercise 20. a. Identify the... Problem 21E: In Exercises 21-22, mark each statement True or False. Justify each answer. 21. a. If two states i... Problem 22E: In Exercises 21-22, mark each statement True or False. Justify each answer. 22. a. If state i is... Problem 23E: Confirm Theorem 5 for the Markov chain in Exercise 7 by taking powers of the transition matrix (see... Problem 24E Problem 25E: Consider the Markov chain on {1, 2, 3} with transition matrix P=[01/2010101/20] a. Explain why this... Problem 26E: Follow the plan of Exercise 25 to confirm Theorem 5 for the Markov chain with transition matrix... Problem 27E Problem 28E Problem 29E format_list_bulleted