Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to characterize this probability. Next we give a more mathematical description of the problem. Now define your fortune at time n ≥ 1 by - Xn = Xn-1+ I{Y=1} = I{Yn=0} with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk = L}. Finally, let Q(x0) = P{XT = . L}. Show that X is a Markov chain and obtain its transition probability matrix. Is it irreducible? Use conditional expectations to prove that Q(x)=pQ(x+1) + (1 − p)Q(x − 1), - x = 1, L-1, (1) with Q(0) 0 and Q(L) = 1. = d) Solve the difference equation in (1) analytically. Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1. Hint: Use the same form of the difference equation and analytical solution as in (d) but use different initial conditions. f) Show that all intermediate states are transient and that both absorbing states are recurrent. Use (f) to show that the gambler's fortune converges almost surely to a binary random variable and characterize the limit.
Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to characterize this probability. Next we give a more mathematical description of the problem. Now define your fortune at time n ≥ 1 by - Xn = Xn-1+ I{Y=1} = I{Yn=0} with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk = L}. Finally, let Q(x0) = P{XT = . L}. Show that X is a Markov chain and obtain its transition probability matrix. Is it irreducible? Use conditional expectations to prove that Q(x)=pQ(x+1) + (1 − p)Q(x − 1), - x = 1, L-1, (1) with Q(0) 0 and Q(L) = 1. = d) Solve the difference equation in (1) analytically. Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1. Hint: Use the same form of the difference equation and analytical solution as in (d) but use different initial conditions. f) Show that all intermediate states are transient and that both absorbing states are recurrent. Use (f) to show that the gambler's fortune converges almost surely to a binary random variable and characterize the limit.
Algebra & Trigonometry with Analytic Geometry
13th Edition
ISBN:9781133382119
Author:Swokowski
Publisher:Swokowski
Chapter5: Inverse, Exponential, And Logarithmic Functions
Section5.6: Exponential And Logarithmic Equations
Problem 64E
Related questions
Question
Transcribed Image Text:Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar
each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars
or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to
characterize this probability. Next we give a more mathematical description of the problem.
Now define your fortune at time n ≥ 1 by
-
Xn = Xn-1+ I{Y=1} = I{Yn=0}
with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk =
L}. Finally, let Q(x0) = P{XT = .
L}.
Show that X is a Markov chain and obtain its transition probability matrix.
Is it irreducible?
Use conditional expectations to prove that
Q(x)=pQ(x+1) + (1 − p)Q(x − 1),
-
x = 1, L-1,
(1)
with Q(0) 0 and Q(L) = 1.
=
d) Solve the difference equation in (1) analytically.
Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1.
Hint: Use the same form of the difference equation and analytical solution as in (d) but use
different initial conditions.
f) Show that all intermediate states are transient and that both absorbing states are recurrent.
Use (f) to show that the gambler's fortune converges almost surely to a binary random
variable and characterize the limit.
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
This is a popular solution!
Trending now
This is a popular solution!
Step by step
Solved in 2 steps
Recommended textbooks for you
Algebra & Trigonometry with Analytic Geometry
Algebra
ISBN:
9781133382119
Author:
Swokowski
Publisher:
Cengage
Algebra and Trigonometry (MindTap Course List)
Algebra
ISBN:
9781305071742
Author:
James Stewart, Lothar Redlin, Saleem Watson
Publisher:
Cengage Learning
Trigonometry (MindTap Course List)
Trigonometry
ISBN:
9781337278461
Author:
Ron Larson
Publisher:
Cengage Learning
Algebra & Trigonometry with Analytic Geometry
Algebra
ISBN:
9781133382119
Author:
Swokowski
Publisher:
Cengage
Algebra and Trigonometry (MindTap Course List)
Algebra
ISBN:
9781305071742
Author:
James Stewart, Lothar Redlin, Saleem Watson
Publisher:
Cengage Learning
Trigonometry (MindTap Course List)
Trigonometry
ISBN:
9781337278461
Author:
Ron Larson
Publisher:
Cengage Learning