4512. Every Saturday night a man plays poker at his home with the same group of friends. If heprovides refreshments for the group (at an expected cost of $30) on any given Saturday night,the group will begin the following Saturday night in a good mood with probability and in a badmood with probability. However, if he fails to provide refreshments, the group will begin thefollowing Saturday night in a good mood with probability and in a bad mood with probability,regardless of their mood this Saturday. Furthermore, if the group begins the night in a bad moodand then he fails to provide refreshments, the group will gang up on him so that he incurs1expected poker losses of $70. Under other circumstances, he averages no gain or loss on hispoker play. The man wishes to find the policy regarding when to provide refreshments that willminimize his (long-run) expected average cost per week.(a) Formulate this problem as a Markov decision process by identifying the states and decisionsand then finding the Cik.(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrixand write an expression for the (long-run) expected average cost per period in terms of theunknown steady-state probabilities (To,₁,TM).(c) Find these steady-state probabilities (To, ₁,,TM) for each policy. Then evaluate theexpression obtained in part (b) to find the optimal policy by exhaustive enumeration.

Answered: Image /qna-images/answer/02c5d15d-faf5-4466-b8bc-52fa0f2b0724.jpg

Answered: (a) Formulate this problem as a Markov…

A First Course in Probability (10th Edition)

10th Edition

ISBN:9780134753119

Author:Sheldon Ross

Publisher:Sheldon Ross

Chapter1: Combinatorial Analysis

Section: Chapter Questions

Problem 1.1P: a. How many different 7-place license plates are possible if the first 2 places are for letters and...

See similar textbooks

Similar questions

3. suppose that the manufacturer keeps a spare machine that only is used when the primary machine is being repaired. During a repair day, the spare machine has a probability of 0.1 of breaking down, in which case it is repaired the next day. Denote the state of the system by (x, y), where x and y, respectively, take on the values 1 or 0 depending upon whether the primary machine (x) and the spare machine (y) are operational (value of 1) or not operational (value of 0) at the end of the day. [Hint: Note that (0, 0) is not a possible state.] (a) Construct the (one-step) transition matrix for this Markov chain. (b) Find the expected recurrence time for the state (1, 0).
Which of the following can be accurately described by a “deterministic” model, that is, a model which does not require any concept of probability? (a) The position of a small particle in space (b) The velocity of an object dropped from the leaning tower of Pisa (c) The lifetime of a heavy smoker (d) The value of a stock which was purchased for $20 one month ago (e) The number of servers at a large data center which crash on a given day
(i) If volume is high this week, then next week it will be high with a probability of 0.6 and low with a probability of 0.4. (ii) If volume is low this week then it will be high next week with a probability of 0.1. Assume that state 1 is high volume and that state 2 is low volume (1) Find the transition matrix for this Markov process. (2) If the volume this week is high, what is the probability that the volume will be high two weeks from now? (3) What is the probability that volume will be high for three consecutive weeks?
We will use Markov chain to model weather XYZ city. According to the city’s meteorologist, every day in XYZ is either sunny, cloudy or rainy. The meteorologist have informed us that the city never has two consecutive sunny days. If it is sunny one day, then it is equally likely to be either cloudy or rainy the next day. If it is rainy or cloudy one day, then there is one chance in two that it will be the same the next possibilities. In the long run, what proportion of days are cloudy, sunny and rainy? Show the transition matrix.
Please show steps and solutions to solve problem
In an office complex of 1000 employees, on any given day some are at work and the rest are absent. It is known that if an employee is at work today, there is an 85% chance that she will be at work tomorrow, and if the employee is absent today, there is a 70% chance that she will be absent tomorrow. Suppose that today there are 740 employees at work. (A graphing calculator is recommended.) (a) Find the transition matrix A for this scenario. at work absent A = (b) Predict the number that will be at work five days from now. (Round your answer to the nearest integer.) employees (c) Find the steady-state vector x. (Round your answer to two decimal places.)
A market demand has 3 possible states namely GOOD, NORMAL and BAD for each period. At each period there are 2 possible decisions for the manager as do nothing/ make promotion. The transition matrix of states regarding for possible decisions are given as below Pof do nothing GOOD NORMAL BAD GOOD 0.4 0.2 04 0.3 0.2 0.5 NORMAL 0.5 BAD 0.1 0.4 Pof make promotion GOOD NORMAL BAD 0.4 0.4 0.5 GOOD 0.2 NORMAL 04 0.1 BAD 0.3 0.4 0.3 It is known that the income for each period when the state in GOOD, NORMAL and BAD are 10000, 7000, 2000. Cost for do nothing is 0, and make promotion is 3000. a) Given at period 0 the state is NORMAL, estimate expected beneft obtain two periods when the sequence of decisions is do nothing, do nothing b) Given at period 0 the state is NORMAL, estimate expected beneft obtain two periods when the sequence of decisions is make promotion, make promotion
Determine whether the statement below is true or false. Justify the answer. If (x) is a Markov chain, then X₁+1 must depend only on the transition matrix and xn- Choose the correct answer below. O A. The statement is false because x, depends on X₁+1 and the transition matrix. B. The statement is true because it is part of the definition of a Markov chain. C. The statement is false because X₁ +1 can also depend on X-1 D. The statement is false because X₁ + 1 can also depend on any previous entry in the chain.
3. (a) A Markov process with 2 states is used to model the weather in a certain town. State 1 corresponds to a suny day. State 2 corresponds to a rainy day. The transition matrix for this Markov process is [0.7 0.4] 0.3 0.6 %3D (i) If today is rainy, what is the probability that tomorrow will be sunny? (ii) Find the steady state probability vector. (iii) In the long run, how many days a week are sunny?
Suppose a Markov Chain has transition matrix % 0 % % If the system starts in state 3, what is the probability that it goes to state 2 on the next observation, and then goes to state 4 on the following observation? (A) 24 (B) 1 (C) 4 (D) ½24 (E) % (F) %32 (G) 0 (H) %
A Markov system with two states satisfies the following rule. If you are in state 1 then of the time you change to state 2. If you are in state 2 then of the time you remain in state 2. At time t = 0, there are 100 people in state 1 and no people in the other state. state 1 Write the transition matrix for this system using the state vector v = state 2 T = Write the state vector for time t = 0. Vo Compute the state vectors for time t = 1 and t = 2. Vị = V2

Recommended textbooks for you

A First Course in Probability (10th Edition)

Probability

ISBN:

9780134753119

Author:

Sheldon Ross

Publisher:

PEARSON

A First Course in Probability

Probability

ISBN:

9780321794772

Author:

Sheldon Ross

Publisher:

PEARSON

A First Course in Probability (10th Edition)

Probability

ISBN:

9780134753119

Author:

Sheldon Ross

Publisher:

PEARSON

A First Course in Probability

Probability

ISBN:

9780321794772

Author:

Sheldon Ross

Publisher:

PEARSON

SEE MORE TEXTBOOKS