Implement a passive learning agent in a simple environment, such as the 4 x 3 world. For the case of an initially unknown environment model, compare the learning performance of the direct utility estimation, TD, and ADP algorithms. Do the comparison for the optimal policy and for several random policies. For which do the utility estimates converge faster! What happens when the size of the environment is increased? (Try environments with and without obstacles.)

Implement a passive learning agent in a simple environment, such as the 4 x 3 world. For the case of an initially unknown environment model, compare the learning performance of the direct utility estimation, TD, and ADP algorithms. Do the comparison for the optimal policy and for several random policies. For which do the utility estimates converge faster! What happens when the size of the environment is increased? (Try environments with and without obstacles.)

Database System Concepts

7th Edition

ISBN:9780078022159

Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan

Chapter1: Introduction

Section: Chapter Questions

Problem 1PE

See similar textbooks

Similar questions

We are collecting data for estimating the transition function and the reward function. What is the type of this reinforcement learning algorithm? Select one answer Model-based learning Model-free learning Q-Learning Temporal Difference
Explain why you would choose one particular machine learning model over another, giving specific instances. Two distinct grouping strategies exist: K-Nearest Neighbor (a), Going Backwards (c), and Learning More (d)
In deep learning, is each feature map in a convolutional neural network extracted from input by a filter by a) using an affine transformation (i.e. biased convolution), a nonlinear activation and a pooling function? or b) by using an affine transformation (i.e. biased linear combination) and a pooling function?
A model is used to obtain a prediction by accepting features as inputs and returning a prediction in the is.... subset of machine learning. The following are some new cutting-edge models that have become feasible:
Write the updating rule at iteration k+1 for momentum gradient descent learning with a momentum value of 0.9. Explain how this method improves convergence properties of gradient descent learning.
Reinforcement learning can be a useful tool for (choose the closest alternative): Select one: a. For feature learning, where the underlying structure of data is discovered. b. For autonomous robots. c. For fraud detection.
The learning problem is to find the unknown (functional) relationship hy between objects x X and targets y y based solely on a sample z = (x, y) = ((x1, Y1), ..., (xms Ym)) = (X × y)m of size meN drawn iid from an unknown distribution Pxy. If the output space Y contains a finite number Y of elements then the task is called a classification learning problem.
In reinforcement learning, we have to predict an action or value of a state/action, this is like a supervised learning task. What makes reinforcement learning more difficult than classification? Select one: a. It is hard to get samples. b. The supervision is delayed c. There is no supervision in any form. d. It is hard to make a state.
Consider a fully-connected artificial neural network with one hidden layer, i.e., a multilayer perceptron (MLP), which has 5 inputs, 3 neurons in the hidden layer, and 1 output neuron. The relation between the output y and the inputs x = [x1, . . . , x5] is given by y(x) = f (w, φ(x)), where φ(x) = [φ1(x), φ2(x), φ3(x)] 1. Draw the diagram that shows the inputs, nuerons, connections, correspond-ing weight parameters, and activation functions. 2. Explain the relation y(x) = f (w, φ(x)): write the explicit relation, explainthe role of functions f and φ(x), and state examples of functions.
You have a data set which contains Ethno, HbA1c and SBP, indicators in blood. Ethno is a categorical variable and HbA1c and SBP are continuous. They are not evenly distributed. Explain which one of the supervised learning algorithms and why they would be most appropriate for predicting the response Ethno using HbA1c and SBP as the feature variables. The learning algorithms are: Linear Discriminant Analysis, Quadratic Discriminant Analysis and the Naïve Bayes Classifier
Consider the decision model/game theoretic model from Chapter 19 for the spread of a new behavior through a social network. Suppose we have the social network depicted in Figure 2; suppose that each node starts with the behavior B, and each node has a threshold of q = 2/5 for switching to behavior A. Now, let c and d form a two-node set S of initial adopters of behavior A. If other nodes follow the threshold rule for choosing behaviors, which nodes will eventually switch to A? Give a brief (1-2 sentence) explanation for your answer. Find a cluster of density greater than 1 − q = 3/5 in the part of the graph outside S that blocks behavior A from spreading to all nodes, starting from S, at threshold q. Give a brief (1-2 sentence) explanation for your answer. Suppose you were allowed to add a single edge to the given network, connecting one of nodes c or d to any one node that it is not currently connected to. Could you do this in such a way that now behavior A, starting from S and…
An input image to a Convolutional Neural Network has the size of 100 by 100 and the filter (kernel) has the size of 15 by 15. If we use "same" convolution method, with the stride size is 1, the padding size becomes ____?______ . Fill in the blank.