HW - 1_ DS 6140 (1)
.pdf
keyboard_arrow_up
School
Northeastern University *
*We aren’t endorsed by this school
Course
6410
Subject
Industrial Engineering
Date
Feb 20, 2024
Type
Pages
3
Uploaded by ProfRavenMaster1034
DS 6140 – MACHINE LEARNING HOMEWORK – 1 Due – Oct 2
nd
6 pm PDT Gauss - pelliarmus !
(20 pts) 1.
Suppose you are examining a set of data to understand the relationship between multiple independent variables (X1, X2, X3) and a dependent variable (Y). For your analysis, you decide to deploy a statistical model that assumes a Gaussian distribution for the errors. (a)
In the context of your analysis, describe the importance of understanding the mean and variance of the Gaussian distribution. How do these parameters influence your regression model, and what insights can they provide about the relationship between your variables? (b)
Given the following sample data for the independent variables and dependent variable: Compute the multivariate means for the independent variables and the variance for the dependent variable Y. Based on these calculations, discuss any preliminary insights or patterns you might discern from this data and any implications for further statistical analyses. How might these values shape your understanding of the relationships in your dataset and any real-world applications of your findings? Welcome to the Matrix- Neo (30 pts)
2.
Consider a matrix M representing data from a multivariate experiment. The matrix M is given by:
(a) Calculate the L2 norm (Euclidean norm) of the matrix M.
(b) Determine if the matrix M is orthogonal. If not, provide insights on how you could approximate M to an orthogonal matrix.
(c) Compute the eigenvalues of the matrix M. (d) Perform the Eigen decomposition of M and express it in terms of its eigenvalues and eigenvectors.
(e) Discuss the effects of the eigenvalues you found in part (c) on the stability and behavior of systems described by the matrix M.
The Curious Case of Overfitting Oscar and Linear Larry! (30 pts) 3.
Given a dataset, we choose two models: A linear regression model A 10th-degree polynomial regression mode Assume that the true underlying model is a 3rd-degree polynomial: (a)
Which of the two models, f^1(x) or f
^2(
x
), is likely to have a higher bias when trying to approximate the true model f
3(
X
)? Justify your answer using the concept of flexibility. (b)
As we increase the flexibility of a model, the variance typically goes up. Which of the two models, f
^1(
x
) or f
^2(
x
), is expected to have a higher variance? Justify your answer. (c)
Suppose you plotted the expected test Mean Squared Error (MSE) against the model complexity for both f
^1(
x
) and f
^2(
x
). Describe the shape you would expect to see, specifically noting the regions of high bias and high variance for each model. (d)
Using the equation provided: Estimate the error components (bias and variance) for both models. Which model would you expect to have a smaller expected test error, given that the true model is f
3(
X
)?
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help