MIE1626 Midterm Practice Problems with Soln

Practice Problems MIE1626 Page 1 of 6 QUESTION 1 [2 marks]: In the expression Sales ≈ f(TV, Radio, Newspaper), "Sales" is the: A) Response B) Training Data C) Independent Variable D) Feature QUESTION 2 [2 marks]: In a predictive modeling project using regression, you fit a linear model to your data set. Which of the following is most likely true if you fit a quadratic model to the data set? A) Using the Quadratic Model will decrease your Irreducible Error. B) Using the Quadratic Model will decrease the Bias of your model. C) Using the Quadratic Model will decrease the Variance of your model D) Using the Quadratic Model will decrease your Reducible Error QUESTION 3 [2 marks]: One way of carrying out the bootstrap is to average equally over all possible bootstrap samples from the original data set (where two bootstrap data sets are different if they have the same observations but in different order). Unlike the usual implementation of the bootstrap, this method has the advantage of not introducing extra noise due to resampling randomly. To carry out this implementation on a data set with n data points, how many bootstrap data sets would we need to average over? A) 2 𝑛 B) 𝑛 2 C) 𝑛 𝑛 D) 𝑛! QUESTION 4 [2 marks]: Which of the following statements is more accurate about classification methods: Logistic Regression, Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), and naive Bayes? A) Logistic regression is not a suitable method when the classes are well-separated. B) LDA is useful when n is large and for problems with more than 2 classes. C) Assuming Gaussian distributions in each class, QDA is less flexible than naive Bayes. D) Naive bayes is most useful when the number of features and samples are roughly the same.

Practice Problems MIE1626 Page 2 of 6 Answers for Q1-4 1 2 3 4 A B C A QUESTION 5 [18 marks]: For predicting p, the probability of credit default, you have used a logistic regression model with variables X 1 = credit score and X 2 = credit card balance. Using historical data with class labels, you have fitted the model Logit(p) ≈ 𝛽 ̂ 0 + 𝛽 ̂ 1 ? 1 + 𝛽 ̂ 2 ? 2 and obtained the estimated coefficients 𝛽 0 ̂ = −50 , 𝛽 1 ̂ = −1 , and 𝛽 2 ̂ = 0.2 . Part (a). [1 mark]: Explain in plain English what the estimated intercept means and provide a numerical example for its role in the model. Answer: the intercept determines the prediction for a sample where X 1 and X 2 are both 0. Accordingly p= e^(-50)/(1+e^(-50))= 1.9e-22 which is a very small number indicating the very small estimated probability of default for a person with a credit score of 0 and a balance of 0. So, in plain English, our model predicts that a person with a credit score of 0 and a balance of 0 is very unlikely to have a credit default. Part (b). [2 marks]: Explain what 𝛽 2 ̂ means and provide a numerical example for how it impacts logit(p). Answer: 𝛽 2 ̂ is the estimated coefficient for credit card balance in the logistic regression model. A one unit increase in balance results in an increase of size 𝛽 2 ̂ = 0.2 in the logit(p) if the other variable (credit score) remains unchanged. Logit(p) is the logarithm of the odds p/(1-p) where p is the probability of default. Logit(p)=Ln (p/(1-p)) = 𝛽 ̂ 0 + 𝛽 ̂ 1 ? 1 + 𝛽 ̂ 2 ? 2 Part (c). [1 mark]: Provide a numerical example for how 𝛽 2 ̂ impacts the odds of default. Odds = p/(1-p) = 𝑒 (𝛽 ̂ 0 +𝛽 ̂ 1 𝑥 1 +𝛽 ̂ 2 𝑥 2 )