1157 Words5 Pages

Introduction
Ordinary least-squares (OLS) regression, also called linear regression, is one of the most commonly used modelling techniques, helping us examine the relationship between variables. OLS regression assumes that there is a linear relationship beteen the dependent variable and the independent variable. Basic Features
With a single independent variable, this relationship can be represented as y = β0 + β1x, where β0 is the in- tercept of the model, and β1 is the parameter of the regression or the co- efficient. Our estimation using least squares approach is finding a best- fiiting straight line through the data points with equation yˆ = βˆ0 + βˆ1x, as shown in Figure 1. The reason why we*…show more content…*

Thus, simply adding up all the residuals from the data is not a good appraoch as the negative values and positive ones will cancel out with each other. As a result, we square all the residuals and then add them up to eliminate the influence of cancel- lation. The sum of all the squared residuals is known as the residual sum of squares (RSS), which is a con- venient measure of model-fit for OLS regression. A model that fits the data poorly will deviate significantly from the data and will consequently have a relatively large RSS, while a good- fitting model will not deviate markedly from the data, resulting in a relatively small RSS (a perfectly fitting model will have an RSS of zero, as there is no deviation between the observed and predicted values). Finding Regression Parameters with Gradient Descent Let us define the OLS cost function as: J (w) = 1 X(yˆ(i) y(i))2 2 i=1 Here, yˆ(i) is the ith predicted value, y(i) is the corresponding observed value and m is the number of observations in the data set(note that the term 1/2 is just used for convenience in later derivation). We can represent yˆ(i) = β0 + β1x(i). Hence, we will have the following cost function: yˆ(i) as 1 m J (β0, β1) = X 2 i=1 (β0 + β1x (i) (i) 2 − Now our job is

Thus, simply adding up all the residuals from the data is not a good appraoch as the negative values and positive ones will cancel out with each other. As a result, we square all the residuals and then add them up to eliminate the influence of cancel- lation. The sum of all the squared residuals is known as the residual sum of squares (RSS), which is a con- venient measure of model-fit for OLS regression. A model that fits the data poorly will deviate significantly from the data and will consequently have a relatively large RSS, while a good- fitting model will not deviate markedly from the data, resulting in a relatively small RSS (a perfectly fitting model will have an RSS of zero, as there is no deviation between the observed and predicted values). Finding Regression Parameters with Gradient Descent Let us define the OLS cost function as: J (w) = 1 X(yˆ(i) y(i))2 2 i=1 Here, yˆ(i) is the ith predicted value, y(i) is the corresponding observed value and m is the number of observations in the data set(note that the term 1/2 is just used for convenience in later derivation). We can represent yˆ(i) = β0 + β1x(i). Hence, we will have the following cost function: yˆ(i) as 1 m J (β0, β1) = X 2 i=1 (β0 + β1x (i) (i) 2 − Now our job is

Related

## Statistics - Final Project

1444 Words | 6 Pageswhen it comes to the machines between the different cities. The variables “City” and “Machine” are both

## Kuwait Oil Economy

1475 Words | 6 PagesTerm variable Definition: A quantity, usually represented as a symbol that can take on one of a set of

## The Effect of a Character Education Program on High School Student Achievement

1069 Words | 5 Pagestemptation to assume that if a high correlation exists between for example character education implementation

## The Production of Wheat Essay examples

1173 Words | 5 Pagesrelationship between physical inputs and physical output of a commodity. It is purely a technical relation which

## Homosexuality : The Role Of Religion And Cultural Context Essay

1162 Words | 5 Pageswill be more likely to be opposed to homosexual relations, all else being equal. Religion and views on

## How The Chronic Pain Is A Common Medical Complaint Among Long Term Care

1178 Words | 5 Pagesolder long-term care residents and to examine variables associated with ageist attitude. Study was conducted

## Essay On Longevity

750 Words | 3 PagesIntroduction In this paper, the variables, instruments and analyses of the hypotheses to be used to operationalize

## Money Supply and Inflation Essay

1724 Words | 7 Pages(analysis money supply and its impact on other variable i.e. inflation, interest rate, real GDP and nominal

## Correlation Between Land Use Land Cover And Water Quality Parameters

928 Words | 4 Pagesanalytical methods to determine the relationships between land-use land-cover and water quality parameters

## A Big Role In The World Of Statistics Is Played By What

1162 Words | 5 Pagesknown as variables. For a clearer understanding Siddharth Kalla explains, “In statistics, variables are central