What is Linear Regression?

What is Linear Regression?

Regression analysis is the most widely used method to measure, model, and test relationships between random variables.
Linear regression assumes a linear relationship between an explanatory variable X and a dependent variable Y.

Three conditions need to be satisfied:

1. The relationship between Y and X should be linear.
2. The error term must be additive (i.e., the variance of the error term is independent of the observed data).
3. All X variables should be observable (as the model is inappropriate when you have missing data).

Example:

Y = α + βX + ε

where:

• β, commonly called the slope or the regression coefficient, measures the sensitivity of Y to changes in X;
• α, commonly called the intercept, is a constant; and
• ε, commonly called the shock/innovation/error, represents a component in Y that cannot be explained by X.

The mean annual return Y bar over the past 20 years for a specific stock is 11%, while that for the market X bar is 8.4%. The covariance of annual returns for the stock and the market (σ_x,y) and the variance of the market (σ²x) are shown in the following variance-covariance matrix:

( σ²y σ_x,y σ_x,y σ²x ) = (151.22  132.11  132.11  181.40 )

To calculate the regression,

$ \beta \, = \, \frac{COV(X.Y)}{Var\: X}\, = \, \frac{\sigma _{x,y}}{\sigma ^{2}\, x}\, = \, \frac{132.11}{181.40}\, = \, 0.73 $

α = Y bar – βX bar = 0.11 – 0.73(0.084) = 0.049

Why is it important?

These models have proven to be a reliable and scientific means of forecasting the future because it is a well-known statistical process, their properties are well understood, and their models may be trained quickly. The linear regression model is essential to the Capital Asset Pricing Model (CAPM), which determines the relationship between an asset’s expected return and the associated market risk premium.