Linear Regression
For a given dataset from a real time system or a process, the data is non-linear when plotted on a graph as shown below. For analysis, five data points are used. The data represents 13k, 5k, .. etc. factored as 13, 5, respectively, from a financial system skewed for illustration.
 
  Plot of data


Mathematical Background
We know the values of x and y.  An equation has to be derived for the set of values.

We are all familiar with the equation y = m x + c

m is known as the slope and 
c as the intercept

We first find the average value of x and y.

For i in 1..n, we have 

AVG(x) = Σ xi/n  = (1+2+3+4+5)/5 = 15/5 = 3 
AVG(y) = Σ yi/n  = (13+5+6+6+4)/5 = 34/5 = 6.8

m = Σ (xi - AVG(x)) * (yi - AVG(y)) / Σ ((xi - AVG(x))2)

c = AVG(y) - m * AVG(x)

Start with the numerator (NR) of m: Σ(xi - AVG(x)) * (yi - AVG(y))

NR = (1-3)(13-6.8)+(2-3)(5-6.8)+(3-3)(6-6.8)+(4-3)(6-6.8)+(5-3)(4-6.8)
   = (-2)(6.2) + (-1)(-1.8) + (0)(-0.8) + (1)(-0.8) + (2)(-2.8)
   = (-12.4) + (1.8) + (0) + (-0.8) + (-5.6)
   = -17

The numerator also represents the Covariance(x,y)

Denominator (DR) of m: Σ (xi - AVG(x))2

DR = (1-3)2 + (2-3)2 + (3-3)2 + (4-3)2 + (5-3)2
  = 4 + 1 + 0 + 1 + 4
  = 10

 m = -17/10 = -1.7

 c = 6.8 - (-1.7)(3)
   = 11.9

 The equation y = m x + c becomes

 y = -1.7 x + 11.9
   = 11.9 - 1.7x

 RMSE = √ (ypi - y)2/n

 (ypi - y) = (13-10.2)2 + (8.5-5)2 + (6.8-6)2 + (5.1-6)2 + (3.4-4)2

           = (-2.8)2 + (3.5)2 + (0.8)2 + (-0.9)2 + (-0.6)2

           = 7.84 + 12.25 + 0.64 + 0.81 + 0.36 = 21.9

  RMSE = √ 21.9/5 = SQRT(21.9/5)

       = 2.0928449536


By computing the values of slope m and intercept c, in the equation y = mx + c, a linear relationship is derived as y = 11.9 - 1.7x. The yp is the predicted value of y for each value of x.
ypi = 11.9 - 1.7 xi.
 
Linear regression is a simple way to predict outcome of a dependent variable y by deriving a linear equation.
 
  Plot of data
 
The Root Mean Square Error (RMSE) represents a factor, which is the difference between the actual value and the predicted value. In the above graph, it is clear that three predicted values are close the actual value.


Linear Regression
 
Machine Learning
 
Table of Content


Top


Revised Date: April 16th, 2024