CODEWITHSUNDEEP
pythonpython-3.xstatisticsregressionlinear-regression
Satheesh Kumar
Satheesh Kumar

Reputation: 159

Multiple Linear Regression - Determining Coefficients for 3 independent variables

I am struggling to find the coefficients for b1, b2 and b3. My model has 3 independent variable x1, x2 and x3 and one dependent variable y. 

x1,x2,x3,y
89,4,3.84,7
66,1,3.19,5.4
78,3,3.78,6.6
111,6,3.89,7.4
44,1,3.57,4.8
77,3,3.57,6.4
80,3,3.03,7
66,2,3.51,5.6
109,5,3.54,7.3
76,3,3.25,6.4

I want to use the matrix method to find out the coefficients for b1, b2 and b3. From the tutorial that I am following the value for b1 is 0.0141, b2 is 0.383 and b3 is -0.607.

I am not sure about how to achieve those values mentioned above, when I tried to inverse the matrix containing x1, x2, x3 values I am getting the below error.

raise LinAlgError('Last 2 dimensions of the array must be square')
numpy.linalg.linalg.LinAlgError: Last 2 dimensions of the array must be square

Please someone help me solve this matrix so that I can get the desired values.

Upvotes: 2

Views: 764

Answers (1)

Simon
Simon

Reputation: 10150

In matrix form, the regression coefficients are given by

enter image description here

Where x is your data matrix of predictors, and y is a vector of outcome values

In python (numpy), that looks something like this:

import numpy as np

b = np.dot(x.T, x)
b = np.linalg.inv(b)
b = np.dot(b, x.T)
b = np.dot(b, y)

Using that on your data you get the following coefficients:

0.0589514 , -0.25211869,  0.70097577

Those values don't match your expected output, and it's because the tutorial you're following must also be modelling an intercept. To do that we add a column of ones to the data matrix so it looks like this:

x.insert(loc=0, column='x0', value=np.ones(10))

    x0   x1  x2    x3
0  1.0   89   4  3.84
1  1.0   66   1  3.19
2  1.0   78   3  3.78
3  1.0  111   6  3.89
4  1.0   44   1  3.57
5  1.0   77   3  3.57
6  1.0   80   3  3.03
7  1.0   66   2  3.51
8  1.0  109   5  3.54
9  1.0   76   3  3.25

Now we get the expected regression coefficients (plus an additional value for the intercept):

6.21137766,  0.01412189,  0.38315024, -0.60655271

Upvotes: 1

Related Questions