CODEWITHSUNDEEP
pythonmachine-learningscikit-learnlasso-regression
user9102437
user9102437

Reputation: 742

Why does sklearn's Lasso coefficients not equal to Linear Regression ones?

I am trying to implement sklearn's lasso in my code. To test it out, I have decided to make a test with alpha = 0. This should by definition yield the same result as LinearRegression, but this is not the case.
Here is the code:

import pandas as pd
from sklearn.linear_model import Lasso
from sklearn.linear_model import LinearRegression

# Don't worry about this. It is made so that we can work with the same dataset.
df = pd.read_csv('http://web.stanford.edu/~oleg2/hse/Credit.csv').dropna()
df['Asian'] = df.Ethnicity=='Asian'
df['Caucasian'] = df.Ethnicity=='Caucasian'
df['African American'] = df.Ethnicity=='African American'
df = df.drop(['Ethnicity'],axis=1).replace(['Yes','No','Male','Female',True,False],[1,0,1,0,1,0])
# End of unimportant part

ft = Lasso(alpha=0).fit(x, df.Balance)
print(ft.intercept_)
ft = LinearRegression().fit(x, df.Balance)
print(ft.intercept_)

Output:

-485.3744897927978
-480.89071679937786

The coef_s are also all different.

What am I doing wrong?

Upvotes: 3

Views: 1505

Answers (1)

yatu
yatu

Reputation: 88236

Indeed this seems to yield different results. However, running your code, also yielded the following warning:

ft = Lasso(alpha=0).fit(X, y)
print(ft.intercept_)
ft = LinearRegression().fit(X, y)
print(ft.intercept_)

-485.3744897927984
-480.89071679937854

UserWarning: With alpha=0, this algorithm does not converge well. You are advised to use the LinearRegression estimator

This is letting you know that since alpha=0, meaning that we're only left with an ordinary linear regression, the algorithm won't converge well. Which is why you see a difference in the intercept, and presumably a worsening of the metrics.

Upvotes: 0

Related Questions