CODEWITHSUNDEEP
pythonmachine-learningscikit-learnstatisticslinear-regression
rhombidodecahedron
rhombidodecahedron

Reputation: 7922

Making an L-curve with sklearn's ridge regression

A common way to visualize the solution of ridge regression is an L curve which plots the sum of squared errors against the ridge penalty for different choices of the regularization parameter. Is this possible to make with sklearn?

Upvotes: 0

Views: 1474

Answers (2)

desertnaut
desertnaut

Reputation: 60321

There is no such built-in functionality in scikit-learn, but such functionality is provided by the Yellowbrick library (install with pip or conda); adapting the LassoCV example from their documentation to your RidgeCV case gives:

import numpy as np
from sklearn.linear_model import RidgeCV
from yellowbrick.regressor import AlphaSelection
from yellowbrick.datasets import load_concrete

# Load the regression dataset
X, y = load_concrete()

# Create a list of alphas to cross-validate against
alphas = np.logspace(-10, 1, 40)

# Instantiate the linear model and visualizer
model = RidgeCV(alphas=alphas)
visualizer = AlphaSelection(model)
visualizer.fit(X, y)
visualizer.show()

enter image description here

Upvotes: 0

rhombidodecahedron
rhombidodecahedron

Reputation: 7922

Here's a pure sklearn answer: 

import numpy as np
from sklearn.linear_model import Ridge

alphas = np.logspace(-10, 10, 1000)
solution_norm = []
residual_norm = []

for alpha in alphas: 
    lm = Ridge(alpha=alpha)
    lm.fit(X, y)
    solution_norm += [(lm.coef_**2).sum()]
    residual_norm += [((lm.predict(X) - y)**2).sum()]

plt.loglog(residual_norm, solution_norm, 'k-')
plt.show()

where X and y are your predictors and targets, respectively.

Upvotes: 3

Related Questions