CODEWITHSUNDEEP
pythonscikit-learnregression
Cheshie
Cheshie

Reputation: 2837

How to set the learning rate in scikit-learn's ridge regression?

I'm using scikit-learn's ridge regression:

regr = linear_model.Ridge (alpha = 0.5)

# Train the model using the training sets
regr.fit(X_train, Y_train)

#bias:
print('bias: \n', regr.intercept_)
# The coefficients
print('Coefficients: \n', regr.coef_)

I found (here) the different options for the linear_model.Ridge function, but there is a specific option that I didn't find in the list: How could I set the learning rate (or learning step) of the update function?

By learning rate, I mean:

w_{t+1} = w_t + (learning_rate) * (partial derivative of objective function)

Upvotes: 3

Views: 2676

Answers (1)

ilyas patanam
ilyas patanam

Reputation: 5324

I refer to learning rate as step size.

Your code is not using the sag (stochastic average gradient) solver. The default parameter for solver is set to auto, which will choose a solver depending on the data type. A description of the other solvers and which to use is here.

To use the sag solver:

regr = linear_model.Ridge (alpha = 0.5, solver = 'sag')

However, for this solver you do not set the step size because the solver computes the step size based on your data and alpha. Here is the code for sag solver used for ridge regression, where they explain how the step size is computed.

The step size is set to 1 / (alpha_scaled + L + fit_intercept) where L is the max sum of squares for over all samples.

Line 401 shows how sag_solver being used for ridge regression.

Upvotes: 2

Related Questions