CODEWITHSUNDEEP
machine-learninglinear-regression
sravanreddy001
sravanreddy001

Reputation: 257

Range of regularizer constant in linear regression

Is there any limit on the range of values that can be used for 'Lambda' - regularizer constant in Linear Regression. [Machine Learning Problem]

I am getting a good fit for the data when the Lambda value is 100. Is this in "Practical" acceptable limit?

I think theoretically, and value of Lambda is fine, but for practical use?

[small clarification needed in solving a Homework Problem.]

Upvotes: 1

Views: 289

Answers (1)

Ivan Koblik
Ivan Koblik

Reputation: 4315

I don't think that there is limit on practical value of lambda. You chose what works best for your problem and stick to it. For example if you have too many features and too little data then model will tend to overfit, you'd have to fight it with greater lambda (better thing to do would be get more data or remove some features). On the other hand if you have lots of data with relatively small number of features overfitting will not be such a big issue and lambda can be smaller.

One way to chose Lambda is to train your model with values of lambda [A .. B] (where A>0, B>A) and then plot cost function on cross validation set. It will be close to a convex curve, lambda corresponding to the global minima will be the optimal one.

For a more complete explanation please watch this lecture: https://class.coursera.org/ml/lecture/preview/63

Upvotes: 2

Related Questions