CODEWITHSUNDEEP
machine-learninglogistic-regression
imran khan
imran khan

Reputation: 21

Logistic Regression in Machine Learning

I'm researching on the topic of "logistic Regression" in machine learning.I could understand the entire concept that it's trying to maximize the likelihood of an instance belonging to a particular class label

The algorithm, if run for many iterations, finds a weight vector that separates the instances and then keeps increasing the magnitude of the weight vector. I donot understand why would it try to increase the magnitude of weight vector

Any Help would be highly appreciable!

Upvotes: 1

Views: 1107

Answers (2)

Niki
Niki

Reputation: 15868

I'm guessing your data is linearly separable? IIRC, logistic regression breaks down in that case. I think this is a well-known problem. Quoting from here (the first hit on google for "logistic regression separable data"):

...However, when the training data is linearly separable, two bad things happen: 1. |θ| goes to infinity; 2. There are infinite number of MLE’s. To see this, note any step function (sigmoid with |θ|=∞) that is in the gap between the two classes is an MLE

One way to avoid this is to incorporate a prior on θ in the form of a zero-mean Gaussian with covariance 1/(2λ)I

Upvotes: 1

Rishabh Malhotra
Rishabh Malhotra

Reputation: 269

I think what you are asking about is Regularization in Machine learning, From my understanding of it this is done so as to avoid the phenomena known as overfitting i.e when the hypothesis fits the training data almost perfectly at the cost of giving a poor hypothesis for the test data.

I hope this helps to an extent.

Upvotes: 0

Related Questions