CODEWITHSUNDEEP
python
KonK3
KonK3

Reputation: 81

Error in regression analysis with 3 classes

I am trying to apply one vs all logistic regression:

I am using one vs all method (class1 vs class2+ class3, c2 vs c1+c3, c3 vs c1+c2) to calculate the three cases weights w1,w2,w3:

   for n1 in range(0,50000):

        s1 = np.dot(dt, w1)
        p1 = (1 / (1 + np.exp(-s1)))
        gr1 = (np.dot(dt.T, p1-c1))
        gr1 /= N
        w1 -= 0.01 * gr1

        if n1 % 10000 == 0 and n1!=0:
          loss1 = np.sum( np.log(1 + np.exp(s1))-p1*s1 )
          print('loss1',loss1)

dt is my features,

w1,w2,w3 are initialized as w1=np.zeros((5,1)),

c1=np.vstack((np.ones(40),np.zeros(40),np.zeros(40)))
c2=np.vstack((np.zeros(40),np.ones(40),np.zeros(40)))
c3=np.vstack((np.zeros(40),np.zeros(40),np.ones(40)))

Upvotes: 1

Views: 52

Answers (1)

Júlio Cesar Pereira Rocha
Júlio Cesar Pereira Rocha

Reputation: 109

so. the iris data set is not perfect linearly separable in all the sets. so wen we use a linear classifier like logistic regression the loss in the part that is not linearly separable tends to be unpredictable. you can put a a very small learning hate and a patiently method to avoid overffitting. normalization of you data between 0 and 1 will help too.

Upvotes: 1

Related Questions