CODEWITHSUNDEEP
neural-networkclassificationdeep-learning
filup
filup

Reputation: 195

Binary Classification Cost Function, Neural Networks

I've been tweaking the Deep Learning tutorial to train the weights of a Logistic Regression model for a binary classification problem and the tutorial uses the negative log-likelihood cost function below...

self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b)

def negative_log_likelihood(self, y):  
    return -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])

However, my weights don't seem to be converging properly as my validation error increases over successive epochs.

I was wondering if I'm using the proper cost function to converge upon the proper weights. It might be useful to note that my two classes are very imbalanced and my predictors are already normalized

Upvotes: 1

Views: 991

Answers (1)

London guy
London guy

Reputation: 28022

Few reasons I can think of are:

  1. Your learning rate is too high

  2. For binary classification, try squared error or a cross entropy error instead of negative log likelihood.

  3. You are using just one layer. May be the dataset you are using requires more layers. So connect more hidden layers.

  4. Play around with the number of layers and hidden units.

Upvotes: 5

Related Questions