CODEWITHSUNDEEP
pythontensorflowmachine-learningclassificationloss-function
HotColdWeather
HotColdWeather

Reputation: 23

Where does Tensorflow Weighted Cross Entropy loss function goes in the DNN Classifier Estimator function?

I am currently working on a binomial classification algorithm with a highly skewed data (90% negative/10% positive), using tf.estimator.DNNClassifier. As all models that I train converge to label all samples as negatives, I need to implement a weighted loss function.

I looked at many different questions here and many of them are enlightning. However, I could not get a practical end-to-end answer on how actually implement the functions. This and this threads were the best.

My problem is: I want to use tf.nn.weighted_cross_entropy_with_logits(), but I don't know where I should insert it in my code.

I have a function to construct the Feature Colums:

def construct_feature_columns(input_features):
  return set([tf.feature_column.numeric_column(my_feature)
              for my_feature in input_features])

A function that defines the tf.estimator.DNNClassifier and also other parameters, like optimizer and input function:

def train_nn_classifier_model(
    learning_rate,
    steps,
    batch_size,
    hidden_units,
    training_examples,
    training_targets,
    validation_examples,
    validation_targets):

    dnn_classifier = tf.estimator.DNNClassifier(
        feature_columns=construct_feature_columns(training_examples),
        hidden_units=hidden_units,
        optimizer=my_optimizer)

The training function:

dnn_classifier.train(input_fn=training_input_fn, steps=steps_per_period)

The predict function, to calculate the error while training:

training_probabilities = dnn_classifier.predict(input_fn=predict_training_input_fn)

The Optimizer:

  my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
  my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)

Input functions (for training input, predicting training inputs and validation inputs):

  training_input_fn = lambda: my_input_fn(
      training_examples, 
      training_targets['True/False'], 
      batch_size=batch_size)

Where should I insert tf.nn.weighted_cross_entropy_with_logits, so my model calculates the losses using this function?

Also, how do I call the targets (A Tensor of the same type and shape as logits) inside the cross entropy function? Is it the training_targets DataFrame of is it the output of the input function having the training_targets as input?

And what are the logits specifically? Because for me, they should be the Predictions that come from the function:

training_probabilities = dnn_classifier.predict(input_fn=predict_training_input_fn)

But it wouldn't make sense to me. I tried many different ways to implement it, but none of them worked.

Upvotes: 2

Views: 438

Answers (1)

dennlinger
dennlinger

Reputation: 11460

I hate to be the bearer of bad news, but the DNN Classifier does not support custom loss functions:

Loss is calculated by using softmax cross entropy.

THis is the only mention of loss (functions) in the documentation, and I could not find any post that talked about a working solution to this by directly changing the DNNClassifier. Instead, it looks like you would have to build your own custom Estimator.

Upvotes: 1

Related Questions