CODEWITHSUNDEEP
pythonpython-3.xtensorflowevaluationtensorflow-datasets
Free Url
Free Url

Reputation: 1966

How to evaluate Tensorflow model on a full validation example

I am using Tensorflow's tf.estimator.DNNClassifier.evaluate function to evaluate my network. I am using Tensorflow's Iris data located here. Upon some further investigation and maths, I have determined that no matter the evaluation set size, 128 examples are evaluated.

For instance, the iris_training.csv has only 120 examples, and 128 are evaluated (presumably 8 are evaluated twice). I also have some validation sets with 1k examples, but only 128 examples are evaluated in this case as well.

I figured this out by dividing the total loss by the average loss to get the number of examples evaluated. I would like to be able to vary this arbitrarily, especially when working with larger validation sets. How can this be accomplished?

I have tried with the evaluation function's steps argument = 1, 2, 10, 120, etc... which results in the the same, 128 examples being evaluated, and steps = None which results in the function never returning.

I can't understand from where this 128 metric is originating, even after having scoured the API located here. I also looked into using the similar tf.contrib.learn.DNNClassifier's estimator located here, but the API states that the function is deprecated and will be removed, so I figured that wasn't the best option.

I am assuming that the steps argument isn't what I'm looking for after locating another SO page here.

So, how can I use Tensorflow to evaluate an arbitrary number of validation examples?

Upvotes: 2

Views: 1021

Answers (1)

Free Url
Free Url

Reputation: 1966

The 128 example evaluation metric comes from the tf.estimator.inputs.numpy_input_fn function that I was using to create the input function for my examples. The API for this function located here specifies 128 to be the default argument if another argument for batch_size is not provided. Providing the batch_size argument with the desired number of examples to be evaluated will rectify the problem above.

Upvotes: 2

Related Questions