CODEWITHSUNDEEP
tensorflowobject-detection-api
Ayush Pandey
Ayush Pandey

Reputation: 587

How to evaluate all checkpoints using model_main.py

Currently I am running the following command to evaluate the latest checkpoint on my dataset:

python model_main.py --pipeline_config_path=training/faster_rcnn_inception_v2_pets.config --model_dir=training/ --sample_1_of_n_eval_examples=1 --eval_training_data=True  --sample_1_of_n_eval_on_train_examples=1 --checkpoint_dir=./training/  --run_once=True --alsologtostderr

However, I would like to evaluate all the checkpoints that I have saved. I am not able to figure out how to do this with model_main.py. I don't want to use legacy/eval.py as there are a lot of issues popping up with that script.

EDIT: my checkpoint file looks like this:

model_checkpoint_path: "model.ckpt-900"
all_model_checkpoint_paths: "model.ckpt-0"
all_model_checkpoint_paths: "model.ckpt-100"
all_model_checkpoint_paths: "model.ckpt-200"
all_model_checkpoint_paths: "model.ckpt-300"
all_model_checkpoint_paths: "model.ckpt-400"
all_model_checkpoint_paths: "model.ckpt-500"
all_model_checkpoint_paths: "model.ckpt-600"
all_model_checkpoint_paths: "model.ckpt-700"
all_model_checkpoint_paths: "model.ckpt-800"
all_model_checkpoint_paths: "model.ckpt-900"

Now instead of the latest checkpoint which is denoted by model_checkpoint_path, I want to evaluate all_mode_checkpoint_paths on a particular set.

Upvotes: 1

Views: 1464

Answers (1)

carobnodrvo
carobnodrvo

Reputation: 1051

For your answer a bit more detail is required so I am just going to make a guess here.

In the checkpoint directory ./training/, there should be a checkpoint file. This is a textual file and if you open it you will see something like this:

model_checkpoint_path: "model_10.ckpt"
all_model_checkpoint_paths: "model_01.ckpt"
all_model_checkpoint_paths: "model_02.ckpt"
all_model_checkpoint_paths: "model_03.ckpt"
...

You just need to edit the first line in order to evaluate a different checkpoint. For example, if you want to evaluate first checkpoint, just exchange first line for model_checkpoint_path: "model_01.ckpt".

You can then write small Python or Shell script which would evaluate all checkpoints.

Upvotes: 2

Related Questions