CODEWITHSUNDEEP
object-detectionyolodarknet
SetNug
SetNug

Reputation: 403

Calculating YOLO mAP against test dataset

AFAIK YOLO calculates mAP against validation dataset during training. Now is it possible to calculate the same against unseen test dataset ?

Command:

./darknet detector map obj.data yolo-obj.cfg yolo-obj_best.weights

obj.data:

classes = 1
train = train.txt
valid = test.txt
names = classes.txt
backup = backup

I have directed valid to test dataset containing annotated images. But I always get the following result:

 calculation mAP (mean average precision)...
44
 detections_count = 50, unique_truth_count = 43  
class_id = 0, name = traffic_light, ap = 100.00%     (TP = 43, FP = 0) 

 for conf_thresh = 0.25, precision = 1.00, recall = 1.00, F1-score = 1.00 
 for conf_thresh = 0.25, TP = 43, FP = 0, FN = 0, average IoU = 85.24 % 

 IoU threshold = 50 %, used Area-Under-Curve for each unique Recall 
 mean average precision ([email protected]) = 1.000000, or 100.00 % 
Total Detection Time: 118 Seconds

It's not that I'm not happy with 100% mAP, but it's definitely wrong isn't it?

Any advice would be greatly appreciated.

Regards,

Setnug

Upvotes: 4

Views: 12133

Answers (1)

Venkatesh Wadawadagi
Venkatesh Wadawadagi

Reputation: 2943

Now is it possible to calculate the same against unseen test dataset ?

Yes, mAP calculation needs images with corresponding labels/annotation that's all.

I have directed valid to test dataset containing annotated images.

Yes, this is the way to do what you wanted.

There is a possibility that what you're seeing here is this known bug, provided you're using old code and haven't updated after that. In that case suggest you to pull the latest darknet and try.

Note that if the model is trained really well and if your test set is simple in terms of complexity (though it's unseen) or it's visually similar to that of train set, it's possible to get such numbers as well, as we're talking about small number of test samples.

Upvotes: 1

Related Questions