CODEWITHSUNDEEP
machine-learningdeep-learningobject-detectionyoloyolov8
HarriS
HarriS

Reputation: 850

YOLOv8: How to calculate map on test set

Let's say I have a folder called 'test' with folders inside, 'images' and 'labels'. I also have a YOLOv8 model which I've trained called 'best.pt. My labels are polygons (yolo-obb .txt files).

I want to find the mean average precision (MAP) of my YOLOv8 model on this test set.

I've read both the documentation for predicting and benchmarking, however, I'm struggling to find an example of calculating map from some test images.

https://docs.ultralytics.com/modes/predict/

https://docs.ultralytics.com/modes/benchmark/

from ultralytics import YOLO

# Load a pretrained YOLOv8n model
model = YOLO('best.pt')

# Run inference on an image
results = model(['test/images/bus.jpg', 'test/images/zidane.jpg'])  # list of 2 Results objects

I imagine I have to put the list of images in the above, then write code to calculate map for everything in the test folder and average it. Are there packages that have already done this?

What's the code to achieve this task?

Upvotes: 1

Views: 5188

Answers (2)

Reggie
Reggie

Reputation: 31

There is a very convenient way with Yolov8 to solve your Problem without chanching the path for val. You can have your test images in folder "test" with folders "images" & "labels" inside and just change the input argument split of model.val() to split="test". The default for the split arg is "val", which means it takes the images valid folder by default. Changing the arg to split="test" you specify that the folder "test" should be used for running validation. Like so:

model = YOLO(model="best.pt") 

model.val(data=yaml_file_path_of_yourData, split="test")

Upvotes: 3

hanna_liavoshka
hanna_liavoshka

Reputation: 1315

To validate YOLOv8 model on a test set do the following:

  1. In the data.yaml file specify the test folder path as a val argument:
path: ../dataset  # dataset root dir
train: train  
val: test  # test directory path for validation

names:
  0: person
  1: bicycle
  1. Validate the model:
from ultralytics import YOLO

# Load a pretrained YOLOv8 model
model = YOLO('best.pt')

# Run validation on a set specified as 'val' argument
metrics = model.val(data='data.yaml')
  1. Get the relevant metrics:
metrics.results_dict

# output example

{'metrics/precision(B)': 0.8539022762405677,
 'metrics/recall(B)': 0.8012653662055587,
 'metrics/mAP50(B)': 0.8833459433957722,
 'metrics/mAP50-95(B)': 0.6608424795290528,
 'fitness': 0.6830928259157248}

More about the validation mode: https://docs.ultralytics.com/modes/val/

Upvotes: 3

Related Questions