CODEWITHSUNDEEP
pythoncomputer-visionobject-detectiondetectronkeypoint
Amit
Amit

Reputation: 51

In detectron2, how could I put the keypoint names on the image?

First, I will explain the problem statement. I am wuing Detectron2 for keypoint detection for my custom dataset. I am using the below code to visualize the bounding box and the keyoints.

outputs = predictor(im[..., ::-1])
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
plt.figure(figsize=(60,30))
plt.imshow(out.get_image()[..., ::-1][..., ::-1])

I get the output as an image below.

enter image description here

As you can see, the class label and the two keypoints are drawn on the image by the detectron2 visualizer. The two keypoint names are [keypoint_up, keypoint_down].

I want to print the names of the two keypoint on the image as well. Someting like below example.

enter image description here

I know detectron provide a function called draw_text. If anyone has an idea of how I could use it to put the keypoint name on the image, that would be a great help. A coding example would be very useful. Any other solution except the draw_text function is also welcomed. Thank you for your time

Upvotes: 1

Views: 475

Answers (1)

Tyem
Tyem

Reputation: 1

Your approach to display the names of the keypoints alongside the detected keypoints is quite reasonable. Let's complete the solution by implementing it step by step:

  1. Iterate through predicted keypoints:
  2. Extract x and y coordinates:
  3. Map keypoints to their names:
  4. Draw text on the image:

Here's what I used for visualizing keypoints' labels on the image:

outputs = predictor(im[..., ::-1])
v = Visualizer(im[:, :, ::-1], MetadataCatalog.get(cfg.DATASETS.TRAIN[0]), scale=1.2)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
# Iterate over each detected instance and its keypoints
instances = outputs["instances"].to("cpu")

for i in range(len(instances)):
    keypoints = instances.pred_keypoints[i]  # Get keypoints for the i-th instance
      for j, predicted_keypoint in enumerate(keypoints):
        x, y, prob = predicted_keypoint
        if prob > 0.5:  # Adjust the probability threshold if necessary
           position = (x.item(), y.item())
           text = MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).keypoint_names[j]  # Get the corresponding keypoint name
           keypoints_positions.append((text, position))

# Draw all keypoints names on the image at once
for text, position in keypoints_positions:
    v.draw_text(text, position, font_size=12, color="g", horizontal_alignment="center")

# Visualize the result
plt.figure(figsize=(60, 30))
plt.imshow(out.get_image())
plt.show()

In this solution:

  • We iterate through each detected instance and its keypoints.
  • For each keypoint, we extract its coordinates and the probability.
  • We map each keypoint to its corresponding name from the keypoint_names list.
  • We use v.draw_text to place the keypoint name on the image.

This should result in an image that displays both the keypoints as dots and their corresponding names. Hope this helps you achieve the desired visualization! 🛠️ I hope it'll help. Don't hesitate when you have questions about my suggested solution.

Upvotes: 0

Related Questions