Ideas for Extracting Blade Tip Coordinates from masked Wind Turbine Image

Question

I am looking for image processing tools in python to get the coordinates from the Blade Tips of a Wind Turbine, in this case a small model of one. The blades already get segmented by a yoloV8 Segmantation Model, and now I want to use that image to get the xy coordinates of the tips. Example image: masked wings of a wind energy turbine. Can someone recommend some ideas how I could go about this? The rotor can be rotated, so the three tips could be anywhere on an ellipse.

I already tried training a yolo-pose model for keypoint detection, but it didn't give precise enough results. I will use these coordinates to calculate the excentricity of the rotor disk, so the points need to be somewhat precise.

Answer 1

You'll need several processing to robustly find the tips coordinates of the turbine, no matter its rotation. Basically you want to find the edges of the turbine, then compute the center of the shape and cluster each points of the contour in 3 groups (cause there is 3 blades). You can do everything using opencv and numpy

Loading the image

We use opencv to load the image and rasterize the colors to pure black or white. This will later help us find the contour:

def load_image(path):

    # Load the image
    image = cv2.imread(path)
    # Convert to grayscale
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    # Threshold the image to get a binary image
    _, binary = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY)

    return image, binary

It's a good start but as you can see the mask you got from yolo isn't 100% perfect. We'll easily fix that while computing the contours.

Compute the turbine contour

We can use cv2.findContours function to compute the different contours of the binary image. This will returns the contour of the turbine as well as the mask artefacts. So we will only keep the contour with the largest area, assuming that artifacts are smaller than the actual turbine:

def find_contour(binary):

    # Find the contours
    contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Since the masking is not perfect, we have several contours
    # Assuming the largest contour area corresponds to the turbine
    if len(contours) > 0:
        largest_contour = max(contours, key=cv2.contourArea)
        return largest_contour
    
    # If no contours, return False
    return False

Find the tips coordinates

In order to find the tips we'll cluster the contour points on 3 groups, for the 3 blades. So let's first find the center of the turbine:

def find_tips(contour):

    # Calculate the centroid of the contour
    M = cv2.moments(contour)
    cx = int(M['m10'] / M['m00'])
    cy = int(M['m01'] / M['m00'])
    centroid = (cx, cy)

    [...]

Then we need to compute the angles and distances from the center for each points. I also sort the angles_distances list by angle:

def find_tips(contour):

    [...]

    # Calculate angles and distances from centroid
    angles_distances = []
    for point in contour:
        x, y = point[0]
        angle = angle_from_centroid(centroid, (x, y))
        distance = np.sqrt((x - cx)**2 + (y - cy)**2)
        angles_distances.append((angle, distance, (x, y)))

        # Sort by angle
        angles_distances.sort()

    [...]

Here is my angle_from_centroid function to calculate the angle (in radians):

def angle_from_centroid(centroid, point):
    # Compute the angle of the vector from centroid to the point
    angle = np.arctan2(point[1] - centroid[1], point[0] - centroid[0])
    return angle

We can finally split our points into 3 cluster and save the farthest point of each cluster, these are our tips coordinates:

    # Divide into three clusters and find farthest point in each
    num_points = len(angles_distances)
    tips = []
    for i in range(3):
        cluster = angles_distances[i * num_points // 3: (i + 1) * num_points // 3]
        farthest_point = max(cluster, key=lambda x: x[1])[2]
        tips.append(farthest_point)
            
    return tips

This last part is a bit complexe, I'm not getting to deep into it. But we can now draw the tips of the blades.

Final result

We can now run the 3 functions described above and draw the tips coordinates on the image:

image, binary = load_image('resources/trBo33Jy.png')
contour = find_contour(binary)
coordinates = find_tips(contour)

# Draw the tips coordinates on the image
for point in coordinates:
    cv2.circle(image, point, 20, (0, 255, 0), -1)
    print(point)

# Display the image with marked points
cv2.imshow('Image with Tips', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

If you're looking for the most precise result possible, you could try to calculate the average angle of each cluster to get the vector of the blades

Answer 2

Due to some bad segmentations you will detect the background also but here is how to make it :

here you can find a colab notebook with this code ->colab but you need to upload your image.

In colab it uses a diffrent opencv patch. Thats why colab code and the one i write down is diffrent.

You might need to play with binarization to optimize your solution.

It basically binarizes the image then, then using countour detection it finds biggest contour. Since image has some false positive segmentations. The it find extremites in given contour.

We used to use this logic for detecting cell ends this is pretty similar so it solves this too.

import cv2
import numpy as np

# Load the image i have saved yours as img.png
image = cv2.imread("img.png", cv2.IMREAD_GRAYSCALE)

   # Binarization turns your detect pixels into big white blob so it is easiet to find contours.
_, binary = cv2.threshold(image, 10, 255, cv2.THRESH_BINARY)


# Find contours in the binary image we assume all the detected pixel are for windmill
contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# Check if contours were found
if len(contours) == 0:
    print("No contours detected. Check the input image or thresholding step.")
else:
    # Find the largest contour (most likely the windmill blades as white blob)
    largest_contour = max(contours, key=cv2.contourArea)

    # Find the extreme points (leftmost, rightmost, topmost, bottommost)
    leftmost = tuple(largest_contour[largest_contour[:, :, 0].argmin()][0])   # Smallest X
    rightmost = tuple(largest_contour[largest_contour[:, :, 0].argmax()][0])  # Largest X
    topmost = tuple(largest_contour[largest_contour[:, :, 1].argmin()][0])    # Smallest Y
    bottommost = tuple(largest_contour[largest_contour[:, :, 1].argmax()][0]) # Largest Y

    # Print the coordinates of the extreme points
    print(f"Leftmost: {leftmost}")
    print(f"Rightmost: {rightmost}")
    print(f"Topmost: {topmost}")
    print(f"Bottommost: {bottommost}")

    # Draw the results on the image
    output = cv2.cvtColor(binary, cv2.COLOR_GRAY2BGR)
    
    # Mark the extreme points on the image with red circles
    for point in [leftmost, rightmost, topmost]:
        cv2.circle(output, point, 10, (0, 0, 255), -1)  # Red color for points

    # Draw the contour in blue and convex hull in green
    cv2.drawContours(output, [largest_contour], -1, (255, 255, 0), 2)
    cv2.drawContours(output, [cv2.convexHull(largest_contour)], -1, (0, 255, 0), 2)

    
    cv2.imshow("result",output)
    cv2.waitKey(0)
    cv2.destroyAllWindows()