CODEWITHSUNDEEP
androidalgorithmopencv
Aletheios
Aletheios

Reputation: 4020

Get coordinates of non-geometric lines in a binary image

I'm trying to recognize hand positions in OpenCV for Android. I'd like to reduce a detected hand shape to a set of simple lines (= point sequences). I'm using a thinning algorithm to find the skeleton lines of detected hand shapes. Here's an exemplary result (image of my left hand):

enter image description here

In this image I'd like to get the coordinates of the skeleton lines, i.e. "vectorize" the image. I've tried HoughLinesP but this only produces huge sets of very short lines, which is not what I want.

My second approach uses findContours:

// Get contours
Mat skeletonFrame; //image above
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(skeletonFrame, contours, new Mat(), Imgproc.RETR_CCOMP, Imgproc.CHAIN_APPROX_SIMPLE);

// Find longest contour
double maxLen = 0;
MatOfPoint max = null;
for (MatOfPoint c : contours) {
    double len = Imgproc.arcLength(Util.convert(c), true);  //Util.convert converts between MatOfPoint and MatOfPoint2f
    if (len > maxLen) {
        maxLen = len;
        max = c;
    }
}

// Simplify detected contour
MatOfPoint2f result = new MatOfPoint2f();
Imgproc.approxPolyDP(Util.convert(max), result, 5.0, false);

This basically works; however, the contours returned by findContours are always closed, which means that all the skeleton lines are represented twice.

Exemplary result: (gray lines = detected contours, not skeleton lines of first image)

enter image description here

So my question is: How can I avoid these closed contours and only get a collection of "single stroke" point sequences?

Did I miss something in the OpenCV docs? I'm not necessarily asking for code, a hint for an algorithm I could implement myself would also be great. Thanks!

Upvotes: 3

Views: 522

Answers (1)

Spektre
Spektre

Reputation: 51845

I would start with real hand skeleton as kinematic

  1. find fingers endpoints and hand/wrist base and perimeter boundary (red)

  2. solve inverse kinematics

    for example by CCD to match the fingers endpoints and not overlapping image. This way you should obtain anatomically correct answer

  3. for simplification you can use kinematics like this

    xray/kinematics

    you should handle male/female/child differently (different finger lengths) or use some kind of calibration or measurement because of the different finger lengths. As you can see I skip the hand/wrist base bones they are not that important. The red outline can be found where perimeter has smaller curve radius.

How to solve your problem in your current implementation?

The first thinning approach is better so when you got the huge set of lines connect them to polylines after that compute angle of each line. If two joined lines have similar angle (up to treshold) then join them that should do what you want but do not expect you will get the lines similar to human bones especially for curves the result will be much different. In both count of lines and shape.

For better result you need to use geometrical thinning

But I have no Idea if it is present in OpenCV (I do not use this lib) the idea is to find the perimeter line and shift it perpendicular inwards by some small step similar to this. Stop if desired width is acquired

geometrical thinning

When shifted perimeter leads to too thin shape stop there and connect to thinned point from previous step (yellow line). This is all done on vectors (polylines) not on image pixels !!! Width can be computed as smallest perpendicular distance to any nearby line.

Upvotes: 2

Related Questions