Overlapping shapes recognition (OpenCV)

Question

I have a simple image containing some shapes: some rectangles and some ellipses, in total number of 4 or 5. The shapes can be rotated, scaled and overlapped. There is a sample input: My task is to detect all of these figures and prepare some information about them: size, position, rotation, etc. In my opinion, the core problem is the fact that the shapes can be overlapped by each other. I tried to search some information about this kind of problem and find that OpenCV library can be very useful.

OpenCV has the ability to detect contours and then try to fit ellipses or rectangles to these contours. The problem is when shapes are overllaped, the contours are mixed up.

I think about following algorithm: detect all characteristic points: and put white dot at them. I got something like these where every figure is divided into separate sections: Then I can try to link these parts using some information, for example the complexity value (I fit the curve approxPolyDP to the contour and the read how many parts it has). But it starts to be very hard. The other idea is to try all of the permutations of linking the contours and trying to fit the figures to them. The best compilation will be output.

Any ideas how to create simple but elegant solution?

Answer 1

blurring image helps to find intersections as seen in the code

#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"

using namespace cv;

int main( int argc, char** argv )
{
    Mat src = imread( argv[1] );
    Mat gray, blurred;
    cvtColor( src, gray, COLOR_BGR2GRAY );
    threshold( gray, gray, 127, 255, THRESH_BINARY );
    GaussianBlur( gray, blurred, Size(), 9 );
    threshold( blurred, blurred, 200, 255, THRESH_BINARY_INV );
    gray.setTo( 255, blurred );
    imshow("result",gray);
    waitKey();

    return 0;
}

result image:

step 2

simply, borrowed code from generalContours_demo2.cpp

#include "opencv2/imgproc.hpp"
#include "opencv2/highgui.hpp"

using namespace cv;
using namespace std;

int main( int argc, char** argv )
{
    Mat src = imread( argv[1] );
    Mat gray, blurred;
    cvtColor( src, gray, COLOR_BGR2GRAY );
    threshold( gray, gray, 127, 255, THRESH_BINARY );
    GaussianBlur( gray, blurred, Size(), 5 );
    threshold( blurred, blurred, 180, 255, THRESH_BINARY_INV );
    gray.setTo( 255, blurred );
    imshow("result of step 1",gray);

    vector<vector<Point> > contours;

    /// Find contours
    findContours( gray.clone(), contours, RETR_TREE, CHAIN_APPROX_SIMPLE );

    /// Find the rotated rectangles and ellipses for each contour
    vector<RotatedRect> minRect( contours.size() );
    vector<RotatedRect> minEllipse( contours.size() );

    for( size_t i = 0; i < contours.size(); i++ )
    {
        minRect[i] = minAreaRect( Mat(contours[i]) );
        if( contours[i].size() > 5 )
        {
            minEllipse[i] = fitEllipse( Mat(contours[i]) );
        }
    }

    /// Draw contours + rotated rects + ellipses
    for( size_t i = 0; i< contours.size(); i++ )
    {
        Mat drawing = src.clone();
        // contour
        //drawContours( drawing, contours, (int)i, color, 1, 8, vector<Vec4i>(), 0, Point() );
        // ellipse
        ellipse( drawing, minEllipse[i], Scalar( 0, 0, 255 ), 2 );
        // rotated rectangle
        Point2f rect_points[4];
        minRect[i].points( rect_points );
        for( int j = 0; j < 4; j++ )
            line( drawing, rect_points[j], rect_points[(j+1)%4], Scalar( 0, 255, 0 ), 2 );
        /// Show in a window
        imshow( "results of step 2", drawing );
        waitKey();
    }

    return 0;
}

you can get following result images among others. i hope you will solve final step.