Detection of four corners of a document under different circumstances

I have tried 2 methodologies as follows:-

conversion of image to Mat
apply gaussian blur
then canny edge detection
find contours

The problem with this method is:

too many contours are detected
mostly open contours
doesn't detect what I want to detect

Then I changed my approach and tried adaptive thresholding after gaussian blur/median blur and it is much better and I am able to detect the corners in 50% cases

The current problem I am facing is that the page detection requires contrasting and plain background without any reflections. I think it's too idealistic for real world use.

This is where I would like some help. Even a direction towards the solution is highly appreciated especially in java. Thanks in anticipation works absolutely fine with a significant contrasting background like this

Detected 4 corners This picture gives troubles because the background isn't exactly the most contrasting

Initial largest contour found

Update: median blur did not help much so I traced the cause and found that the page boundary was detected in bits and pieces and not a single contour so it detected the biggest contour as a part of the page boundary Therefore performed some morphological operations to close relatively small gaps and the resultant largest contour is definitely improved but its its not optimum. Any ideas how I can improve the big gaps?

morphed original picture

largest contour found in the morphed image

PS morphing the image in ideal scenarios has led to detection of false contour boundaries. Any condition which can be checked before morphing an image is also a bonus. Thank you

Upvotes: 3

Answers (3)

Peter Wishart

Reputation: 12330

You can pick a single contour by using one or both of:

Use BoundingRect and ContourArea to evaluate the squareness of each contour. boundingRect() returns orthogonal rects., to handle arbitrary rotation better use minAreaRect() which returns optimally rotated ones.

Use Cv.ApproxPoly iteratively to reduce to a 4 sided shape

        var approxIter = 1;
        while (true)
        {
            var approxCurve = Cv.ApproxPoly(largestContour, 0, null, ApproxPolyMethod.DP, approxIter, true);
            var approxCurvePointsTmp = new[] { approxCurve.Select(p => new CvPoint2D32f((int)p.Value.X, (int)p.Value.Y)).ToArray() }.ToArray();
            if (approxCurvePointsTmp[0].Length == 4)
            {
                corners = approxCurvePointsTmp[0];
                break;
            }
            else if (approxCurvePointsTmp[0].Length < 4) throw new InvalidOperationException("Failed to decimate corner points");
            approxIter++;
        }

However neither of these will help if the contour detection gives you two separate contours due to noise / contrast.

I think it would be possible to use the hough line transformation to help detect cases where a line has been split into two contours.

If so the search could be repeated for all combinations of joined contours to see if a bigger / more rectangular match is found.

Upvotes: 1

Andrii Omelchenko

Reputation: 13353

If you use methods like that:

public static RotatedRect getBestRectByArea(List<RotatedRect> boundingRects) {
    RotatedRect bestRect = null;

    if (boundingRects.size() >= 1) {
        RotatedRect boundingRect;
        Point[] vertices = new Point[4];
        Rect rect;
        double maxArea;
        int ixMaxArea = 0;

        // find best rect by area
        boundingRect = boundingRects.get(ixMaxArea);
        boundingRect.points(vertices);
        rect = Imgproc.boundingRect(new MatOfPoint(vertices));
        maxArea = rect.area();

        for (int ix = 1; ix < boundingRects.size(); ix++) {
            boundingRect = boundingRects.get(ix);
            boundingRect.points(vertices);
            rect = Imgproc.boundingRect(new MatOfPoint(vertices));

            if (rect.area() > maxArea) {
                maxArea = rect.area();
                ixMaxArea = ix;
            }
        }

        bestRect = boundingRects.get(ixMaxArea);
    }

    return bestRect;
}

private static Bitmap findROI(Bitmap sourceBitmap) {
    Bitmap roiBitmap = Bitmap.createBitmap(sourceBitmap.getWidth(), sourceBitmap.getHeight(), Bitmap.Config.ARGB_8888);

    Mat sourceMat = new Mat(sourceBitmap.getWidth(), sourceBitmap.getHeight(), CV_8UC3);
    Utils.bitmapToMat(sourceBitmap, sourceMat);

    final Mat mat = new Mat();
    sourceMat.copyTo(mat);

    Imgproc.cvtColor(mat, mat, Imgproc.COLOR_RGB2GRAY);
    Imgproc.threshold(mat, mat, 146, 250, Imgproc.THRESH_BINARY);

    // find contours
    List<MatOfPoint> contours = new ArrayList<>();
    List<RotatedRect> boundingRects = new ArrayList<>();
    Imgproc.findContours(mat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);

    // find appropriate bounding rectangles
    for (MatOfPoint contour : contours) {
        MatOfPoint2f areaPoints = new MatOfPoint2f(contour.toArray());
        RotatedRect boundingRect = Imgproc.minAreaRect(areaPoints);
        boundingRects.add(boundingRect);
    }

    RotatedRect documentRect = getBestRectByArea(boundingRects);
    if (documentRect != null) {
        Point rect_points[] = new Point[4];
        documentRect.points(rect_points);
        for (int i = 0; i < 4; ++i) {
            Imgproc.line(sourceMat, rect_points[i], rect_points[(i + 1) % 4], ROI_COLOR, ROI_WIDTH);
        }
    }
    Utils.matToBitmap(sourceMat, roiBitmap);
    return roiBitmap;
}

you can achieve for your source images results like this:

or that:

If you adjust threshold values and apply filters you can achieve even better results.

Upvotes: 1

user1196549

Reputation:

Stop relying on edge detection, the worst methodology in the universe, and switch to some form of image segmentation.

The paper is white, the background is contrasted, this is the information that you should use.

Upvotes: -1

Detection of four corners of a document under different circumstances

Answers (3)

Related Questions