Detect small rectangles in image AForge

Question

I'm trying to detect rectangles on this image:

with this code:

static void Main(string[] args)
{
    // Open your image
    string path = "test.png";
    Bitmap image = (Bitmap)Bitmap.FromFile(path);

    // locating objects
    BlobCounter blobCounter = new BlobCounter();

    blobCounter.FilterBlobs = true;
    blobCounter.MinHeight = 5;
    blobCounter.MinWidth = 5;

    blobCounter.ProcessImage(image);
    Blob[] blobs = blobCounter.GetObjectsInformation();

    // check for rectangles
    SimpleShapeChecker shapeChecker = new SimpleShapeChecker();

    foreach (var blob in blobs)
    {
        List<IntPoint> edgePoints = blobCounter.GetBlobsEdgePoints(blob);
        List<IntPoint> cornerPoints;

        // use the shape checker to extract the corner points
        if (shapeChecker.IsQuadrilateral(edgePoints, out cornerPoints))
        {
            // only do things if the corners form a rectangle
            if (shapeChecker.CheckPolygonSubType(cornerPoints) == PolygonSubType.Rectangle)
            {
                // here i use the graphics class to draw an overlay, but you
                // could also just use the cornerPoints list to calculate your
                // x, y, width, height values.
                List<Point> Points = new List<Point>();
                foreach (var point in cornerPoints)
                {
                    Points.Add(new Point(point.X, point.Y));
                }

                Graphics g = Graphics.FromImage(image);
                g.DrawPolygon(new Pen(Color.Red, 5.0f), Points.ToArray());

                image.Save("result.png");
            }
        }
    }
}

but it dont recognize the rectangles (walls). It just recognize the big square, and when I reduce the minHeight and minWidth, it recognize trapezoids on the writing..

Answer 1

Implementation of @adelriosantiago's answer

Imports System.Drawing
Imports AForge.Imaging.Filters

Module Module1
    Sub Main()
        Dim Img = Bitmap.FromFile("Input.png")

        Dim grayImage = Grayscale.CommonAlgorithms.RMY.Apply(Img)
        Dim Edge = New DifferenceEdgeDetector()

        Dim DifferenceEdgeDetector = New DifferenceEdgeDetector()
        DifferenceEdgeDetector.ApplyInPlace(grayImage)

        Dim Dilatation = New Dilatation()
        Dilatation.Apply(grayImage)

        Dim Threshold = New Threshold(75) 'Adjust e.g 100
        Threshold.ApplyInPlace(grayImage)

        Dim Invert = New Invert()
        Invert.ApplyInPlace(grayImage)

        Dim ConnectedComponentsLabeling = New ConnectedComponentsLabeling()
        Dim newImage = ConnectedComponentsLabeling.Apply(grayImage)
        newImage.Save("Out.png")
    End Sub

End Module

Answer 2

We are trying to detect rectangles in so many rectangles (considering gray rectangles of grid). Almost all algorithms will get confused here. You're not eliminating externals from input image. Why not replace grid line color with background color or use threshold above to eliminate all grids first.

Then grow all pixels equal to the width of wall, Find all horizontal and vertical lines thereafter use maths to find rectangles using detected lines. Uncontrolled filling will be risky as when boundries are not closed fill will make two rooms as one rectangle.

Answer 3

If your problem now is to avoid noise due to writings on the image, use FillHoles with widht and height of holes smaller than the smallest rectangle but larger than any of the writings.

If the quality of image is good and no text is touching the border of the image, Invert the image and FillHoles will remove most of the stuff.

Hope I understood your problem correctly.

Answer 4

I propose a different algorithm approach, after working almost a year with image processing algorithms what I can tell is that to create an efficient algorithm, you have to "reflect" how you, as a human would do that, here is the proposed approach:

1. We don't really care about the textures, we care about the edges (rectangles are edges), therefore we will apply an Edge-detection>Difference (http://www.aforgenet.com/framework/docs/html/d0eb5827-33e6-c8bb-8a62-d6dd3634b0c9.htm), this gives us:

2. We want to exaggerate the walls, as humans we know that we are looking for the walls, but the computer does not know this, therefore, apply two rounds of Morphology>Dilatation (http://www.aforgenet.com/framework/docs/html/88f713d4-a469-30d2-dc57-5ceb33210723.htm), this gives us:

3. We care only about the what is wall and what is not, apply a Binarization>Threshold (http://www.aforgenet.com/framework/docs/html/503a43b9-d98b-a19f-b74e-44767916ad65.htm), we get:

4. (Optional) We can apply a blob extraction to erase the labels ("QUARTO, BANHEIRO", etc)

5. We apply a Color>Invert, this is just done because the next step detects the white color not black.

6. Apply a Blob>Processing>Connected Components Labeling (http://www.aforgenet.com/framework/docs/html/240525ea-c114-8b0a-f294-508aae3e95eb.htm), this will give us all the rectangles, like this:

Note that for each colored box you have its coordinates, center, width and height. So you can extract a snip from the real image with that coordinates.

PS: Using the program AForge Image Processing Lab is highly recommended to test your algos.

Answer 5

Each time a rectangle is found, the polygon is drawn on Graphics and the file is saved only for THAT rectangle. This means that result.png will only contain a single rectangle at a time.

Try first saving all the rectangles in a List<List<Points>> and then going over it and add ALL the rectangles to the image. Something like this (Pseudo):

var image..
var rectangles..
var blobs..

foreach (blob in blobs)
{
    if (blob is rectangle)
    {
        rectangles.add(blob);
    }
}

foreach (r in rectangles)
{
   image.draw(r.points);
}

image.save("result.png");