free form image selection (preferably in c++)

Question

I am new to image manipulation. I have noticed that you can specify a rectangular region of interest and others like circles etc in image manipulation libraries like opencv. Basic paint programs like ms-paint incorporate free form selection but i cannot seem to find a function or tutorial on how to do free form image selection in opencv or other image processing libraries. Any ideas on how to achieve this? PS: My preferred language is c/c++.

Answer 1

I wrote a demo to display an image and paint little green dots while your mouse moves, see below.

You need to know that OpenCV was not designed for this type of interaction, so performance is an issue (and it's bad)! You'll see what I mean.

#include <stdio.h>    
#include <cv.h>
#include <highgui.h>

// mouse callback    
void on_mouse(int event, int x, int y, int flags, void* param)
{
    // Remove comment below to paint only when left mouse button is pressed
    //if (event == CV_EVENT_LBUTTONDOWN)
    {
        //fprintf(stderr, "Painting at %dx%d\n", x, y);
        IplImage* img = (IplImage*)param;

        cvCircle(img, cvPoint(x,y), 1, CV_RGB(0, 255, 0), -1, CV_AA, 0);
            cvShowImage("cvPaint", img);
    }
}


int main(int argc, char* argv[])
{
    if (argc < 2)
    {
        fprintf( stderr, "Usage: %s <img>\n", argv[0]);
        return -1;
    }

    IplImage* frame = NULL; 
    frame = cvLoadImage(argv[1], CV_LOAD_IMAGE_UNCHANGED);
    if (!frame)
    {
        fprintf( stderr, "Failed: Couldn't load file %s\n", argv[1]);
        return -1;
    }

    cvNamedWindow("cvPaint", CV_WINDOW_AUTOSIZE);
    cvShowImage("cvPaint", frame);

    cvSetMouseCallback("cvPaint", &on_mouse, frame);
    while (1)
    {
        // Keep looping to prevent the app from exiting, 
        // so the mouse callback can be called by OpenCV and do some painting

        char key = cvWaitKey(10);
        if (key  == 113 || key == 27) // q was pressed on the keyboard
            break;
    }

    cvReleaseImage(&frame);
    cvDestroyWindow("cvPaint");

    return 0;
}

My suggestion is that you use some other window system for this type of task, where performance is better. Take a look at Qt, for instance. But you can also you platform native ways like win32 or X, if you rather.

For the other part of the question, how to crop on user selection, I suggest you take a look at the code available at: OpenCV resizing and cropping image according to pixel value

Also, recording mouse coordinates while the user is painting the image is much more practical then analyzing the image for the painted green dots. Then analise these coordinates and retrieve the smallest rectangle area from it. That's when this logic gets useful:

CvScalar s;        
for (x=0; x<width-1; x++)
{
    for(y=0; y<height-1; y++)
    {
        s = cvGet2D(binImage, y, x);
        if (s.val[0] == 1)
        {
            minX = min(minX, x);
            minY = min(minY, y);
            maxX = max(maxX, x);
            maxY = max(maxY, y);
        }   
    }
}

cvSetImageROI(binImage, cvRect(minX, minY, maxX-minX, maxY-minY));

In this specific case, instead of iterating through the image looking for specific pixels like the user did in that question, you will iterate over your array of coordinates recorded during mouse movement.

Answer 2

One thing you can try:

If the selection can be represented as a sequence of 2d-vectors, you can think of it as a polygon. Allocate a new 1-channel image that will be your mask and fill it with 0. Then use

void cvFillPoly(CvArr* img, CvPoint** pts, int* npts, int contours, CvScalar color, int lineType=8, int shift=0)

documented on

http://opencv.willowgarage.com/documentation/drawing_functions.html

to draw a non-zero region on the mask image to represent the selected part of the image.