Opencv, drawing a box around an object from an image

Question

I am doing a project for fun to learn more about OpenCV. At the moment I want to draw a box given an image. I have a working function that achieves the job but I am having some problem regarding passing the mouse_control function into the setMouseCallback function from OpenCV.

The error I am receiving is this:

 error C3867:  'Box::mouse_control': non-standard syntax; use '&' to create a pointer to member

When I try to place the suggested reference I get this error:

error C2276:  '&': illegal operation on bound member function expression

I am brand new to OpenCV so I am not sure exactly how to debug and fix the code I have.

Here is the header file to the class I am working on:

#pragma once
#include "Image.h"

class Box : public Image
{
public:
    Box() = default;


    Mat draw_box(Mat& img, Rect box);
    void mouse_control(int event, int x, int y, int flag, void* param);

private:
    Rect g_rectangle;
    bool g_drawbox;


};

Here is the .cpp file associated to the header file:

#include "Box.h"
#define WINDOW_NAME "Drawing Rectangle"


void Box::mouse_control(int event, int x, int y, int flag, void* param)
{
    Mat& image = *(cv::Mat*) param;

    switch (event) 
    {
    case EVENT_MOUSEMOVE: 
    {    // When mouse moves, get the current rectangle's width and height
        if (g_drawbox) 
        {
            g_rectangle.width = x - g_rectangle.x;
            g_rectangle.height = y - g_rectangle.y;
        }
    }
    break;

    case EVENT_LBUTTONDOWN: 
    {  // when the left mouse button is pressed down,
       // get the starting corner's coordinates of the rectangle
        g_drawbox = true;
        g_rectangle = Rect(x, y, 0, 0);
    }
    break;

    case EVENT_LBUTTONUP: 
    {   // when the left mouse button is released,
        // draw the rectangle
        g_drawbox = false;
        if (g_rectangle.width < 0) 
        {
            g_rectangle.x += g_rectangle.width;
            g_rectangle.width *= -1;
        }

        if (g_rectangle.height < 0) 
        {
            g_rectangle.y += g_rectangle.height;
            g_rectangle.height *= -1;
        }
        draw_box(image, g_rectangle);
    }
    break;
    }
}

Mat Box::draw_box(Mat& img, Rect box)
{
    // Get input from user to draw box around object of interest
    rectangle(img, box.tl(), box.br(), Scalar(0, 255, 255), FILLED, LINE_8);
    Mat tempImage;
    //Mat srcImage(600, 800, CV_8UC3);
    //srcImage = Scalar::all(0);
    namedWindow(WINDOW_NAME);
    setMouseCallback(WINDOW_NAME, mouse_control, (void*)& img);
    while (1) {
        img.copyTo(tempImage);
        if (g_drawbox)
            draw_box(tempImage, g_rectangle);
        imshow(WINDOW_NAME, tempImage);
        if (waitKey(10) == 27)  // stop drawing rectanglge if the key is 'ESC'
            break;
    }
    return img;
}

The error occurs here:

setMouseCallback(WINDOW_NAME, mouse_control, (void*)& img);

Here is the main file and the other header and cpp file needed to duplicate the error:

#include <iostream>
#include "Box.h"
#include "Image.h"


int main()
{

    // Declare image object and the name of the image (store image in project file)
    Image img_obj;
    std::string image_name = "hawaii.png";

    // Read and display image
    auto img1 = img_obj.get_image(image_name);
    img_obj.display(img1);

    Rect g_rectangle;
    Box b;
    auto new_img = b.draw_box(img1, g_rectangle);
    b.display(new_img);


    std::cin.get();
}

#pragma once
#include <opencv2/opencv.hpp>

using namespace cv;


class Image
{
public:
    Image() = default;


    // Member functions
    const Mat get_image(std::string& image_name);
    void display(Mat& img);
};

#include "Image.h"


const Mat Image::get_image(std::string& image_name)
{
    Mat img = imread(image_name);
    return img;
}

void Image::display(Mat& img)
{
    namedWindow("image", WINDOW_NORMAL);
    imshow("image", img);
    waitKey(0);
}

Answer 1

A C++ member function has a different internal signature than that of a regular function. It will implicitly include a pointer to the function signature. Otherwise, how can the function know what object internal data to modify, if the function has to modify some of them?

Thus, the signature of your member function MouseControl() will look like:

void mouse_control(Box *object, int event, int x, int y, int flag, void* param);

Apparently this is not accepted by the OpenCV library (mismatched function pointer type, i.e. the extra Box * object makes it fail.).

To fix it, try one of the two below:

1. Use a regular function rather than a class member function.

2. Use a static class function (class static function is not bound to an object, and thus has no "extra argument in the signature", i.e. Box *object).

Answer 2

Member function and normal function are two different things.

setMouseCallback function takes as callback normal function whose signature is:

handleMouse(int event, int x, int y, int flag, void* param)

you try to call setMouseCallback passing mouse_control as callback, it cannot work because mouse_control is member function of Box class. You call member function for particular object, you cannot call it "standalone" - without object. That is why your code doesn't compile.

Solution?

The last param of callback is void*. Now, you are passing pointer to Image, but you can pass pointer to Box, isn't it? So, you can make mouse_control as static function of Box class:

class Box : public Image {
public:
    Box() = default;
    Mat draw_box(Mat& img, Rect box);
    static void mouse_control(int event, int x, int y, int flag, void* param); // static added

(static function doesn't need to have an object to be called). Pointer to this you pass by calling:

setMouseCallback(WINDOW_NAME, mouse_control, this);

and inside:

void Box::mouse_control(int event, int x, int y, int flag, void* param) {
    Box* box = (Box*)param;

    // you can access member variable of Box by 
    box->g_rectangle
    box->draw_box

also you need to find out how to store img, maybe as data member of Box?. Or you can make some wrapper like

struct Foo {
    Box* box;
    Image img;
};

and pass pointer to this struct as the last parameter of your callback.