CODEWITHSUNDEEP
javaopencvimage-processingbufferedimage
KDecker
KDecker

Reputation: 7148

Loading image from file as given "type" into an OpenCV Mat?

I recently started working with the Java bindings for OpenCV to make a quick and dirty project to do template matching. Basically I am trying to read a set of jpg images (saved in MS Paint) into Mats and then use template matching to find their locations from a screen shot taken with Java.Robot.

When it comes time to do the template matching this error is thrown

OpenCV Error: Assertion failed ((depth == CV_8U || depth == CV_32F) 
&& type == _templ.type() && _img.dims() <= 2) in cv::matchTemplate

After searching it looks like the issue is that the two Mats I am trying to use do not have the same "type". What I am not sure of is what this refers to. I assume it is the Mats CvType, if I print out the CvType of the image and template I get a type() of 4 == CvType.CV_32SC1 for my template I get a type() of 20 == CvType.CV_32SC3.

But I feel like this is not the correct type() I am trying to compare, I have feeling it refers to the data type of how the data is stored in the Mat? But I have no good links to back this up just remembrances from many SO searches.

Here is my code for loading in my jpg images into a Mat

Mat pic_ = Imgcodecs.imread("MyPath\\image.jpg");
pic_.convertTo(pic_, CvType.CV_32SC1);

Here the second line turns my type() from 20 to 16, though as per my last comment I don't think this is the proper way to alter the Mat to match the image?... Because convertTo'ing this Mat to match the type of the screen shot `(below) does not fix the error?

Here is how I am creating the image Mat

Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);

So I first take a screenshot with Java.Robot.createScreenCapture I then convert it to a Mat with

private Mat bufferedImageToMat(BufferedImage inBuffImg) 
{
    BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_INT_RGB);
    Graphics2D g2d= image.createGraphics();
    g2d.drawImage(inBuffImg, 0, 0, null);
    g2d.dispose();

    Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_32SC1);
    int[] data = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
    mat.put(0, 0, data);

    return mat;
}

From what I could tell the BufferedImage created by Robot is of type BufferedImage.TYPE_3BYTE_BGR which gives me an error "DataBufferInt cannot be cast to DataBufferByte" when trying to get the pixel data. So per the linked question I redraw the BufferedImage as type BufferedImage.TYPE_INT_RGB and pull out the data as a DataBufferInt.

So in all, should I be trying to match the Mat.type() or does my problem lie elsewhere? If not elsewhere how can I alter either of the Mats so that they can be used with Imgproc.matchTemplate properly?

I feel like the easiest solution would be to convert the image loaded from file to match the screenshot Mat?

EDIT: The exact section of code that gives the error is below

// Mat imageTemplate is a function argument; the loaded jpg image
// Take a picture of the screen
Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
BufferedImage screenShot = rob.createScreenCapture(screenRect);
Mat screenImage = bufferedImageToMat(screenShot);

// Create the result matrix
int result_cols = screenImage.cols() - imageTemplate.cols() + 1;
int result_rows = screenImage.rows() - imageTemplate.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32SC1);

newStatus("ScreenType: " + screenImage.type());
newStatus("TemplaType: " + imageTemplate.type());

// Choose a matching method
int matchMethod = Imgproc.TM_SQDIFF_NORMED;

// Do the Matching and Normalize
Imgproc.matchTemplate(screenImage, imageTemplate, result, matchMethod);
// Error occurs on previous line

Upvotes: 3

Views: 2107

Answers (1)

KDecker
KDecker

Reputation: 7148

As @Miki pointed out in the comments the answer was getting the channe type to match for the image and template. I ended up changing my bufferedImageToMat function.

private Mat bufferedImageToMat(BufferedImage inBuffImg) 
{
    BufferedImage image = new BufferedImage(inBuffImg.getWidth(), inBuffImg.getHeight(), BufferedImage.TYPE_4BYTE_ABGR);
    Graphics2D g2d= image.createGraphics();
    g2d.drawImage(inBuffImg, 0, 0, null);
    g2d.dispose();

    Mat mat = new Mat(image.getHeight(), image.getWidth(), CvType.CV_8UC3);
    byte[] data = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
    mat.put(0, 0, data);

    return mat;
}

My templates are read in as CvType.CV_8UC3, so it was just a matter of creating a Mat from the screen image with this type!

Upvotes: 4

Related Questions