CODEWITHSUNDEEP
c#c++opencv
user1858884
user1858884

Reputation: 169

OpenCV create Mat from byte array

In my C++ dll I am creating Mat from byte array:

BYTE * ptrImageData;  //Image data is in this array passed to this function

Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);

The image is created with some gray shade not the original one.

Is this the proper way of creating Mat from byte array?

Please see code

ptrImageData is passed to the C++ dll from C# code.

C# code to pass the image data

System.Drawing.Image srcImage //Has the image
MemoryStream ms = new MemoryStream(); 
Marshal.FreeHGlobal(ptrImageData); 
srcImage.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg); 
byte[] imgArray = ms.ToArray();
ms.Dispose();


int size1 = Marshal.SizeOf(imgArray[0]) * imgArray.Length;
IntPtr ptrImageData = Marshal.AllocHGlobal(size1);
Marshal.Copy(imgArray, 0, ptrImageData, imgArray.Length);

//Calling C++ dll function
ProcessImage(ptrImageData, srcImage.Width, srcImage.Height);

Marshal.FreeHGlobal(ptrImageData);

Upvotes: 15

Views: 32344

Answers (3)

gavinb
gavinb

Reputation: 20018

The C++ code appears ok, in that this creates a matrix wrapper for the supplied image data, assuming the buffer is in the conventional RGB8 format. Note that this constructor does not copy the buffer, so the buffer must remain valid for the duration of this Mat instance (or be copied).

Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);

It appears the problem lies in Your C# code. I am not a C# developer, but I will do my best to help. You are creating a memory stream and using the JPEG codec to write a compressed version of the image into the buffer as if it were a file. But that is not the data format that cv::Mat is expecting, so you will basically see garbage (compressed data interpreted as uncompressed).

Given a System.Image.Drawing.Image instance, you can create a wrapper Bitmap object directly (or maybe use as, since it is a simple downcast). Then you can just use the Bitmap.LockBits() method tog obtain a pointer to the underlying image data.

Bitmap bmp = new Bitmap(sourceImage);

// Lock the bitmap's bits.  
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.BitmapData bmpData =
    bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite,
    bmp.PixelFormat);

// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;

// Declare an array to hold the bytes of the bitmap.
int bytes  = Math.Abs(bmpData.Stride) * bmp.Height;
byte[] rgbBuffer = new byte[bytes];

// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbBuffer, 0, bytes);

// Do your OpenCV processing...
// ...

// Unlock the bits.
bmp.UnlockBits(bmpData);

and then you can pass the rgbBuffer to OpenCV.

I'm not convinced that the memory management in the original code is entirely correct either, but anyway the above will work provided the scope of the buffer ownership is within the lock and unlock method calls. If the image data is to outlive this code block, you will have to copy the buffer.

Be careful with your pixel formats too - you need to make sure the Image/Bitmap instance really contains RGB8 data. OpenCV's cv::Mat has various flags so you can work with a variety of in-memory image formats. But note that these are not the same as the on-disk (typically compressed) formats, such as PNG, TIFF, and so forth.

Upvotes: 5

Vlad
Vlad

Reputation: 1680

Here is link to docs: http://docs.opencv.org/modules/core/doc/basic_structures.html#mat-mat

In general you should take care about two things:

  1. When you pass external data into matrix constructor, the external data is not automatically deallocated, so you should take care of it. If you want OpenCV matrix to care about memory, then you should copy matrix (you can do it in many ways, e.g. using Mat::clone or Mat::copyTo methods.
  2. External data may not be continuous, i.e. size of row may be bigger than width multiplied by number of channels multiplied by size of data element. So you may want specify "step" as last argument of constructor. If you allocate external data manually and 100% sure that it is continuous, then you may not pass step and rely on automatic step calculation.

I am not familiar with C#, but it seems to me that you release data right after ProcessImage call. So if ProcessImage is asynchronous or somehow caches your matrix (i.e. lifetime of matrix is longer that ProcessImage call), then you should care about memory management.

Upvotes: 0

M Miller
M Miller

Reputation: 1

Yes, this is one way to create a Mat from a byte array. You just have to be careful that your array contains what you think it does.

The image is created with some gray shade not the original one.

So you are getting an image in newImg? What was the pixel format of the original data?

Maybe you've switched the red and blue channels. The following line will swap the channels:

cv::cvtColor(newImg,swappedImg,CV_RGB2BGR);

Upvotes: 0

Related Questions