CODEWITHSUNDEEP
androidc++opencvmemorymemory-management
cjurjiu
cjurjiu

Reputation: 1798

cv::cvtColor without allocating memory issue

I am trying to convert a 320x240 image from NV21 into a RGBA one using cv::cvtColor, without allocating new memory.

So I tried various ways of achieving this. I know that cv::cvtColor calls cv::Mat.create(..) to allocate memory for the output Mat, and I've seen that cv::Mat.create(...) only allocates new memory if the size & type of the requested Mat differs from its current values.

Given, these, the following:

//(1)
cv::Mat frame(height + height / 2, width, CV_8UC1, pointerToNV21Data);
//(2)
cv::cvtColor(frame, frame, cv::COLOR_YUV2RGBA_NV21);

Allocates a new continuous memory region of 320 * 240 * 4 channels = 307200 bytes (previously free memory), and discards the previously allocated 320 * 360 * 1 channel = 115200 bytes.

When cv::cvtColor internally calls cv::Mat.create(...), the new memory block is allocated because:

So my most optimistic attempt to solving this was to do the following:

//allocate a byte buffer of 307200 bytes
uchar largeBuffer[307200];

//make sure the 115200 NV21 image bytes are written to the buffer
writeNV21DataToBuffer(largeBuffer);

//create cv::Mat that wraps "largeBuffer" as a NV21 image
cv::Mat frameNV21(height + height / 2, width, CV_8UC1, largeBuffer);

//create a second cv::Mat that wraps around the same "largeBuffer", 
//but this time consider it as pointing to a RGBA image
cv::Mat frameRGBA(height, width, CV_8UC4, largeBuffer);

//call cv::cvtColor
cv::cvtColor(frameNV21, frameRGBA, cv::COLOR_YUV2RGBA_NV21);

By doing this, indeed no new memory is allocated. Printing cv::Mat.dataStart & cv::Mat.dataStart of frameNV21, frameRGBA before cv::cvtColor and frameRGBA after cv::cvtColor confirms that the same memory region was used, and that no new memory was allocated for the result.

However...the conversion itself is not done properly, the result looking like this, when displayed on the screen:

enter image description here

At this point, I'm out of ideas. Is my last approach inherently wrong, or is there no way to achieve this?

Upvotes: 3

Views: 2334

Answers (2)

Costantino Grana
Costantino Grana

Reputation: 3428

Yes, you can use cvtColor without having it allocate new memory: just use a larger buffer:

//allocate a byte buffer of 307200 + 115200 bytes
uchar largeBuffer[307200 + 115200];

//make sure the 115200 NV21 image bytes are written to the buffer
writeNV21DataToBuffer(largeBuffer + 307200);

//create cv::Mat that wraps "largeBuffer + 307200" as a NV21 image
cv::Mat frameNV21(height + height / 2, width, CV_8UC1, largeBuffer + 307200);

//create a second cv::Mat that wraps around the same "largeBuffer", 
//but this time consider it as pointing to a RGBA image
cv::Mat frameRGBA(height, width, CV_8UC4, largeBuffer);

//call cv::cvtColor
cv::cvtColor(frameNV21, frameRGBA, cv::COLOR_YUV2RGBA_NV21);

If your question instead was: "can I have cvtColor work inplace?", meaning without additional memory for storing the source bitmap, the answer is no. At least not with cvtColor.

The problem is that you need the data you are overwriting with cvtColor. In fact if everything was done sequentially, you could have just put the UV part of the image outside the RGBA data and save width * height bytes. (Un)fortunately OpenCV is performing cvtColor in parallel, so you will see errors at strange spots (likely you will have some correctly converted horizontal stripes and some totally wrong ones). So you will need to have a different buffer.

The conversion can be done inplace, but is much more slow, since you will need to rearrange your data.

Upvotes: 2

Hakes
Hakes

Reputation: 621

This is more like a comment than an answer however I couldnot fir it in a comment. So here we go: I have no NV21 encoder in OpenCV it can only decode, so i tried this code:

cv::Mat ssOrg = cv::imread( "/home/user/ss.png");

cv::Mat ssGray;
cv::cvtColor( ssOrg, ssGray, CV_BGR2YUV_I420);
cv::resize( ssGray, ssGray, cv::Size(320, 360));

uchar largeBuffer[307200];
cv::Mat frameNV21( 360, 320, CV_8UC1, largeBuffer);
cv::Mat frameRGBA( 320, 240, CV_8UC4, largeBuffer);

const uchar * nv21Ptr = ssGray.ptr();
for ( int i = 0; i < 360 * 320; ++i){
    largeBuffer[i] = *nv21Ptr++;
}
cv::cvtColor(frameNV21, frameRGBA, CV_YUV2BGRA_I420);
cv::imshow( "rgb", frameRGBA);

cv::waitKey();

This code did work, so I think your approach is correct. However when I changed the line cv::cvtColor(frameNV21, frameRGBA, CV_YUV2BGRA_I420); to cv::cvtColor(frameNV21, frameRGBA, CV_YUV2BGRA_NV21); I got some artifacts on the image. Can you check if you can convert back from NV21 format? Also I am testing on PC and you are testing on android.

Upvotes: 1

Related Questions