CODEWITHSUNDEEP
mp4libpnglibav
xtingray
xtingray

Reputation: 503

How to convert an AVFrame to PNG image

Currently, I am working on a little Libav example to split a small MP4 video into a PNG sequence. So far, this is what I have right now: https://github.com/xtingray/MP4toPNG/blob/main/cutter.c

If you clone the repository on a Unix environment, it's effortless to test the example by running the build.sh script. Most of the implementation is working: I load the MP4 input file, split the video into AVFrame variables, and then use the libpng library to create the PNG files. My only problem is that the images generated are misconfigured and gray-colored, just like this one: https://github.com/xtingray/MP4toPNG/blob/main/output/frame-1.png

I am guessing the error is related to the way I take the AVFrame data and store it for the PNG file, but I am not sure at all:

  png_bytep *row_pointers = (png_bytep *) malloc(sizeof(png_bytep) * frame->height);
  for (int y = 0; y < frame->height; y++) {
     row_pointers[y] = (png_bytep) (frame->data[0] + y * frame->linesize[0]);
  }

Any hint?

Upvotes: 1

Views: 638

Answers (1)

xtingray
xtingray

Reputation: 503

I tried to fix the problem by modifying the PNG creation procedure... trying to include the other color components in the PNG data, with no luck. After that, I tried a different approach: What if the AVFrame format is not the one required by the PNG format? What if I need to make some kind of data translation? Understanding that the MP4 video container holds frames with the YUV420P format, and PNG files contain image data with an RGB structure.

So, I decided to implement this piece of code before creating the PNG files:

// To create the PNG files, the AVFrame data must be translated from YUV420P format into RGB24
struct SwsContext *sws_ctx = sws_getContext(
    pFrame->width, pFrame->height, pFrame->format,
    pFrame->width, pFrame->height, AV_PIX_FMT_RGB24,
    SWS_BILINEAR, NULL, NULL, NULL);

    // Allocate a new AVFrame for the output RGB24 image
    AVFrame* rgb_frame = av_frame_alloc();

    // Set the properties of the output AVFrame
    rgb_frame->format = AV_PIX_FMT_RGB24;
    rgb_frame->width = pFrame->width;
    rgb_frame->height = pFrame->height;

    int ret = av_frame_get_buffer(rgb_frame, 0);
    if (ret < 0) {
        logging("Error while preparing RGB frame: %s", av_err2str(ret));
        return ret;
    }

    logging("Transforming frame format from YUV420P into RGB24...");
    ret = sws_scale(sws_ctx, pFrame->data, pFrame->linesize, 0,   
                    pFrame->height, rgb_frame->data, rgb_frame->linesize);
    if (ret < 0) {
        logging("Error while translating the frame format from YUV420P into RGB24: %s", av_err2str(ret));
        return ret;
    }

As result, the rgb_frame contains the accurate data input required to create the PNG files in a successful way. The final solution is available right here: https://github.com/xtingray/MP4toPNG

Upvotes: 2

Related Questions