Creating MJEPG video from multiple JPEG encoded images without using cv::imdecode()

Question

I need to store multiple encoded frames in memory.
I am using cv::imencode(".jpg", ...) for compressing and storing the encoded images to std::list<std::vector<u_char>> compressed_images - a list of compressed images.

I want to create a video from compressed_images, but I must use cv::imdecode() to decode all the images to cv::Mat, then use cv::VideoWriter to save the images to MJPEG video.

Can I skip cv::imdecode(), or use other solution for avoid encoding two times?

Answer 1

You may PIPE the encoded images to FFmpeg.

According to the following post, you can "simply mux the JEPG images to make a video".

In case the frames are in memory, you can write the encoded images to an input PIPE of FFmpeg.

Instead of -f image2, use -f image2pipe format flag.

Implementing the solution in C++ is too difficult for me.
I have implemented a Python code sample.

The code sample:

• Builds a list of 100 encoded frames (green frame counter).
• PIPE the encoded frames to ffmpeg sub-process.
  The encoded images are written to stdin input stream of the sub-process.

Here is the Python code sample:

import numpy as np
import cv2
import subprocess as sp

# Generate 100 synthetic JPEG encoded images in memory:
###############################################################################
# List of JPEG encoded frames.
jpeg_frames = []

width, height, n_frames = 640, 480, 100  # 100 frames, resolution 640x480

for i in range(n_frames):
    img = np.full((height, width, 3), 60, np.uint8)
    cv2.putText(img, str(i+1), (width//2-100*len(str(i+1)), height//2+100), cv2.FONT_HERSHEY_DUPLEX, 10, (30, 255, 30), 20)  # Green number

    # JPEG Encode img into jpeg_img
    _, jpeg_img = cv2.imencode('.JPEG', img)

    # Append encoded image to list.
    jpeg_frames.append(jpeg_img)
###############################################################################

#FFmpeg input PIPE: JPEG encoded images
#FFmpeg output AVI file encoded with MJPEG codec.
# https://video.stackexchange.com/questions/7903/how-to-losslessly-encode-a-jpg-image-sequence-to-a-video-in-ffmpeg
process = sp.Popen('ffmpeg -y -f image2pipe -r 10 -i pipe: -codec copy out.avi', stdin=sp.PIPE)

# Iterate list of encoded frames and write the encoded frames to process.stdin
for jpeg_img in jpeg_frames:
    process.stdin.write(jpeg_img)

# Close and flush stdin
process.stdin.close()

# Wait one more second and terminate the sub-process
try:
    process.wait(1)
except (sp.TimeoutExpired):
    process.kill()

---

Update: C++ implementation:

#include "opencv2/opencv.hpp"
#include "opencv2/highgui.hpp"

#ifdef _MSC_VER
#include <Windows.h>    //For Sleep(1000)
#endif

#include <stdio.h>

int main()
{
    int width = 640;
    int height = 480;
    int n_frames = 100;

    //Generate 100 synthetic JPEG encoded images in memory:
    //////////////////////////////////////////////////////////////////////////
    std::list<std::vector<uchar>> jpeg_frames;

    for (int i = 0; i < n_frames; i++)
    {
        cv::Mat img = cv::Mat(height, width, CV_8UC3);
        img = cv::Scalar(60, 60, 60);

        cv::putText(img, std::to_string(i + 1), cv::Point(width / 2 - 100 * (int)(std::to_string(i + 1).length()), height / 2 + 100), cv::FONT_HERSHEY_DUPLEX, 10, cv::Scalar(30, 255, 30), 20);  // Green number

        //cv::imshow("img", img);cv::waitKey(1);

        std::vector<uchar> jpeg_img;

        cv::imencode(".JPEG", img, jpeg_img);

        jpeg_frames.push_back(jpeg_img);
    }
    //////////////////////////////////////////////////////////////////////////

    //In Windows (using Visual Studio) we need to use _popen and in Linux popen
#ifdef _MSC_VER
    //ffmpeg.exe must be in the system path (or in the working directory)
    FILE *pipeout = _popen("ffmpeg -y -f image2pipe -r 10 -i pipe: -codec copy out.avi", "wb"); //For Windows use "wb"
#else
    //https://batchloaf.wordpress.com/2017/02/12/a-simple-way-to-read-and-write-audio-and-video-files-in-c-using-ffmpeg-part-2-video/
    FILE *pipeout = popen("ffmpeg -y -f image2pipe -r 10 -i pipe: -codec copy out.avi", "w"); //For Linux use "w"

    //In case ffmpeg is not in the execution path, you may use full path:
    //popen("/usr/bin/ffmpeg -y -f image2pipe -r 10 -i pipe: -codec copy out.avi", "w");
#endif

    std::list<std::vector<uchar>>::iterator it;

    //Iterate list of encoded frames and write the encoded frames to pipeout
    for (it = jpeg_frames.begin(); it != jpeg_frames.end(); ++it) 
    {
        std::vector<uchar> jpeg_img = *it;

        // Write this frame to the output pipe
        fwrite(jpeg_img.data(), 1, jpeg_img.size(), pipeout);
    }

    // Flush and close input and output pipes
    fflush(pipeout);

#ifdef _MSC_VER
    _pclose(pipeout);
#else
    pclose(pipeout);
#endif

    //It looks like we need to wait one more second at the end.
    Sleep(1000);

    return 0;
}