CODEWITHSUNDEEP
c#.net.net-core
Stevie
Stevie

Reputation: 2130

How to Parse an MPEG Video Stream using .NET Core

For a personal project, I am attempting to read from a IP Camera's MPEG stream and perform some computer-vision tasks on the individual frames I receive (Using .NET Core 2.2).

I perform a GET request to the Camera's MPEG endpoint and I'm returned a multipart/x-mixed-replace response that continually streams what appear to be individual frames as JPEG images.

What I'm having trouble understanding is the correct way to parse out the frames from the multi-part response - how the frames are retrieved is unimportant to me, if there's a better way to parse this kind of stream, I'm certainly open to change - video retrieval/processing is a whole new world for me :)

For reference:

MPEG Endpoint:

GET http://192.168.0.14/mjpeg.cgi

Example Response Header:

Content-Type: multipart/x-mixed-replace;boundary=--video boundary--

Example Response Body:

Content-length: 41142
Date: 02-02-2019 12:43:19 AM
Content-type: image/jpeg

[payload]

--video boundary--
Content-length: 41220
Date: 02-02-2019 12:43:19 AM
Content-type: image/jpeg

[payload]

--video boundary--

What I have so far:

var client = new HttpClient() {
    BaseAddress = new Uri(streamUrl)
};

var stream = await client.GetStreamAsync(resource);

using (var streamReader = new StreamReader(stream)) {                
    while(true) {
        await GetFrameStart(streamReader);
        var frame = await ReadFrame(streamReader);

        // Get byte[] from returned frame above and construct image
    };
}

// ---

// Fast forward to the start of the Frame's bytes
static async Task GetFrameStart(StreamReader reader) {
    string buffer = null;
    while (buffer != string.Empty) {
        buffer = await reader.ReadLineAsync();
    }
}

// Read entire byte array until the video boundary is met
static async Task<String> ReadFrame(StreamReader reader) {
    string result = string.Empty;
    string line = null;

    while(!isBoundary(line)) {
        line = await reader.ReadLineAsync();
        if (!isBoundary(line)) result += line;
    };

    return result;
}

The above seems to be giving me back the individual [payload] of each frame, however if I when I convert the string to bytes and write to disk, the image is not a valid jpeg:

var bytes = Encoding.UTF8.GetBytes(frame);

using (var fs = new FileStream("test.jpeg", FileMode.Create, FileAccess.Write)) {
    fs.Write(bytes, 0, bytes.Length);
}

Upvotes: 4

Views: 2731

Answers (2)

timpur
timpur

Reputation: 11

You could use Microsoft.AspNetCore.WebUtilities.MultipartReader

 var boundary = GetBoundary(context.Request.ContentType);
 var reader = new MultipartReader(boundary, context.Request.Body);
 var section = await reader.ReadNextSectionAsync();

See this Answer for more info:

Hope this helps.

Upvotes: 1

codernr
codernr

Reputation: 121

You could use ffmpeg with a single line:

ffmpeg -i http://192.168.0.14/mjpeg.cgi -vcodec copy frame%d.jpg

This extracts all the frames with numbered names.

If you install ffmpeg, you can call it from your application with System.Diagnostics.Process like this:

var process = new Process()
{
    StartInfo = new ProcessStartInfo()
    {
        FileName = "ffmpeg",
        Arguments = "-i http://192.168.0.14/mjpeg.cgi -vcodec copy frame%d.jpg",
        UseShellExecute = false,
        CreateNoWindow = true,
        RedirectStandardOutput = true,
        RedirectStandardError = true
    },
    EnableRaisingEvents = true
};

process.ErrorDataReceived += (sender, data) => Console.WriteLine(data.Data);

process.Start();
process.BeginErrorReadLine();

Upvotes: 0

Related Questions