CODEWITHSUNDEEP
javascriptnode.jsmp4
Jimmy Leahy
Jimmy Leahy

Reputation: 666

Extract RGB frame data from MP4 in NodeJS

I have seen multiple questions similar to this but have not found a satisfactory answer.

Is there any possible way to extract video data from a NodeJS process? More specifically I am trying to extract frame data from videos. I am looking for a single-dimensional array of RGB data for every frame. I was using opencv4nodejs but that is extreme overkill and I am having trouble including the library with Electron.

Any ideas?

Upvotes: 2

Views: 1194

Answers (1)

John Allard
John Allard

Reputation: 3914

You might think that openCV is overkill but it really isn't. Getting RGB data from a video means decoding that video, and decoding a video is one of the more complex things that you can do with a computer (in the general case). If you want to be able to handle a standard H.264-encoded video inside of an mp4 container you're going to need something like GStreamer or FFMPEG and their associated libraries, OpenCV relies on LibAVCodec and LibAVFormat I believe (these are the libraries that underly FFMPEG).

There is no "simple" way to get RGB data of an arbitrary video without having a video decoding library and those are not going to be JS-native, they're going to be monolithic and written in C.

I was going to say you could look for something like this (https://github.com/fluent-ffmpeg/node-fluent-ffmpeg), but you're actually probably best off using OpenCV if you want a literal multi-dimensional array of RGB data. If you're trying to do encoding/decoding/remuxing, then take a look at the FFMPEG plugins for NodeJS.

Upvotes: 7

Related Questions