How to add a codec

Question

I am working on the two applications: one runs under Windows, processes proprietary video in real time and sends it over TCPIP; another application receives it in Linux and plays. Right now it works with uncompressed video frames which requires Gigabit network. I would like to compress video. Can you recommend a codec that is (a) available in both Windows and Linux, (b) can be used as API - something live CompressFrame(), UncompressFrame()?

Answer 1

libavcodec, libavformat, and other ffmpeg projects - a wide range of codecs to use.

Windows Media Format SDK - can encode video into WMV streams. Related Windows technologies inlcude DirectShow and Media Foundation.

Ogg Theora - not the greatest quality codec compared, but is free and easy to use. Requires conversion of RGB to YUV. I have sample code for this I could dig up for you on how to do this.

VP8 and WebM from Google.

MJPG - which is nothing more than just encoding each frame as a JPG. High CPU and high bit rate, but is simplest to implement. Any JPG encoder you can find will do.

One note about codec APIs. Intuitively, you just want to have functions that "compress" and "uncompress". (MJPG meets this requirement) But video is more complicated than just compressing individual frames. The majority of compressed frame buffers are based on diffs of the previous frame without encoding the whole image over again. They having tunings for different target bitrates, lossy (UDP) vs. lossless (TCP) networking, frame rates, I-frame interfaces, etc... Some advanced codecs even expect to get QOS information back from the receiver such that they can self-tune their output to match network conditions.

Answer 2

Intel Integrated Performance Primitives get you choices for encoders and decoders, both in Windows and Linux. Another option is ffmpeg/libvacodec.

To be more open for interoperability you might prefer to implement some well known protocol for data streaming over network, such as RTSP/RTP.