Live video encoding using...?

Question

I'm attempting to write a fairly simplistic application that will stream video/audio from a webcam to someone else across the internet (ala Skypebut with more control).

There seems to be very little useful/relevant information on the subjectand what I can find is largely outdated. From my research so far, x264 seems to be the way to go as it offers an ultrafast option which is designed for this situation

I'm able to turn on the webcam and receive a stream of images. I can also listen on an audio device and get samples.

Where I'm failing is encoding that information in such a way as to be able to stream with a minimum of latency (from what I've read, 200ms delay is the goal for no obvious lag, including network latency - so let's aim for 100-150ms)

Things I've tried

ffmpeg

This seems to be the most widely used option for encoding. I've had two real issues using it. Firstly, even using x264 with no look-aheads and the bare minimum buffers for stability, the delay seems to be on the order of 700ms using image2pipe. Secondly, it requires ffmpeg to be installed - being able to do this without an external dependency would be nice.

VLC

As with ffmpeg this requires an external program which is a negative. Even worse, I can't seem to get a latency of under 2 seconds which seems to increase over time. I've also only been able to get VLC to capture the camera itself rather than take a stream of images which means I don't get a chance to pre-process them.

DirectShow

I've seen a number of sites recommending using the windows direct show encoders but I haven't been able to find one that works at anything like real time. In fact, the only one I've managed to get going reliably is a Windows Media codec that has a massive latency and fairly large size.

Other considerations

None of the above address the problem of adding an audio stream to the video. I'm not sure if I should attempt to encode them together or send a separate stream alongside the video.

In short, I've been Googling for a week or so now and haven't found a decent way to do this. Can someone please point me at a decent example/guide?

Answer 1

One way would be to use x264vfw via either DirectShow or even Video for Windows. This codec has "zero latency" option where it basically works in "frame in - frame out" mode. I described how I reduced x264 encoding latency in a DirectShow app here.