Finding the bottleneck on TV capture RTMP latency

Question

I'm trying to locally stream a TV capture from one computer to another but my latency is higher than I want it to be.

My setup is a 12GB, i5 x4 3.2ghz, Geforce 970, with an Elgato HD60 Pro capture card. This machine is running an Ubuntu instance with Nginx+RTMP (https://github.com/arut/nginx-rtmp-module) installed.

It comes with capture/streaming software that allows you to adjust the bandwidth+resolution. It's set to stream to the local RTMP at rtmp://192.168.1.200/capture.

On my receiving machine, I've tried using VLC (Open network) and also FFPLAY (ffplay -fflags nobuffer rtmp://192.168.1.200/capture -loglevel verbose).

The FFPLAY has less latency than the VLC which seems to make sense considering the nobuffer flag. However it's still about 2-3 seconds until I see the correct updates.

• Response time on the capture preview is almost immediate (~100ms maybe)
• Response time from the FFPLAY stream is between 2-3 seconds

I imagine that means there's a bottleneck between Elgato and RTMP server OR between the RTMP server and my ffplay stream OR both.

Things I've tried:

• Increase Mbps from capture software between 1.00 and 8.00
• Decrease quality of capture from highest to lowest
• Decrease resolution of capture from 1080, to 720, to standard
• Decrease framerate from 60fps to 30fps
• Use FFPLAY instead of VLC

Note: There are no special options in my RTMP NGINX config. It's a standard live on and that's pretty much it.

What's the best approach to diagnose where the issue is? I'd like to get it < 1s.

Thanks!

Answer 1

My guess is the video encoding and the video decoding introduces the largest delay. Let's assume your compressing to H.264 (AVC) - I'd turn off B-frames.

Concerning the diagnostics - I'd run Wireshark on the client machine. Make sure client and server clocks are in sync. Then I'd compare the timestamps from the RTMP stream with the client's clock. This should give you an idea about the encoding delay. Total delay minus encoding gives you decoding delay. You can probably neglect the network buffering and the transmission delay.

It is an interesting problem and the industry offers some solutions for it - for example http://www.ineoquest.com/.

Answer 2

I think in your case the bottleneck is video processing. Usually, software video encoding is not so fast. If you decreasing the video quality, bitrate etc, you add more processing and just increasing the delay. If you need to send video not too far from the source, just use video interface like HDMI. It will pass video how it is (without processing) If you really interested in network streaming consider using special hardware encoder devices, like PCI or external ones. Those devices have embedded hardware video encoders and do not use CPU for video processing. Usually with hardware encoders delay is less than 1s, but it is still true statement: "more video processing = more delay"