CODEWITHSUNDEEP
videoffmpegvideo-streamingvideo-processing
Abdelsalam ElTamawy
Abdelsalam ElTamawy

Reputation: 173

bad ffmpeg performace compared to ffplay and VLC

I've been having bad performance with ffmpeg that I don't see with ffplay and VLC. I find this strange since ffplay uses ffmpeg under the hood.

Simply if I stream an HEVC/h265 stream with ffmpeg and pipe it to ffplay, it would stutter and perform worse than if it played the stream directly with ffplay.

What I'm trying to say is that I expect this command ffplay rtsp://stream-ip to perform the same as ffmpeg -i rtsp://stream-ip -f matroska - | ffplay - but it doesn't, ffplay plays noticeably smoother. Also, if I try to stream it with VLC, it is as smooth as ffplay.

here is the log for ffplay:

$ ffplay rtsp://admin:[email protected]:554/StdCh1
ffplay version n4.4 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 11.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://admin:[email protected]:554/StdCh1':
  Metadata:
    title           : KNS-224C
    comment         : http80 rtsp554 a0 ba0 pt0 rc2 rv2 ua2 tk3 vs3 vd3 pc0 sc0 af0
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt709), 1920x1080, 30.30 tbr, 90k tbn, 90k tbc
  Stream #0:1: Data: none

and here is the log for ffmpeg piped into ffplay:

$ ffmpeg -i rtsp://admin:[email protected]:554/StdCh1 -f matroska - | ffplay -
ffplay version n4.4ffmpeg version n4.4 Copyright (c) 2003-2021 the FFmpeg developers Copyright (c) 2000-2021 the FFmpeg developers

  built with gcc 11.1.0 (GCC)
  built with gcc 11.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavutil      56. 70.100 / 56. 70.100
  libavformat    58. 76.100 / 58. 76.100
  libavcodec     58.134.100 / 58.134.100
  libavdevice    58. 13.100 / 58. 13.100
  libavformat    58. 76.100 / 58. 76.100
  libavfilter     7.110.100 /  7.110.100
  libavdevice    58. 13.100 / 58. 13.100
  libswscale      5.  9.100 /  5.  9.100
  libavfilter     7.110.100 /  7.110.100
  libswresample   3.  9.100 /  3.  9.100
  libswscale      5.  9.100 /  5.  9.100
  libpostproc    55.  9.100 / 55.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://admin:[email protected]:554/StdCh1':
  Metadata:
    title           : KNS-224C
    comment         : http80 rtsp554 a0 ba0 pt0 rc2 rv2 ua2 tk3 vs3 vd3 pc0 sc0 af0
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt709), 1920x1080, 30.30 tbr, 90k tbn, 90k tbc
  Stream #0:1: Data: none
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[hevc @ 0x561e88468480] Could not find ref with POC 0
[libx264 @ 0x561e8845a480] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x561e8845a480] profile High, level 4.2, 4:2:0, 8-bit0
[libx264 @ 0x561e8845a480] 264 - core 161 r3039 544c61f - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'pipe:':
  Metadata:
    title           : KNS-224C
    comment         : http80 rtsp554 a0 ba0 pt0 rc2 rv2 ua2 tk3 vs3 vd3 pc0 sc0 af0
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuvj420p(pc, bt709, progressive), 1920x1080, q=2-31, 30.30 fps, 1k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Input #0, matroska,webm, from 'pipe:': vq=    0KB sq=    0B f=0/0   /A speed=   0x
  Metadata:
    title           : KNS-224C
    COMMENT         : http80 rtsp554 a0 ba0 pt0 rc2 rv2 ua2 tk3 vs3 vd3 pc0 sc0 af0
    ENCODER         : Lavf58.76.100
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1920x1080, 30.30 fps, 30.30 tbr, 1k tbn, 60.61 tbc (default)
    Metadata:
      ENCODER         : Lavc58.134.100 libx264

I'm wondering why is this happening? shouldn't hardware acceleration be applied in both cases?

Beyond this I'm trying to create a video mosaic, the performance problem becomes very clear when displaying the streams together in a command like this:

$ ffmpeg -i rtsp://stream1 \
-i rtsp://stream2 \
-i rtsp://stream3 \
-filter_complex "[0:v] scale=960x540 [a0];
[1:v] scale=960x540 [a1];
[2:v] scale=960x540 [a2];
[a0][a1][a2] xstack=inputs=3:layout=0_0|0_h0|w0_0[out]" \
 -map "[out]" -f matroska - | ffplay -

This performs much worse than trying the same mosaic in VLC. Is there something I'm doing wrong?

I want to note that it performs perfectly fine if I independently streamed all sources with ffplay at the same time.

Upvotes: 0

Views: 2412

Answers (1)

llogan
llogan

Reputation: 133743

Avoid expensive compression:

$ ffmpeg -i rtsp://stream1 \
-i rtsp://stream2 \
-i rtsp://stream3 \
-filter_complex "[0:v] scale=960x540 [a0];
[1:v] scale=960x540 [a1];
[2:v] scale=960x540 [a2];
[a0][a1][a2] xstack=inputs=3:layout=0_0|0_h0|w0_0[out]" \
 -map "[out]" -c:v rawvideo -f nut - | ffplay -

Even better, use ffplay (or mpv) directly:

ffplay -autoexit -f lavfi "movie=rtsp\\\://stream1,scale=960x540[v1];movie=rtsp\\\://stream2,scale=960x540[v2];movie=rtsp\\\://stream3,scale=960x540[v3];[v0][v1][v2]xstack=inputs=3:layout=0_0|0_h0|w0_0[out0]"

Upvotes: 2

Related Questions