Desktop grabbing with FFmpeg at 60 fps using NVENC codec

Question

I'm having trouble recording my desktop at 60FPS using the latest Windows compiled FFmpeg with NVENC codec. Metadata says the file is 60 fps but when I play it, I can see clearly see it is not 60FPS.

The command-line I use is the following:

ffmpeg -y -rtbufsize 2000M -f gdigrab -framerate 60 -offset_x 0 -offset_y 0 -video_size 1920x1080 -i desktop -c:v h264_nvenc -preset:v fast -pix_fmt nv12 out.mp4

I tried using a real time buffer, using another DirectShow device, changing the profile or forcing a bitrate, but the video always seems to be at 30fps.

Recording the screen using NVIDIA's ShadowPlay works well, so I know it's feasible on my machine.

Using FFprobe to check the ShadowPlay's output file I can see:

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt470m), 1920x1080 [SAR 1:1 DAR 16:9], 4573 kb/s, 59.38 fps, 240 tbr, 60k tbn, 120 tbc (default)

But If I force my output to have the same bitrate and profile I get:

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 5519 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default)

I can see tbr and tbn are different, so I know my output is duplicating frames.

For testing, all of my recordings had this 60 frame rate test page on the background, and I could clearly see the differences.

I know ShadowPlay probably does a lot more under the hood than FFmpeg using the same codec. I know OBS can do it quite easily but I want to understand what I am doing wrong. Maybe it's some FFmpeg limitation ?

Full console output

Using -v trace command:

[gdigrab @ 0000000002572cc0] Capturing whole desktop as 1920x1080x32 at (0,0)
[gdigrab @ 0000000002572cc0] Cursor pos (1850,750) -> (1842,741)
[gdigrab @ 0000000002572cc0] Probe buffer size limit of 5000000 bytes reached
[gdigrab @ 0000000002572cc0] Stream #0: not enough frames to estimate rate; consider increasing probesize
[gdigrab @ 0000000002572cc0] stream 0: start_time: 1467123648.275 duration: -9223372036854.775
[gdigrab @ 0000000002572cc0] format: start_time: 1467123648.275 duration: -9223372036854.775 bitrate=3981337 kb/s
Input #0, gdigrab, from 'desktop':
  Duration: N/A, start: 1467123648.275484, bitrate: 3981337 kb/s
    Stream #0:0, 1, 1/1000000: Video: bmp, 1 reference frame, bgra, 1920x1080 (0x0), 0/1, 3981337 kb/s, 60 fps, 1000k tbr, 1000k tbn, 1000k tbc
Successfully opened the file.
Parsing a group of options: output file out.mp4.
Applying option c:v (codec name) with argument h264_nvenc.
Applying option pix_fmt (set pixel format) with argument nv12.
Successfully parsed a group of options.
Opening an output file: out.mp4.
[file @ 0000000000e3a7c0] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 8 logical cores
[graph 0 input from stream 0:0 @ 000000000257ec00] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 000000000257ec00] Setting 'pix_fmt' to value '30'
[graph 0 input from stream 0:0 @ 000000000257ec00] Setting 'time_base' to value '1/1000000'
[graph 0 input from stream 0:0 @ 000000000257ec00] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 000000000257ec00] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 000000000257ec00] Setting 'frame_rate' to value '60/1'
[graph 0 input from stream 0:0 @ 000000000257ec00] w:1920 h:1080 pixfmt:bgra tb:1/1000000 fr:60/1 sar:0/1 sws_param:flags=2
[format @ 000000000257ffc0] compat: called with args=[nv12]
[format @ 000000000257ffc0] Setting 'pix_fmts' to value 'nv12'
[auto-inserted scaler 0 @ 00000000025802c0] Setting 'flags' to value 'bicubic'
[auto-inserted scaler 0 @ 00000000025802c0] w:iw h:ih flags:'bicubic' interl:0
[format @ 000000000257ffc0] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 0000000000e373c0] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
[auto-inserted scaler 0 @ 00000000025802c0] w:1920 h:1080 fmt:bgra sar:0/1 -> w:1920 h:1080 fmt:nv12 sar:0/1 flags:0x4
[h264_nvenc @ 0000000000e3ca20] Nvenc initialized successfully
[h264_nvenc @ 0000000000e3ca20] 1 CUDA capable devices found
[h264_nvenc @ 0000000000e3ca20] [ GPU #0 - < GeForce GTX 670 > has Compute SM 3.0 ]
[h264_nvenc @ 0000000000e3ca20] supports NVENC
[mp4 @ 0000000000e3b580] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf57.40.101
    Stream #0:0, 0, 1/15360: Video: h264 (h264_nvenc) (Main), 1 reference frame ([33][0][0][0] / 0x0021), nv12, 1920x1080, 0/1, q=-1--1, 2000 kb/s, 60 fps, 15360 tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.47.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (bmp (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Clipping frame in rate conversion by 0.000008
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[gdigrab @ 0000000002572cc0] Cursor pos (1850,750) -> (1842,741)
*** 35 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1850,750) -> (1842,741)
*** 7 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1850,649) -> (1850,649)
*** 1 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1858,535) -> (1858,535)
*** 3 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1859,454) -> (1859,454)
*** 2 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1865,384) -> (1865,384)
*** 2 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1846,348) -> (1846,348)
*** 3 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1770,347) -> (1770,347)
*** 2 dup!
[gdigrab @ 0000000002572cc0] Cursor pos (1545,388) -> (1545,388)
*** 4 dup!
frame=   69 fps=0.0 q=35.0 size=     184kB time=00:00:00.63 bitrate=2384.0kbits/[gdigrab @ 0000000002572cc0] Cursor pos (1523,389) -> (1519,378)

Answer 1

At present, FFmpeg supports ddagrab filter which can be used to set an output frame rate via the private framerate option.

This new ddagrab filter exclusively returns D3D11 hardware frames, for on-gpu encoding or processing, and as such, any form of software processing will require a hwdownload invocation, and should thus be significantly faster than the older GDI capture method(s).

See the example below, which captures the entire desktop:

ffmpeg -fflags +genpts -init_hw_device d3d11va -threads:v 1 ^
-filter_complex "ddagrab=0:framerate=60:draw_mouse=0:video_size=1920x1080" ^
-c:v h264_nvenc -rc:v vbr -tune:v ll -preset:v p2 ^
-rc-lookahead:v 0 -delay:v 0 -b:v 0 -cq:v 19 -zerolatency:v 1 ^
-y -f mp4 "out.mp4"

You may print out ddagrab usage options via:

ffmpeg -h filter=ddagrab

Sample output:

Filter ddagrab
  Grab Windows Desktop images using Desktop Duplication API
    Inputs:
        none (source filter)
    Outputs:
       #0: default (video)
ddagrab AVOptions:
   output_idx        <int>        ..FV....... dda output index to capture (from 0 to INT_MAX) (default 0)
   draw_mouse        <boolean>    ..FV....... draw the mouse pointer (default true)
   framerate         <video_rate> ..FV....... set video frame rate (default "30")
   video_size        <image_size> ..FV....... set video frame size
   offset_x          <int>        ..FV....... capture area x offset (from INT_MIN to INT_MAX) (default 0)
   offset_y          <int>        ..FV....... capture area y offset (from INT_MIN to INT_MAX) (default 0)
   output_fmt        <int>        ..FV....... desired output format (from 0 to INT_MAX) (default 8bit)
     auto            0            ..FV....... let dda pick its preferred format
     8bit            87           ..FV....... only output default 8 Bit format
     bgra            87           ..FV....... only output 8 Bit BGRA
     10bit           24           ..FV....... only output default 10 Bit format
     x2bgr10         24           ..FV....... only output 10 Bit X2BGR10
     16bit           10           ..FV....... only output default 16 Bit format
     rgbaf16         10           ..FV....... only output 16 Bit RGBAF16
   allow_fallback    <boolean>    ..FV....... don't error on fallback to default 8 Bit format (default false)
   force_fmt         <boolean>    ..FV....... exclude BGRA from format list (experimental, discouraged by Microsoft) (default false)

The ffmpeg snippet above already takes into consideration the omission of a specific bitrate as set in your case above, adhering to an appropriate rate control method with the lowest latency possible.

Additional references:

Kindly refer to this answer on the various rate control and presets available on specific FFmpeg NVENC implementations, and their impact on throughput.Your mileage may vary, based on driver and ffmpeg componenent versions.

For outputs that require strict output size controls, see this answer on the impact of combining various rate control methods with bitrate values, and the justification(s) for the options used in the example command provided above.

Answer 2

Assuming your DirectShow filter uses GDI, it is highly likely the entire desktop cannot be captured at 60 fps (Windows is not very good at that). FFmpeg probably doubles some images to end up with 60 fps. DXGI should give better performance (https://msdn.microsoft.com/en-us/library/windows/desktop/hh404487(v=vs.85).aspx) but I am not aware FFmpeg has built-in support for this. Yet?