Reputation: 756
Why does the frame count change when scaling with FFmpeg?
I use this to scale 1920x1080 H.264 videos:
ffmpeg -i IMG_1438.MOV -threads 2 -vf scale=-2:600 IMG_1438_scaledTo600.MOV
And it works great! But here is my question: most of the time, the frame rate stays exactly the same between the original file and the scaled file. For example:
$ mediainfo -F IMG_1426.MOV | grep Frame\ rate
Frame rate : 29.970
Frame rate : 29.970 FPS
Frame rate mode : VFR
Frame rate mode : Variable
Frame rate : 29.970
Frame rate : 29.970 (29970/1000) FPS
$ mediainfo -F IMG_1426_scaledTo600.MOV | grep Frame\ rate
Frame rate : 29.970
Frame rate : 29.970 FPS
Frame rate mode : CFR
Frame rate mode : Constant
Frame rate : 29.970
Frame rate : 29.970 (30000/1001) FPS
But sometimes, the frame rate increases dramatically:
$ mediainfo -F IMG_1438.MOV | grep Frame\ rate
Frame rate : 25.044
Frame rate : 25.044 FPS
Frame rate mode : VFR
Frame rate mode : Variable
Frame rate : 25.044
Frame rate : 25.044 FPS
$ mediainfo -F IMG_1438_scaledTo600.MOV | grep Frame\ rate
Frame rate : 120.000
Frame rate : 120.000 FPS
Frame rate mode : CFR
Frame rate mode : Constant
Frame rate : 120.000
Frame rate : 120.000 FPS
What should I know about FFmpeg or libx264 or libswscale that will help me understand why this happens? (Hoping to hear from LordNeckbeard, in particular).
mediainfo IMG_1438.MOV --Full outputs:
General
Count : 327
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Count of video streams : 1
Count of audio streams : 1
OtherCount : 2
Video_Format_List : AVC
Video_Format_WithHint_List : AVC
Codecs Video : AVC
Audio_Format_List : AAC
Audio_Format_WithHint_List : AAC
Audio codecs : AAC LC
Complete name : IMG_1438.MOV
File name : IMG_1438
File extension : MOV
Format : MPEG-4
Format : MPEG-4
Format/Extensions usually used : mp4 m4v m4a m4b m4p 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma f4v
Commercial name : MPEG-4
Format profile : QuickTime
Internet media type : video/mp4
Codec ID : qt
Codec ID : qt 0000.00 (qt )
Codec ID/Url : http://www.apple.com/quicktime/download/standalone.html
CodecID_Version : 0000.00
CodecID_Compatible : qt
Codec : MPEG-4
Codec : MPEG-4
Codec/Extensions usually used : mp4 m4v m4a m4b m4p 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma f4v
File size : 113990140
File size : 109 MiB
File size : 109 MiB
File size : 109 MiB
File size : 109 MiB
File size : 108.7 MiB
Duration : 52268
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 00:00:52.268
Duration : 00:00:52:09
Duration : 00:00:52.268 (00:00:52:09)
Overall bit rate : 17447026
Overall bit rate : 17.4 Mb/s
Frame rate : 25.044
Frame rate : 25.044 FPS
Frame count : 1309
Stream size : 56670
Stream size : 55.3 KiB (0%)
Stream size : 55 KiB
Stream size : 55 KiB
Stream size : 55.3 KiB
Stream size : 55.34 KiB
Stream size : 55.3 KiB (0%)
Proportion of this stream : 0.00050
HeaderSize : 28
DataSize : 113966271
FooterSize : 23841
IsStreamable : No
Encoded date : UTC 2016-10-08 22:51:19
Tagged date : UTC 2016-10-08 22:52:12
File last modification date : UTC 2016-10-08 22:51:19
File last modification date (local) : 2016-10-08 17:51:19
Writing library : Apple QuickTime
Writing library : Apple QuickTime
Encoded_Library_Name : Apple QuickTime
com.apple.quicktime.make : Apple
com.apple.quicktime.model : iPhone 5
com.apple.quicktime.software : 10.0.2
com.apple.quicktime.creationdate : 2016-10-08T17:51:19-0500
Video
Count : 334
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
StreamOrder : 0
ID : 1
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format/Url : http://developers.videolan.org/x264.html
Commercial name : AVC
Format profile : [email protected]
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Yes
Format settings, CABAC : Yes
Format settings, ReFrames : 1
Format settings, ReFrames : 1 frame
Internet media type : video/H264
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Codec ID/Url : http://www.apple.com/quicktime/download/standalone.html
Codec : AVC
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec/Url : http://developers.videolan.org/x264.html
Codec/CC : avc1
Codec profile : [email protected]
Codec settings : CABAC / 1 Ref Frames
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 1
Duration : 52268
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 00:00:52.268
Duration : 00:00:52:09
Duration : 00:00:52.268 (00:00:52:09)
Bit rate : 17375530
Bit rate : 17.4 Mb/s
Width : 1920
Width : 1 920 pixels
Height : 1080
Height : 1 080 pixels
Stored_Height : 1088
Sampled_Width : 1920
Sampled_Height : 1080
Pixel aspect ratio : 1.000
Display aspect ratio : 1.778
Display aspect ratio : 16:9
Rotation : 90.000
Rotation : 90°
Frame rate mode : VFR
Frame rate mode : Variable
Frame rate : 25.044
Frame rate : 25.044 FPS
Minimum frame rate : 23.077
Minimum frame rate : 23.077 FPS
Maximum frame rate : 30.000
Maximum frame rate : 30.000 FPS
Frame count : 1309
Resolution : 8
Resolution : 8 bits
Colorimetry : 4:2:0
Color space : YUV
Chroma subsampling : 4:2:0
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Interlacement : PPF
Interlacement : Progressive
Bits/(Pixel*Frame) : 0.335
Stream size : 113523046
Stream size : 108 MiB (100%)
Stream size : 108 MiB
Stream size : 108 MiB
Stream size : 108 MiB
Stream size : 108.3 MiB
Stream size : 108 MiB (100%)
Proportion of this stream : 0.99590
Title : Core Media Video
Encoded date : UTC 2016-10-08 22:51:19
Tagged date : UTC 2016-10-08 22:52:12
Color range : Limited
colour_description_present : Yes
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Audio
Count : 272
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
StreamOrder : 1
ID : 2
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Commercial name : AAC
Format profile : LC
Codec ID : 40
Codec : AAC LC
Codec : AAC LC
Codec/Family : AAC
Codec/CC : 40
Duration : 52268
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 00:00:52.268
Duration : 00:00:52:15
Duration : 00:00:52.268 (00:00:52:15)
Source duration : 52338
Source duration : 52 s 338 ms
Source duration : 52 s 338 ms
Source duration : 52 s 338 ms
Source duration : 00:00:52.338
Bit rate mode : CBR
Bit rate mode : Constant
Bit rate : 64000
Bit rate : 64.0 kb/s
Channel(s) : 1
Channel(s) : 1 channel
Channel positions : Front: C
Channel positions : 1/0/0
ChannelLayout : C
Samples per frame : 1024
Sampling rate : 44100
Sampling rate : 44.1 kHz
Samples count : 2305019
Frame rate : 43.066
Frame rate : 43.066 FPS (1024 spf)
Frame count : 2251
Source frame count : 2254
Compression mode : Lossy
Compression mode : Lossy
Stream size : 410424
Stream size : 401 KiB (0%)
Stream size : 401 KiB
Stream size : 401 KiB
Stream size : 401 KiB
Stream size : 400.8 KiB
Stream size : 401 KiB (0%)
Proportion of this stream : 0.00360
Source stream size : 410894
Source stream size : 401 KiB (0%)
Source stream size : 401 KiB
Source stream size : 401 KiB
Source stream size : 401 KiB
Source stream size : 401.3 KiB
Source stream size : 401 KiB (0%)
Source_StreamSize_Proportion : 0.00360
Title : Core Media Audio
Encoded date : UTC 2016-10-08 22:51:19
Tagged date : UTC 2016-10-08 22:52:12
Other #1
Count : 112
Count of stream of this kind : 2
Kind of stream : Other
Kind of stream : Other
Stream identifier : 0
Stream identifier : 1
Type : meta
Duration : 52268
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 00:00:52.268
Duration : 00:00:52.268
Frame count : 6
Bit rate mode : VBR
Other #2
Count : 112
Count of stream of this kind : 2
Kind of stream : Other
Kind of stream : Other
Stream identifier : 1
Stream identifier : 2
Type : meta
Duration : 52268
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 52 s 268 ms
Duration : 00:00:52.268
Duration : 00:00:52.268
Frame count : 1
Bit rate mode : CBR
and ffprobe IMG_1438.MOV outputs:
ffprobe version 3.1.3 Copyright (c) 2007-2016 the FFmpeg developers
built with Apple LLVM version 7.3.0 (clang-703.0.31)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.1.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --disable-lzma --enable-vda
libavutil 55. 28.100 / 55. 28.100
libavcodec 57. 48.101 / 57. 48.101
libavformat 57. 41.100 / 57. 41.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 47.100 / 6. 47.100
libavresample 3. 0. 0 / 3. 0. 0
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'IMG_1438.MOV':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2016-10-08 22:51:19
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 5
com.apple.quicktime.software: 10.0.2
com.apple.quicktime.creationdate: 2016-10-08T17:51:19-0500
Duration: 00:00:52.27, start: 0.000000, bitrate: 17446 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 17375 kb/s, 25.04 fps, 120 tbr, 600 tbn, 1200 tbc (default)
Metadata:
rotate : 90
creation_time : 2016-10-08 22:51:19
handler_name : Core Media Data Handler
encoder : H.264
Side data:
displaymatrix: rotation of -90.00 degrees
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 62 kb/s (default)
Metadata:
creation_time : 2016-10-08 22:51:19
handler_name : Core Media Data Handler
Stream #0:2(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-10-08 22:51:19
handler_name : Core Media Data Handler
Stream #0:3(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2016-10-08 22:51:19
handler_name : Core Media Data Handler
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 0 for input stream 3
UPDATE
To clarify: my video above, the one with the high framerate (120 FPS) output after scaling, plays perfectly before and after scaling with FFmpeg (no sync issues, and 120 FPS is only about 14% larger in file size), I am simply trying to understand why this increase in framerate happens (just a little beyond Mulvya's note that the framerate stored in the container is wrong).
From a programming perspective, the initial issue I ran into was that I was using frame= from FFmpeg's sterr console output to determine progress, which reports erroneous results when the frame count increases dramatically on output ("I'm 372% done encoding?!"); I have since read another stackoverflow answer and changed my code to use time=, which appears to be a more robust way for me to display FFmpeg progress. (Also, there is FFmpeg's -progress option, of course).
Improving on the original command
My new command to scale, preserve a useful framerate, and optimize threads:
ffmpeg -i IMG_1438.MOV -vf scale=-2:600 -r 30 -vsync 0 IMG_1438_scaledTo600.MOV
Where 30 is the "Maximum frame rate" from mediainfo.
Thanks to help in the comments, I now know I do not fully understand FFmpeg's use of three different time bases for timestamps: tbn, tbc, and tbr.
They were explained by Robert Swain in 2009 and his explanation was also used to answer a Stackoverflow question about tbn, tbc, tbr.
It sounds to me, as I'm pulling together comments from Mulvya below and Michael Rampe at another forum, that tbr is guessed; it is frequently but not always the best value to use when changing from a variable to a constant frame rate video.
Which leaves these 2 questions...
(1) tbr is incorrect when "field rate and frame rate" differ? Does this happen a lot?
(2) Is -r 30 where 30 is the maximum frame rate reported by mediainfo the best way to do it for most codec/container combinations? (Or should I only use this method when I am scaling a H.264/MPEG-4 AVC video?)
Upvotes: 2
Views: 3766
Answers (1)
Reputation: 93068
(1) tbr is incorrect when "field rate and frame rate" differ? Does this happen a lot?
Depends on the encoding/rendering application and codec used. Note that tbr refers to frame rate and not field rate. 50 field pairs are still stored in 25 frames. So, that in itself is not the answer.
(2) Is -r=30 where 30 is the maximum frame rate reported by mediainfo the best way to do it ...
If you wish to encode to a constant frame rate stream, then yes, add -r max (there's no = symbol in between.). else use just -vsync 0. FFmpeg, by default, encodes to a CRF stream when outputting to MOV/MP4, and it references the tbr for the value to use. So, when the stream actually contains fewer frames, it duplicates frames so that the tbr value is met. The reason to use the max. frame rate for CFR encoding is that so no frames are dropped. With a lower value, the input will have portions with a greater number of frames and ffmpeg will drop to meet the r that you set.
Upvotes: 1
Related Questions
- ffmpeg output scale not work with framerate
- FFmpeg: inconsistent frame counts
- ffmpeg - how can I scale the video while perserving the total frames in it?
- ffmpeg scaling not working for video
- FFmpeg scaling issue
- ffmpeg scale doesn't work if destination res is larger than source
- ffmpeg - scale not working on all my cases
- ffmpeg: same video, same framerate, different number of output frames. Why?
- FFMPEG Incorrect frame size, How to resolve?
- FFMPEG - Scale video filter not providing expected results