How to remove long silent and unchanged video sections with ffmpeg?

Question

I want to identify areas in a .mp4 (H264 + AAC) video that are silent and unchanged frames and cut them out. Of course there would be some fine-tuning regarding thresholds and algorithms to measure unchanged frames.

• My problem is more general, regarding how I would go about automating this?
• Is it possible to solve this with ffmpeg? (preferably with C or python)
• How can I programatically analyse the audio?
• How can I programatically analyse video frames?

Answer 1

For audio silence see this.

For still video scenes ffmpeg might not be the ideal tool.

You could use scene change detection with a low threshold to find the specific frames, then extract those frames and compare them with something like imagemagick's compare function:

ffprobe -show_frames -print_format compact -f lavfi "movie=test.mp4,select=gt(scene\,.1)"

compare -metric RMSE frame1.png frame0.png

I don't expect this to work very well.

Your best bet is to use something like OpenCV to find differences between frames.

OpenCV Simple Motion Detection