CODEWITHSUNDEEP
bashvideoffmpeg
Hans J
Hans J

Reputation: 45

How to accurately match an image to a video frame and exit with ffmpeg

In Bash, I am trying to match an image to a frame in ffmpeg. I also want to exit the ffmpeg process when the match is found. Here is a (simplified version) of the code currently: 

ffmpeg --hide_banner -ss 0 -to 60 \
-i "video.mp4" -i "image.jpg" -filter_complex \
"blend=difference, blackframe" -f null - </dev/null 2>log.txt &
pid=$!
trap "kill $pid 2>/dev/null" EXIT
while kill -0 $pid 2>/dev/null; do
     # (grep command to monitor log file)
     # if grep finds blackframe match, return blackframe time
done

To my understanding, if the video actually contains a blackframe I will get a false-positive. How can I effectively mitigate this?

While this is unnecessary to answer the question, I would like to exit the ffmpeg process without having to use grep to constantly monitor the log file, instead using pure ffmpeg

Edit: I say this because while I understand the blend filter is computing the difference, I am getting a false positive on a blackframe in my video and I don't know why.

Edit: A possible solution to this issue is to not use blackframe at all, but psnr (Peak Signal to Noise Ratio) but normal usage is by comparing two videos frame by frame, and I don't know how to effectively use it with an image as input.

Upvotes: 2

Views: 2012

Answers (1)

Gyan
Gyan

Reputation: 93299

Use

ffmpeg -ss 0 -t 60 -copyts -i video.mp4 -i image.jpg -filter_complex "[0]extractplanes=y[v];[1]extractplanes=y[i];[v][i]blend=difference,blackframe=0,metadata=select:key=lavfi.blackframe.pblack:value=100:function=equal,trim=duration=0.0001,metadata=print:file=-" -an -v 0 -vsync 0 -f null -

If a match is found, it will print to stdout a line of the form,

frame:179  pts:2316800 pts_time:6.03333
lavfi.blackframe.pblack=100

else no lines will be printed. It will exit after the first match, if found, or till whole input is processed.

Since blackframe only looks at luma, I use extractplanes both to speed up blend and also avoid any unexpected format conversions blend may request.

blackframe threshold is set to 0, so all frames have the blackframe value metadata tagged. False positives are not possible since blend computes the difference. The difference between a black input frame and the reference frame is equal to the reference frame, unless the reference is a black frame, in which case, it is not a false positive.

The first metadata filter only passes through frames with blackframe value of 100. The trim filter stops a 2nd frame from passing through (except if your video's fps is greater than 10000). The 2nd metadata filter prints the selected frame's metadata.

Upvotes: 3

Related Questions