CODEWITHSUNDEEP
pythonvideomp4
Alex
Alex

Reputation: 1420

Ffmpeg alpha channel video generation

I am trying to remove a background from a video using ffmpeg and a PY library that does that, the PY lib (backgroundremover) just creates a matte.mp4 file as an output, having the background as black and the person as white silhouette.

PY lib: https://github.com/nadermx/backgroundremover#advance-usage-for-video

What I am doing at the moment:

Shrink & convert the video to MP4

ffmpeg -i ios.mov -s 320x240 -filter:v fps=30 -vf scale=320:-2 edited.mp4

Create the matte video

backgroundremover -i edited.mp4 -wn 4 -mk -o matte.mp4

Create video with alpha channel (the problem)

ffmpeg -i edited.mp4 -i matte.mp4 -filter_complex "[0:v][1:v]alphamerge" -shortest -c:v qtrle -an output.mov

Last command fails with invalid frame sizes, how do I force a frame size or skip this check?

Error:

[swscaler @ 0x7ff5c957b000] No accelerated colorspace conversion found from yuv420p to argb.
[Parsed_alphamerge_0 @ 0x7ff5c4e6d480] Input frame sizes do not match (320x240 vs 426x320).
[Parsed_alphamerge_0 @ 0x7ff5c4e6d480] Failed to configure output pad on Parsed_alphamerge_0
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument

Answer:

ffmpeg -y -i edited.mp4 -i matte.mp4 -f lavfi -i color=c=black:s=320x240 -filter_complex "[1:v]scale=320:240,setsar=1:1,split[vs][alpha];[0:v][vs]alphamerge[vt];[2:v][vt]overlay=shortest=1[rgb];[rgb][alpha]alphamerge" -shortest -c:v hevc_videotoolbox -allow_sw 1 -alpha_quality 0.75 -vtag hvc1 -pix_fmt yuva420p -an output.mov

Upvotes: 3

Views: 7454

Answers (1)

Rotem
Rotem

Reputation: 32124

The error Input frame sizes do not match (320x240 vs 426x320) is "self explained".

  • The resolution of edited.mp4 is 320x240.
  • The resolution of matte.mp4 is 426x320.
    I don't know why backgroundremover modifies the resolution from 320x240 to 426x320.

The rest of the messages are just warnings.

I am not sure about it, but I think the first FFmpeg command should be:

ffmpeg -y -i ios.mov -filter:v fps=30 -vf scale=320:240,setsar=1:1 edited.mp4

It's not solving the issue - the resolution of matte.mp4 is still 426x320.
It could be a bug in backgroundremover...

You may solve the error message using scale filer.
The alpha merge should be followed by an overlay filter:

ffmpeg -y -i edited.mp4 -i matte.mp4 -f lavfi -i color=c=black:s=320x240 -filter_complex "[1:v]scale=320:240,setsar=1:1[vs];[0:v][vs]alphamerge[vt];[2:v][vt]overlay=shortest=1" -shortest -c:v qtrle -an output.mov

Sample output:
enter image description here

Upvotes: 2

Related Questions