How to blend/merge two videos in android programmatically

Question

As description says I am trying to create a video which consists of background video and semi-transparent video overlay on top of it.

My initial attempt was to use ffmpeg for android https://github.com/WritingMinds/ffmpeg-android-java and use blend filter in overlay mode.

But it turned out that ffmpeg android library is built on top of ffmpeg 2.4 but blend filter is available since ffmpeg 2.6. So far ffmpeg failed for me and I am looking for alternative solution.

Any advices are warmly appreciated.

UPDATE#1

I finally managed to build ffmpeg 2.6.4 for android which contains blend filter but another issue appeared. When it start video processing it just stops in the beginning and never ends, there is not output from ffmpeg tool, no errors, no warnings, there is nothing that can help me to trace down the problem.

On android device it runs following command:

/data/user/0/com.test.app/files/ffmpeg 
-i /storage/emulated/0/Android/data/com.test.app/files/video/vid1.mp4 
-i /storage/emulated/0/Android/data/com.test.app/files/video/overlay.mp4 
-filter_complex blend=all_mode='overlay':all_opacity=0.8" 
-strict -2 /storage/emulated/0/Android/data/com.test.app/files/video/blended.mp4

And there is following output:

ffmpeg version n2.6.4 Copyright (c) 2000-2015 the FFmpeg developers
   built with gcc 4.8 (GCC)
   configuration: --target-os=linux --cross-prefix=/media/ubuntu/data/ffmpeg-android/toolchain-android/bin/arm-linux-androideabi- --arch=arm --cpu=cortex-a8 --enable-runtime-cpudetect --sysroot=/media/ubuntu/data/ffmpeg-android/toolchain-android/sysroot --enable-pic --enable-libx264 --enable-libass --enable-libfreetype --enable-libfribidi --enable-fontconfig --enable-pthreads --disable-debug --disable-ffserver --enable-version3 --enable-hardcoded-tables --disable-ffplay --disable-ffprobe --enable-gpl --enable-yasm --disable-doc --disable-shared --enable-static --pkg-config=/media/ubuntu/data/ffmpeg-android/ffmpeg-pkg-config --prefix=/media/ubuntu/data/ffmpeg-android/build/armeabi-v7a-neon --extra-cflags='-I/media/ubuntu/data/ffmpeg-android/toolchain-android/include -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-overflow -fstack-protector-all -mfpu=neon' --extra-ldflags='-L/media/ubuntu/data/ffmpeg-android/toolchain-android/lib -Wl,-z,relro -Wl,-z,now -pie' --extra-libs='-lpng -lexpat -lm' --extra-cxxflags=
   libavutil      54. 20.100 / 54. 20.100
   libavcodec     56. 26.100 / 56. 26.100
   libavformat    56. 25.101 / 56. 25.101
   libavdevice    56.  4.100 / 56.  4.100
   libavfilter     5. 11.102 /  5. 11.102
   libswscale      3.  1.101 /  3.  1.101
   libswresample   1.  1.100 /  1.  1.100
   libpostproc    53.  3.100 / 53.  3.100
 [h264 @ 0xb5e03000] no frame!
 [aac @ 0xb5e03400] Input buffer exhausted before END element found
 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/Android/data/com.test.app/files/video/vid1.mp4':
   Metadata:
     major_brand     : mp42
     minor_version   : 0
     compatible_brands: isommp42
     creation_time   : 2015-08-30 17:19:07
   Duration: 00:00:05.03, start: 0.000000, bitrate: 3449 kb/s
     Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 640x640, 3381 kb/s, SAR 1:1 DAR 1:1, 30 fps, 30 tbr, 90k tbn, 180k tbc (default)
     Metadata:
       creation_time   : 2015-08-30 17:19:07
       handler_name    : VideoHandle
     Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
     Metadata:
       creation_time   : 2015-08-30 17:19:07
       handler_name    : SoundHandle
 [h264 @ 0xb5e03800] no frame!
 [aac @ 0xb5e04400] Input buffer exhausted before END element found
 Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/Android/data/com.test.app/files/video/overlay.mp4':
   Metadata:
     major_brand     : mp42
     minor_version   : 0
     compatible_brands: isommp42
     creation_time   : 2015-08-30 17:19:07
   Duration: 00:00:05.03, start: 0.000000, bitrate: 3449 kb/s
     Stream #1:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 640x640, 3381 kb/s, SAR 1:1 DAR 1:1, 30 fps, 30 tbr, 90k tbn, 180k tbc (default)
     Metadata:
       creation_time   : 2015-08-30 17:19:07
       handler_name    : VideoHandle
     Stream #1:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
     Metadata:
       creation_time   : 2015-08-30 17:19:07
       handler_name    : SoundHandle

After this output ffmpeg process is still running but it doesn't provide any response... I am not sure how to deal with such issue, maybe I can try to use different codecs which works better on android or enable some debugging logging in ffmpeg. Any advices would help.

UPDATE#2

Thank you

Answer 1

If you want blend filter, well then, you should build ffmpeg yourself.

But, you can overlay video into video without blend filter.

First, change your video transparency, colorchannelmixer filter can do.

colorchannelmixer=[0 to 1] // 0 = 0%, 0.5 = 50%, 1 = 100%

then merger this into main video. you can use macro for position of X and Y(main_h, main_w and over_h, over_w).

overlay=X:Y // X : x of position, Y : y of position

full command line is as follows :

ffmpeg -i OVERLAY_VIDEO.mp4 -i MAIN_VIDEO.mp4 -filter_complex "[0:v]colorchannelmixer=aa=0.5[ov];[1:v][ov]overlay=0:0[video_out]" -map [video_out] -map 1:a OUTPUT.mp4

[0:v] is video stream of OVERLAY_VIDEO.mp4 because first -i is OVERLAY_VIDEO.mp4 and it is mapped in 0 automatically. also i defined [ov] map to get output from transparency filter,

Then i merge two videos using overlay filter with [1:v](video stream of MAIN_VIDEO.mp4) and [ov], which i just make shortly before.

Finally, i defined [video_out] to get output of overlay filter then map it again with 1:a, which is audio stream of MAIN_VIDEO.mp4

If you have any question, let me know :)

answer for update #1

You didn't provide map into filter so nothing was provided into filter, no output.

following command would work :

/data/user/0/com.test.app/files/ffmpeg \ 
-i /storage/emulated/0/Android/data/com.test.app/files/video/vid1.mp4 \
-i /storage/emulated/0/Android/data/com.test.app/files/video/overlay.mp4 \
-filter_complex "[0:v][1:v]blend=all_mode='overlay':all_opacity=0.8[v_out]" \
-map [v_out] -map 0:a -strict -2 /storage/emulated/0/Android/data/com.test.app/files/video/blended.mp4