How to specify an offset and a seek position for a video stream of a mp4 container?

Question

It seems like I can have one or the other (-ss or -itsoffset), but never both at the same time, or they kind of cancel each other.

I create output.mp4 with this command:

ffmpeg -i input.mp3 -itsoffset 4 -t 4 -ss 3 -i input.mp4 -c:v copy output.mp4

With this command, I expect the audio to start playing right away, and the video to start playing 4 seconds later, seeked at 3 seconds into the video.

However, when I play input.mp4, the video starts only one second later than the audio (as opposed to 4 seconds). I suppose it does a "offset - seek = 1" type of thing.

How can I prevent the seeking from affecting the offset?

Cheers!

Answer 1

"I expect the audio to start playing right away, and the video to start playing 4 seconds later,
(seeked at 3 seconds into the video).

How can it be done without re-encoding?"

Try using as:

ffmpeg -ss 00:00:03 -itsoffset 4 -i video.mp4 -ss 00:00:00 -i audio.mp3 -c:v copy output.mp4

What the commands mean (and their entry/order is also important)...

-ss 00:00:03 = seek into 3 seconds of input video.mp4 to set as starting point of output video track.

-itsoffset 4 = set delay before input's playback as output. Gives still frame for 4 seconds.

-i video.mp4 = set video input at this point.

-ss 00:00:00 = seek into 0 seconds of input audio.mp4 as starting point of output audio track.

-i audio.mp3 = set audio input at this point... (see audio side-note below).

-c:v copy = (avoids re-encode of video track).

output.mp4 = set output a/v container.

Side notes:

about video: An MPEG video can only seek to keyframes (also called i-frames). Things may not work as expected, if you set -ss 3 whilst also using -c:v copy, and there is no keyframe found at time of -ss.

Possible lack of a keyframe at 3 seconds might explain your:

"When I play input.mp4, the video starts only one second later than the audio
(as opposed to 4 seconds)"

This is even warned about in the docs (see: FFmpeg wiki page - Seeking).

Seeking while doing a codec copy

Using -ss as input option together with -c:v copy might not be accurate since ffmpeg is forced to only use/split on i-frames. Though it will —if possible adjust the start time of the stream to a negative value to compensate for that.

Basically, if you specify "second 157" and there is no key frame until second 159, it will include two seconds of audio (with no video) at the start, then will start from the first key frame.

So be careful when splitting and doing codec copy.

Only solution is to keep test/adjusting the video -ss value to find the next-best compromise.

• Consider using the HOURS:MM:SS.MILLISECONDS time format to get closest to expected 3 seconds. example: -ss 00:00:02.895

about audio: MP4 usually has an AAC audio track. Note that since your input audio is in MP3 format, this will be automatically re-encoded into AAC format by FFmpeg.

• To avoid an audio format re-encode: Force MP3 with -c:a copy
  (can be added after the above -c:v copy).

Answer 2

As Gyan suggested, I got it to work by encoding the video stream seperately with the seeking position and the duration:

ffmpeg -t 4 -ss 3 -i input.mp4 -c:v libx264 output.mp4

Then mixing it in the final .mp4 with the offset:

ffmpeg -i input.mp3 -itsoffset 4 -i input.mp4 -c:v copy output.mp4