JAVA - Xuggler - combine an MP3 audio file and a MP4 movie

Question

Using JAVA and Xuggler - the following code combines an MP3 audio file and a MP4 movie file and outputs a combined mp4 file.

I want the duration of the output video to be equal to the time duration of the input video file

If I set the loop condition to be:

while (containerVideo.readNextPacket(packetvideo) >= 0) 

It gives 400 iterations - making the movie run for 15 seconds. exactly like I want.

But for some reason the audio is cut after 10 seconds- making the last 5 seconds of the movie silent.

It looks as if the time duration of a single iteration of the video IStreamCoder is different than the time duration of a single iteration of the audio IStreamCoder.

How can I have the sound fill the whole 15 seconds of the movie?

    String inputVideoFilePath = "in.mp4";
    String inputAudioFilePath = "in.mp3";
    String outputVideoFilePath = "out.mp4";

    IMediaWriter mWriter = ToolFactory.makeWriter(outputVideoFilePath);

    IContainer containerVideo = IContainer.make();
    IContainer containerAudio = IContainer.make();

    // check files are readable
    if (containerVideo.open(inputVideoFilePath, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + inputVideoFilePath);
    if (containerAudio.open(inputAudioFilePath, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + inputAudioFilePath);

    // read video file and create stream
    IStreamCoder coderVideo = containerVideo.getStream(0).getStreamCoder();
    if (coderVideo.open(null, null) < 0)
        throw new RuntimeException("Cant open video coder");
    IPacket packetvideo = IPacket.make();
    int width = coderVideo.getWidth();
    int height = coderVideo.getHeight();

    // read audio file and create stream
    IStreamCoder coderAudio = containerAudio.getStream(0).getStreamCoder();
    if (coderAudio.open(null, null) < 0)
        throw new RuntimeException("Cant open audio coder");
    IPacket packetaudio = IPacket.make();

    mWriter.addAudioStream(1, 0, coderAudio.getChannels(), coderAudio.getSampleRate());
    mWriter.addVideoStream(0, 0, width, height);

    while (containerVideo.readNextPacket(packetvideo) >= 0) {

        containerAudio.readNextPacket(packetaudio);

        // video packet
        IVideoPicture picture = IVideoPicture.make(coderVideo.getPixelType(), width, height);
        coderVideo.decodeVideo(picture, packetvideo, 0);
        if (picture.isComplete()) 
            mWriter.encodeVideo(0, picture);

        // audio packet 
        IAudioSamples samples = IAudioSamples.make(512, coderAudio.getChannels(), IAudioSamples.Format.FMT_S32);
        coderAudio.decodeAudio(samples, packetaudio, 0);
        if (samples.isComplete()) 
            mWriter.encodeAudio(1, samples);

    }

    coderAudio.close();
    coderVideo.close();
    containerAudio.close();
    containerVideo.close();
    mWriter.close();

Answer 1

I realize this is old, but I came across it, and it did exactly what I needed, but I had the same problem (my audio stopped prematurely). Eli's answer worked but took some tweaking. I'm posting the change here in hopes of saving somebody some time down the road. Disclaimer: I don't know Xuggle well, and I don't know if this is the best way to do this, but I do know it got he job done for me:

    String inputVideoFilePath = "in.mp4";
    String inputAudioFilePath = "in.mp3";
    String outputVideoFilePath = "out.mp4";

    IMediaWriter mWriter = ToolFactory.makeWriter(outputVideoFilePath);

    IContainer containerVideo = IContainer.make();
    IContainer containerAudio = IContainer.make();

    // check files are readable
    if (containerVideo.open(inputVideoFilePath, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + inputVideoFilePath);
    if (containerAudio.open(inputAudioFilePath, IContainer.Type.READ, null) < 0)
        throw new IllegalArgumentException("Cant find " + inputAudioFilePath);

    // read video file and create stream
    IStreamCoder coderVideo = containerVideo.getStream(0).getStreamCoder();
    if (coderVideo.open(null, null) < 0)
        throw new RuntimeException("Cant open video coder");
    IPacket packetvideo = IPacket.make();
    int width = coderVideo.getWidth();
    int height = coderVideo.getHeight();

    // read audio file and create stream
    IStreamCoder coderAudio = containerAudio.getStream(0).getStreamCoder();
    if (coderAudio.open(null, null) < 0)
        throw new RuntimeException("Cant open audio coder");
    IPacket packetaudio = IPacket.make();

    mWriter.addAudioStream(1, 0, coderAudio.getChannels(), coderAudio.getSampleRate());
    mWriter.addVideoStream(0, 0, width, height);

    while (containerVideo.readNextPacket(packetvideo) >= 0) {

        containerAudio.readNextPacket(packetaudio);

        // video packet
        IVideoPicture picture = IVideoPicture.make(coderVideo.getPixelType(), width, height);
        coderVideo.decodeVideo(picture, packetvideo, 0);
        if (picture.isComplete()) 
            mWriter.encodeVideo(0, picture);

        // audio packet 
        IAudioSamples samples = IAudioSamples.make(512, coderAudio.getChannels(), IAudioSamples.Format.FMT_S32);
        coderAudio.decodeAudio(samples, packetaudio, 0);
        if (samples.isComplete()) 
            mWriter.encodeAudio(1, samples);

    }


//<added_code> This is Eli Sokal's code tweaked to work with gilad s' code

    IAudioSamples samples;
    do {
        samples = IAudioSamples.make(512, coderAudio.getChannels(), IAudioSamples.Format.FMT_S32);
        containerAudio.readNextPacket(packetaudio);
        coderAudio.decodeAudio(samples, packetaudio, 0);
        mWriter.encodeAudio(1, samples);
    }while (samples.isComplete() );

//</added_code>


    coderAudio.close();
    coderVideo.close();
    containerAudio.close();
    containerVideo.close();
    mWriter.close();

Answer 2

Nice code. Thanks. To capture the remaining audio, add a loop at the end to read audio packets until there are no more.

    while (samples.isComplete() ) {
        containerAudio.readNextPacket(packetaudio);
        coderAudio.decodeAudio(samples, packetaudio, 0);
        writer.encodeAudio(iAudioStream, samples);
        Utility.console(String.format("%s %d", Utility.timeStamp(),
            samples.getPts() ));            
    }