CODEWITHSUNDEEP
javascriptvideovideo-streaminghtml5-videomedia-source
Sergio Rinaudo
Sergio Rinaudo

Reputation: 2363

Javascript MediaSource API and H264 video

I have a problem playing a H264 video using javascript MediaSource Extension API.

I'll describe the scenario with details below.

I've already successfully achieved the result playing audio and video source of vp8, vp9, opus and vorbis codec, also from a range request ( if server has the capability, using any byte range ) or chunked files, chunks done using shaka packager.

The problem comes when the source is an H264 video, in details in my case codecs are avc1.64001e and mp4a.40.2, full codec string is video/mp4;codecs="avc1.64001e, mp4a.40.2" but the issue still happens with any other avc1 codec.

What I am trying to do is to play a 10 megabytes chunk of the full video, chunk generated by a byterange curl request saving the response locally using -o.

Below the stream info from shaka packager passing this file as input

[0530/161459:INFO:demuxer.cc(88)] Demuxer::Run() on file '10mega.mp4'.
[0530/161459:INFO:demuxer.cc(160)] Initialize Demuxer for file '10mega.mp4'.

File "10mega.mp4":
Found 2 stream(s).
Stream [0] type: Video
 codec_string: avc1.64001e
 time_scale: 17595
 duration: 57805440 (3285.3 seconds)
 is_encrypted: false
 codec: H264
 width: 720
 height: 384
 pixel_aspect_ratio: 1:1
 trick_play_factor: 0
 nalu_length_size: 4

Stream [1] type: Audio
 codec_string: mp4a.40.2
 time_scale: 44100
 duration: 144883809 (3285.3 seconds)
 is_encrypted: false
 codec: AAC
 sample_bits: 16
 num_channels: 2
 sampling_frequency: 44100
 language: und

Packaging completed successfully.

The chunk is playable with external media player applications ( like VLC ) and more important, it plays without problem adding it to the webpage using the < source > tag.

This is the error I can see in the Chrome console

Uncaught (in promise) DOMException: Failed to load because no supported source was found.

Here below the html and js code if you want to reproduce ( I did all local tests using the built-in php7.2 dev server )

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8"/>
    <title>VideoTest</title>
    <link rel="icon" href="/favicon.ico" />
    <script type="text/javascript" src="/script.js"></script>

    <style>
        video {
            width: 98%;
            height: 300px;
            border: 0px solid #000;
            display: flex;
        }
    </style>
</head>
<body>

<div id="videoContainer">
    <video controls></video>
</div>

<video controls>
    <source src="/media/10mega.mp4" type="video/mp4">
</video>

</body>
</html>

And here below the JS code ( scripjs )

class MediaTest {
    constructor() {

    }

    init(link) {
        this.link = link;
        this.media = new MediaSource();
        this.container = document.getElementsByTagName('video')[0];
        this.container.src = window.URL.createObjectURL(this.media);

        return new Promise(resolve => {
            this.media.addEventListener('sourceopen', (e) => {
                this.media = e.target;
                return resolve(this);
            });
        });
    }

    addSourceBuffer() {
        let codec = 'video/mp4;codecs="avc1.64001e, mp4a.40.2"';
        let sourceBuffer = this.media.addSourceBuffer(codec);

        // These are the same headers sent by the < source > tag
        // with or without the issue remains
        let headers = new Headers({
            'Range': 'bytes=0-131072',
            'Accept-Encoding': 'identity;q=1, *;q=0'
        });

        let requestData = {
            headers: headers
        };
        let request = new Request(this.link, requestData);

        return new Promise(resolve => {
            fetch(request).then((response) => {
                if(200 !== response.status) {
                    throw new Error('addSourceBuffer error with status ' + response.status);
                }

                return response.arrayBuffer();
            }).then((buffer) => {
                sourceBuffer.appendBuffer(buffer);
                console.log('Buffer appended');
                return resolve(this);
            }).catch(function(e) {
                console.log('addSourceBuffer error');
                console.log(e);
            });
        });
    }

    play() {
        this.container.play();
    }
}

window.addEventListener('load', () => {
    let media = new MediaTest();
    media.init('/media/10mega.mp4').then(() => {
        console.log('init ok');
        return media.addSourceBuffer();
    }).then((obj) => {
        console.log('play');
        media.play();
    });
});

What I want to achieve is to play the file with MediaSource API since it plays well using < source > tag. I don't want to demux and re-encode it, but use it as is.

Here below the error dump taken from chrome://media-internals

render_id: 180 player_id: 11 pipeline_state: kStopped event: WEBMEDIAPLAYER_DESTROYED

To reproduce I think it is possible to use any H264 video that has audio and video track within it.

This question is strictly related with this other question I've found H264 video works using src attribute. Same video fails using the MediaSource API (Chromium) but it is from 4 years ago so I decided not to answer there.

Does anybody have some idea about this issue? Is there any way to solve it or h264 It is just not compatible with MSE?

Thanks in advance

Upvotes: 4

Views: 3529

Answers (1)

szatmary
szatmary

Reputation: 31100

Its not the codec, its the container. MSE requires fragmented mp4 files. Standard mp4 is not supported. for standard mp4 you must use <video src="my.mp4">

Upvotes: 4

Related Questions