CODEWITHSUNDEEP
androidgoogle-chromevideowebgl
Erik Živković
Erik Živković

Reputation: 5455

WebGL video texture size limits on android - how to use 3840 * 2160?

I am developing a WebGL application that gets a stream of textures from HTML5 video, using HLS.js. It's working great on desktop, and it's working for 1920 * 1080 video on mobile Android, but not for 3840 * 2160.

I have tested the app on a couple of high-end devices (Xperia X Performance, Samsung Galaxy S8), both fail for the 4k video.

I know that the video can be played on those devices, because I also have a debug mode where the video element is attached to the DOM, and the video renders perfectly.

I have also used http://webglreport.com/ on those devices and that page shows that I should be able to use 4096 * 4096 textures.

I have also manually generated a 3840 * 2160 using Javascript ArrayBuffer and that texture was properly rendered.

This is how I copy the video

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, this.videoElement);

When performing this call for a video of size 3840 * 2160 on Chrome Android I get the following log print

SurfaceUtils: set up nativeWindow 0xc9ef2008 for 3840x2160, color 0x7fa30c04, rotation 0, usage 0x20002900

chromium: [INFO:CONSOLE(11818)] "WebGL: INVALID_VALUE: texImage2D: width or height out of range"

Which maps to this error code from gl.getError():

GL_INVALID_VALUE, 0x0501

These are the parameters I use for the backing texture

const tex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, tex);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);

And here is how I generate and upload the test texture

const width = 3840;
const height = 2160;

const generateTex = (w: number, h: number): number[] => {
    const res = [];
    for (let i = 0; i < w * h; i++) {
        res.push(0, 255, 0);
    }
    return res;
};

const image = new Uint8Array(generateTex(width, height));
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, width, height, 0, gl.RGB, gl.UNSIGNED_BYTE, image);

The documentation for texImage2D says

source of type HTMLVideoElement

The width and height of the texture are set to the width and height of the uploaded frame of the video in pixels.

The question is: Is there a specification somewhere that can explain why my 3840 * 2160 texture does not get rendered?

TL;DR

Thanks!

Upvotes: 3

Views: 1449

Answers (1)

Erik Živković
Erik Živković

Reputation: 5455

After a week of effort to understand the problem, it is now clear the error was on our end.

The video was indeed created as 3840 x 2160, but was created from a source video with a resolution of 3840 x 4320. When converting the video to 3840 x 2160, ffmpeg was setting the aspect ratio to (3840/4320), leading Android to inflate the video back to that size. The inflated size has a height > 4096 and that was the reason for the texture not being able to be created.

We didn't see this issue on other platforms (native apps) so it might be some quirk of the Android media player.

We have fixed the issue on our end by setting the "correct" aspect ratio of (3840/2160) manually as an ffmpeg parameter.

What I learned:

Don't assume that the resolution reported by applications such as VLC or QuickTime will properly reflect what will happen on every other platform. After implementing the code necessary to check the resolution of the HTML Video Element, it became apparent that something different was happening on Android compared to other platforms.

Upvotes: 2

Related Questions