rotating image using webGL in a wide canvas

Question

i am developing a web app that will be able to do some image transformations one of them is image rotation, the problem is: as long as the canvas is square ( width = height ) every thing is ok, but other than that the image is getting skewed like the attached screenshots. i tried to change the scaling according to rotation angle using these equations

const scalingY = 1 + (aspectRatio - 1) * Math.pow(Math.sin(angelInRadian/2), 1);

const scalingX = (1/aspectRatio) +  (1-(1/aspectRatio)) * Math.pow(Math.cos(angelInRadian), 1);

but the image still skewed during rotaion

const scalingY = 1 + (aspectRatio - 1) * Math.pow(Math.sin(angelInRadian/2), 1);

const scalingX = (1/aspectRatio) +  (1-(1/aspectRatio)) * Math.pow(Math.cos(angelInRadian), 1)
const aspectRatioCompensation = new Float32Array([
                 1, 0, 0, 0,
                 0, 1, 0, 0,
                 0, 0, 1, 0,
                 0, 0, 0, 1
                            ]);
//const aspectRatioCompensation = new Float32Array([
                 scalingX, 0, 0, 0,
                 0, scalingY, 0, 0,
                 0, 0, 1, 0,
                 0, 0, 0, 1
                            ]);

const cosTheta = Math.cos(angelInRadian);
const sinTheta = Math.sin(angelInRadian);
const rotationMatrix = new Float32Array([
               cosTheta, -sinTheta, 0, 0,
               sinTheta, cosTheta,  0, 0,
                   0,         0,    1, 0,
                   0,         0,    0, 1
                            ]);

// Combine the aspect ratio adjustment and rotation
const combinedMatrix = mat4.create();

mat4.multiply(combinedMatrix, aspectRatioCompensation, rotationMatrix );

vertex shader :

                attribute vec4 a_position;
            attribute vec2 a_texCoord;
            varying vec2 v_position;
            varying vec2 v_texCoord;

            void main() {
                gl_Position = a_position;
                v_position = a_position.xy;
                v_texCoord = a_texCoord;
            }

fragment shader:

precision mediump float;
            varying vec2 v_texCoord;
            uniform sampler2D u_texture;
            uniform vec4 u_color;
            uniform int u_renderMode; // 0 for texture, 1 for shape


            void main() {
                    if (u_renderMode == 0) {
                        gl_FragColor = texture2D(u_texture, v_texCoord);
                    } else if (u_renderMode == 1) {
                        gl_FragColor = u_color;
                    }
            }

gl setup (this function is a part of a class that's responsible for screen update):

    updateScreen() {

        const { gl, program, positionBuffer, texCoordBuffer } = this;

        gl.useProgram(program);


        // Bind the position buffer for this layer
        gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
        const positionLocation = gl.getAttribLocation(program!, 'a_position');
        gl.enableVertexAttribArray(positionLocation);
        gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);

        // Bind the texCoord buffer for this layer
        gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
        const texCoordLocation = gl.getAttribLocation(program!, 'a_texCoord');
        gl.enableVertexAttribArray(texCoordLocation);
        gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0);

        const error = gl.getError();

        if (error !== gl.NO_ERROR) {
            console.error('WebGL error:', error);
        }

        // Enable blending
        //TODO: need more understanding of the blending functions
        gl.enable(gl.BLEND);
        gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);

        const hitTestLayer = globalState.get('hitTestLayer');

        this.model?.documents.activeDocument!.layers.forEach((layer: any) => {

            gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
            gl.bufferData(gl.ARRAY_BUFFER, layer.positionCoordArray, gl.STATIC_DRAW);

            gl.bindTexture(gl.TEXTURE_2D, layer.texture);

            const textureLocation = gl.getUniformLocation(program!, 'u_texture');

            gl.uniform1i(textureLocation, 0);

            layer.shapes.performDrawingCalls(gl, program!, 0)

            const pixels = new Uint8Array(4);

            gl.readPixels(5, 5, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

        })

        gl.useProgram(null);


    }

Answer 1

The issue is that selecting any coordinate in NDC along the X axis results in a different distance from the origin compared to selecting it along the Y axis. This discrepancy arises because NDC ranges from -1 to 1, regardless of the actual length of the axis. To address this, I decided to scale down the X axis by 1/aspectRatio

​ , then rotate, and finally scale up the X axis by the aspectRatio. This approach worked for me.

 mat4.scale(finalTransformationMatrix, initialTransformationMatrix, [1/aspectRatio, 1, 1])

mat4.rotateZ(finalTransformationMatrix, finalTransformationMatrix, -angelInRadian)

mat4.scale(finalTransformationMatrix, finalTransformationMatrix, [ aspectRatio, 1, 1])