Unsure why 3x3 matrix as a vertex attribute in GLSL is not working

Question

I am trying to make a simple 2D rectangle renderer in OpenGL. I'm storing transform of each rectangle as a 3x3 matrix inside a buffer object, and passing into my vertex shader as a vertex attribute. The problem is that nothing is being displayed, the screen is blank.

I'm still fairly new to OpenGL and I'm not really sure what the cause of the problem is.

VBO and VAO creation code

let mut quad_vbo: GLuint = 0;
let mut instance_vbo: GLuint = 0;
let mut vao: GLuint = 0;

let instance_buffer_width = 100;

unsafe { 
    gl::GenVertexArrays(1, &mut vao);
    gl::GenBuffers(1, &mut quad_vbo);
    gl::BindVertexArray(vao);
    gl::BindBuffer(gl::ARRAY_BUFFER, quad_vbo);
    let (_, data_bytes, _) = quad_vertex_data.align_to::<u8>();
    gl::BufferData(
        gl::ARRAY_BUFFER,
        (size_of::<Vertex>() * quad_vertex_data.len()) as GLsizeiptr,
        data_bytes.as_ptr() as *const GLvoid,
        gl::STATIC_DRAW
    );

    let position_offset = memoffset::offset_of!(Vertex, pos) as i32;
    gl::EnableVertexAttribArray(0);
    gl::VertexAttribPointer(
        0,
        2,
        gl::FLOAT,
        gl::FALSE,
        size_of::<Vertex>() as i32,
        position_offset as *const std::ffi::c_void,
    );

    gl::BindBuffer(gl::ARRAY_BUFFER, 0);
    
    gl::GenBuffers(1, &mut instance_vbo);
    gl::BindBuffer(gl::ARRAY_BUFFER, instance_vbo);
    gl::BufferData(
        gl::ARRAY_BUFFER,
        (size_of::<Matrix3<f32>>() * instance_buffer_width) as GLsizeiptr,
        0 as *const GLvoid,
        gl::STATIC_DRAW
    );

    let vec_3_size = size_of::<Matrix3<f32>>() as i32;

    gl::EnableVertexAttribArray(1);
    gl::VertexAttribPointer(
        1,
        3,
        gl::FLOAT,
        gl::FALSE,
        vec_3_size * 3,
        0 as *const std::ffi::c_void,
    );
    gl::EnableVertexAttribArray(2);
    gl::VertexAttribPointer(
        2,
        3,
        gl::FLOAT,
        gl::FALSE,
        vec_3_size * 3,
        (1 * vec_3_size) as *const std::ffi::c_void,
    );
    gl::EnableVertexAttribArray(3);
    gl::VertexAttribPointer(
        3,
        3,
        gl::FLOAT,
        gl::FALSE,
        vec_3_size * 3,
        (2 * vec_3_size) as *const std::ffi::c_void,
    );

    gl::VertexAttribDivisor(1, 1);
    gl::VertexAttribDivisor(2, 1);
    gl::VertexAttribDivisor(3, 1);

    gl::BindVertexArray(0);
}

Add object to buffer code

pub fn add(&mut self, transform: Matrix3<f32>) {
    self.shader_program.use_();
    // Use a uniform instead of a buffer
    // self.shader_program.set_matrix_3f("uModel", &transform);
    unsafe {
        let transform_slice = [transform];
        let (_, data_bytes, _) = transform_slice.align_to::<u8>();
        gl::BindBuffer(gl::ARRAY_BUFFER, self.instance_vbo);
        gl::BufferSubData(
            gl::ARRAY_BUFFER,
            (self.index*data_bytes.len()) as GLsizeiptr,
            data_bytes.len() as GLsizeiptr,
            data_bytes.as_ptr() as *const GLvoid,
        );
        println!("Added object at index: {}, {}, {:?}", (self.index*data_bytes.len()), data_bytes.len(), data_bytes);
    }
    self.index += 1;
}

Draw code

pub fn present(&mut self) {
    unsafe {
        self.shader_program.use_();
        gl::BindVertexArray(self.vao);
        gl::DrawArraysInstanced(gl::TRIANGLE_FAN, 0, 4, self.index as i32);
        gl::BindVertexArray(0);
    }
}

Vertex shader

#version 330 core
layout (location = 0) in vec2 position;
layout (location = 1) in mat3 model;

uniform mat4 uProjection;
// uniform mat3 uModel;
// Replacing model with uModel works just fine

void main() {
    gl_Position = uProjection * vec4(model * vec3(position, 1.0), 1.0);
}

Sorry if this is a long post but I haven't been able to pinpoint the error.

I have tried just manually setting a uniform variable instead of using an array buffer, doing that properly displays the rectangle. For this reason I am fairly sure it is an issue relating to the creation of the buffer/attribute pointers, or the way I add values to the buffer.

Answer 1

Shader attributes (not just for the vertex shader, but also if you pass data between different shader stages) are capped to vec4. Anything larger (ie. matrix types) require multiple attribute slots. In your case, you'll need three attributes, one for each row of the 3x3 matrix. Don't forget to properly set the offset in glVertexAttribPointer for each of the attributes, as they each start at a different row!

In the shader you don't need to change anything (yet). However, if you were to add additional attributes, their location needs to respect that the matrix attribute takes multiple slots.