CODEWITHSUNDEEP
androidc++unity-game-engineopengl-es
hGen
hGen

Reputation: 2285

Unity native OpenGL texture displayed four times

I'm currently facing a problem I simply don't understand. I employ ARCore for an inside out tracking task. Since I need to do some additional image processing I use Unitys capability to load a native c++ plugin. At the very end of each frame I pass the image in YUV_420_888 format as raw byte array to my native plugin.

A texture handle is created right at the beginning of the components initialization:

private void CreateTextureAndPassToPlugin()
{

    Texture2D tex = new Texture2D(640, 480, TextureFormat.RGBA32, false);

    tex.filterMode = FilterMode.Point;
    tex.Apply();
    debug_screen_.GetComponent<Renderer>().material.mainTexture = tex;

    // Pass texture pointer to the plugin
    SetTextureFromUnity(tex.GetNativeTexturePtr(), tex.width, tex.height);
}

Since I only need the grayscale image I basically ignore the UV part of the image and only use the y coordinates as displayed in the following:

uchar *p_out;
int channels = 4;
for (int r = 0; r < image_matrix->rows; r++) {
    p_out = image_matrix->ptr<uchar>(r);
    for (int c = 0; c < image_matrix->cols * channels; c++) {
        unsigned int idx = r * y_row_stride + c;
        p_out[c] = static_cast<uchar>(image_data[idx]);
        p_out[c + 1] = static_cast<uchar>(image_data[idx]);
        p_out[c + 2] = static_cast<uchar>(image_data[idx]);
        p_out[c + 3] = static_cast<uchar>(255);
    }
}

then each frame the image data is put into a GL texture:

GLuint gltex = (GLuint)(size_t)(g_TextureHandle);
glBindTexture(GL_TEXTURE_2D, gltex);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 640, 480, GL_RGBA, GL_UNSIGNED_BYTE, current_image.data);

I know that I use way too much memory by creating and passing the texture as RGBA but since GL_R8 is not supported by OpenGL ES3 and GL_ALPHA always lead to internal OpenGL errors I just pass the greyscale value to each color component.

However in the end the texture is rendered as can be seen in the following image:

image

At first I thought, that the reason for this may lie in the other channels having the same values, however setting all other channels than the first one to any value does not have any impact.

Am I missing something OpenGL texture creation wise?

Upvotes: 0

Views: 358

Answers (1)

solidpixel
solidpixel

Reputation: 12159

YUV_420_888 is a multiplane texture, where the luminance plane only contains a single channel per pixel.

for (int c = 0; c < image_matrix->cols * channels; c++) {
    unsigned int idx = r * y_row_stride + c;

Your loop bounds assume c is in multiple of 4 channels, which is right for the output surface, but you then use it also when computing the input surface index. The input surface plane you are using only contains one channel, so idx is wrong.

In general you are also over writing the same memory multiple times - the loop increments c by one each iteration but you then write to c, c+1, c+2, and c+3 so overwrite three of the values you wrote last time.

Shorter answer - your OpenGL ES code is fine, but I think you're filling the texture with bad data.

Untested, but I think you need:

for (int c = 0; c < image_matrix->cols * channels; c += channels) {
    unsigned int idx = (r * y_row_stride) + (c / channels);

Upvotes: 1

Related Questions