CODEWITHSUNDEEP
opengl-esopengl-es-2.0glsl
Timothy
Timothy

Reputation: 469

Loss of precision in GLSL fragment shader

I am now using opengl-es and I use the gl shading language. I hope to render to texture but I found a loss of precision. For example, when I write a float value of 0.5 to the texture, I found the actual value stored in the texture was approximately 0.498. What should I do to achieve higher precision?

Upvotes: 2

Views: 1909

Answers (4)

TheWhiteLlama
TheWhiteLlama

Reputation: 1286

You probably should consider storing your values (if just one value per pixel/texel) via packing-unpacking your values:

vec4 packFloat(const float value) {
    const vec4 bitSh = vec4(256.0 * 256.0 * 256.0, 256.0 * 256.0, 256.0, 1.0);
    const vec4 bitMsk = vec4(0.0, 1.0 / 256.0, 1.0 / 256.0, 1.0 / 256.0);
    vec4 res = fract(value * bitSh);
    res -= res.xxyz * bitMsk;
    return res;
}

float unpackFloat(const vec4 value) {
    const vec4 bitSh = vec4(1.0 / (256.0 * 256.0 * 256.0), 1.0 / (256.0 * 256.0), 1.0 / 256.0, 1.0);
    return (dot(value, bitSh));
}

This might be okay for storing values for something like depth-maps And this would be kind of a 32 bit range for each pixel/texel

Upvotes: 3

Aki Suihkonen
Aki Suihkonen

Reputation: 20017

Even the next higher precision might not be enough, because the final stage of the rendering pipeline scales the pixel values to a range of 0..1, the end points inclusive. Thus 1 will be represented as 255, which suggest a factor of 1/255 instead of 1/256.

The same applies to all precisions: 0.5 can't be represented exactly.

Upvotes: 1

Nicol Bolas
Nicol Bolas

Reputation: 473192

Render to a texture that uses more than 8 bits per component. If you don't have the appropriate OpenGL ES extensions for that, then generally there's not much you can do.

Upvotes: 1

A E
A E

Reputation: 586

Try adding the highp precision qualifier in front of your variables.

Upvotes: 1

Related Questions