Michael O'Brien
Michael O'Brien

Reputation: 41

Metal float precision when adding values

I am not getting the right value when adding two floats in Metal.

I have an image stored as a 16bit float per channel. In Metal, I read the values from the image and try to add them together. The metal code looks like this:

out.write(float4(a, b, a + b, 1.0), id);

Here's what I get in the metal debugger:

Metal display of precision issue

I would be fine if the precision caused the result to be 1.93, but it looks like Metal is skewing towards the smaller value. Any idea why this is happenning?

Upvotes: 0

Views: 325

Answers (1)

Michael O'Brien
Michael O'Brien

Reputation: 41

Whelp. The issue had nothing to do with addition not working or addition doing something strange with precision. The issue was in the display. The Metal debugger does not display exponents when showing float numbers. The top number is actually 1.93125E-05, but the e-5 portion is not displayed.

Upvotes: 3

Related Questions