CODEWITHSUNDEEP
c#algorithmstring-formatting
GazTheDestroyer
GazTheDestroyer

Reputation: 21261

Finding minimum precision

I'm trying to write an algorithm to find the minimum number of precision digits to show at least one significant figure using .Net string formatter.

eg.

Value      Precision wanted:
-----      -----------------
10         0
 1         0
 0.1       1
 0.99      1   
 0.01      2
 0.009     3

(don't care about further digits, only the first, hence 0.99 only need a precision of 1.)

The best I can come up with is:

int precision = (int)Math.Abs(Math.Min(0, Math.Floor(Math.Log10(value))));

This works fine but I can't help thinking there's a more elegant solution. Can any maths gurus help me out?

Upvotes: 3

Views: 565

Answers (2)

Stephen Garle
Stephen Garle

Reputation: 299

A float is a binary representation of a fractional value. If your initial value is a single precision floating point number, then the number is multiplied by an exponent to get the value,

((f & 0x7f800000) >> 23)-127 .

If the exponent is non-negative, based on what you have said you would get back zero (you have numbers before the decimal point).

If the exponent is negative, well, that is annoying since binary digits don't line up well with decimal. It should be doable with a look up table, though. Check out https://math.stackexchange.com/questions/3987/how-to-calculate-the-number-of-decimal-digits-for-a-binary-number#4080.

Edit: you should read about the way floating point numbers are stored.

Upvotes: 1

Henrik
Henrik

Reputation: 23324

Slightly shorter:

int precision = (int)Math.Max(0, -Math.Floor(Math.Log10(value)));

Upvotes: 3

Related Questions