CODEWITHSUNDEEP
formatoctaveprecision
Fernando
Fernando

Reputation: 11

Odd behaviour Using num2str in Octave

Having issue with the num2str function in Octave.

string = num2str(8.395,3);

String returns "8.39" whereas,

string = num2str(1.395,3);

returns "1.4".

How can it be possible? How can I get a consistent number of decimals?

Upvotes: 1

Views: 81

Answers (2)

Tasos Papastylianou
Tasos Papastylianou

Reputation: 22225

The question seems to be "why is 8.395 rounded "down", whereas 1.395 is rounded "up". Is that a bug?

Yes and no. This is an inherent limitation of floating-point arithmetic. Effectively, neither of those two numbers can be expressed exactly in floating-point format, therefore their nearest approximation is used. It just so happens that the nearest approximation for 8.395 is just "under" that value, whereas for 1.395 is just "above" it. Hence octave rounds the first "downwards", and the latter "upwards". You can confirm this if you print more significant digits for each:

sprintf("%.20f", 8.395)   % ans = 8.39499999999999957367
sprintf("%.20f", 1.395)   % ans = 1.39500000000000001776

So, as far as the "actual numbers in memory" are concerned, octave is doing the right thing.

If you do not care about 'actual precision' and you just want to see 'visually-desired precision', then you can create a simple function that uses "round" (i.e. to the nearest integer) under the hood. E.g.

ndigits   = @(x)    10 ^ floor( log10( x ) );
sigdigits = @(x, s) round( x / ndigits(x) * 10^s ) / 10^s * ndigits(x);

sigdigits(8.395, 2)   % ans =  8.4000
sigdigits(1.395, 2)   % ans =  1.4000

Upvotes: 0

Cris Luengo
Cris Luengo

Reputation: 60504

IEEE floating point rounds to the nearest even number when exactly half-way. So the first case it rounds down towards 8, and in the second one up towards 2.

I think this will always show 3 digits (ref):

num2str(x,'%4.2f')

Upvotes: 2

Related Questions