CODEWITHSUNDEEP
pythondecimalroundingtruncate
Zak Stucke
Zak Stucke

Reputation: 452

Decimal.quantize rounding error

I've stumbled across an issue with the quantize method when using the Decimal datatype, it seems to be giving rounding errors: 

Decimal('1.0055').quantize(Decimal('0.000')) # Should output 1.006
>> Decimal('1.006') # CORRECT output
Decimal('1.0045').quantize(Decimal('0.000')) # Should output 1.005
>> Decimal('1.004') # INCORRECT output

Why is it sometimes rounding up and sometimes rounding down?

Upvotes: 4

Views: 1195

Answers (1)

NPE
NPE

Reputation: 500177

tl;dr This is what's called bankers' rounding.

The default rounding mode is

ROUND_HALF_EVEN (to nearest with ties going to nearest even integer)

which is exactly what you're seeing: ties between 4 and 5, and between 5 and 6, are going to the even numbers (4 and 6 respectively).

If you want a different rounding mode, you need to specify it explicitly.

The choices are:

ROUND_CEILING (towards Infinity),
ROUND_DOWN (towards zero),
ROUND_FLOOR (towards -Infinity),
ROUND_HALF_DOWN (to nearest with ties going towards zero),
ROUND_HALF_EVEN (to nearest with ties going to nearest even integer),
ROUND_HALF_UP (to nearest with ties going away from zero), or
ROUND_UP (away from zero).
ROUND_05UP (away from zero if last digit after rounding towards zero would have been 0 or 5; otherwise towards zero)

Documentation: https://docs.python.org/2/library/decimal.html#decimal.Context

See the rounding argument to quantize(): https://docs.python.org/2/library/decimal.html#decimal.Decimal.quantize

Upvotes: 3

Related Questions