CODEWITHSUNDEEP
python
Anders
Anders

Reputation: 17564

Problem with precision in a python script

I need to convert numbers to a flat file format 0.57 becomes 000000000000000057 in this format (padding zeroes) My method

My method

def toAmount18(a):
   return str(int(a*100)).zfill(18)

For 0.57 it outputes 000000000000000056

The problem lies in a*100 it outputs 56.99999999999999 instead of 57. Ideas?

Upvotes: 0

Views: 98

Answers (3)

Terry Spotts
Terry Spotts

Reputation: 4075

You could avoid maths altogether and invite regex to the party:

re.search(r'\.(\d{,2})', str(0.5)).group(1).zfill(18)
'000000000000000005'

re.search(r'\.(\d{,2})', str(0.57)).group(1).zfill(18)
'000000000000000057'

re.search(r'\.(\d{,2})', str(0.571)).group(1).zfill(18)
'000000000000000057'

re.search(r'\.(\d{,2})', str(0.579)).group(1).zfill(18)
'000000000000000057'

Might need some more sample inputs to get this tuned correctly for the expected output.

Upvotes: 0

Serge Ballesta
Serge Ballesta

Reputation: 149185

It is a floating point accuracy problem. In the early days of Fortran IV, we learned that the conversion from a float to an integer value was I = F + .5

Here you can still use the same trick:

def toAmount18(a):
   return str(int(a*100 + .5)).zfill(18)

or use more modern tools like round

   return str(round(a*100)).zfill(18)

The nice point with the manual method is that you can specify the truncation magnitude to your needs:

  return str(int(a*100 + .00005)).zfill(18)

will only round after the 4th decimal.

Upvotes: 1

CastelPablito
CastelPablito

Reputation: 1

Using int() as default works like a floor on the input number.

Try to use round() method instead of int() as shown in example: https://www.programiz.com/python-programming/methods/built-in/round

You can also add an extra security layer (try statement) which will handle wrong type exceptions.

Upvotes: 0

Related Questions