CODEWITHSUNDEEP
pythondatetimepython-3.x
kjonsson
kjonsson

Reputation: 2799

Converting datetimes to timestamps and back again

I'm having some trouble with datetime in Python. I tried converting a datetime to a timestamp and then back again and no matter how I try the end result is not the same. I always end up with a datetime of datetime(2014, 1, 30, 23, 59, 40, 1998).

import datetime

a = datetime.datetime.timestamp(datetime.datetime(2014, 1, 30, 23, 59, 40, 1999))
b = datetime.datetime.fromtimestamp(a)

print(b)

Upvotes: 5

Views: 2078

Answers (3)

Mark Ransom
Mark Ransom

Reputation: 308520

Very few floating point numbers with a decimal fraction can be exactly represented as a binary floating point number; there will usually be some very small error. Sometimes it will be smaller than the desired number, and sometimes it will be larger, but it should always be very very close. Your example's exact value is 1391147980.0019989013671875 which is within 0.1 microsecond of what you specified.

The conversion from a floating point timestamp back to a datetime should employ rounding to make sure the round-trip conversion gives the same value as the original. As noted by J.F. Sebastian, this was entered as a bug against Python 3.4; it is claimed to be fixed in later releases, but it still exists in Python 3.5.0 using the same values as given in the question. Running a test similar to nigel222 shows an almost 50/50 split between exact matches and results that are low by 1 microsecond.

Since you know that the original value was an integral number of microseconds, you can add an offset that ensures the binary floating point value is always higher than the decimal value while still being small enough that it doesn't affect the result when properly rounded. Since rounding should occur at 0.5 microseconds the ideal offset would be half of that, or 0.25 microseconds.

Here are the results in Python 3.5.0:

>>> a = datetime.datetime.timestamp(datetime.datetime(2014, 1, 30, 23, 59, 40, 1999))
>>> b = datetime.datetime.fromtimestamp(a)
>>> a
1391147980.001999
>>> b
datetime.datetime(2014, 1, 30, 23, 59, 40, 1998)

>>> b = datetime.datetime.fromtimestamp(a + 0.00000025)
>>> b
datetime.datetime(2014, 1, 30, 23, 59, 40, 1999)

>>> counter={}
>>> for i in range(0,1000000):
   # fuzz up some random-ish dates
   d = datetime.datetime(1990+(i%20), 1+(i%12), 1+(i%28), i%24, i%60, i%60, i)
   ts = datetime.datetime.timestamp(d)
   b = datetime.datetime.fromtimestamp(ts + 0.00000025)
   msdif = d.microsecond - b.microsecond 
   if msdif in counter:
     counter[msdif] += 1
   else:
     counter[msdif]=1
   assert b.day==d.day and b.hour==d.hour and b.minute==d.minute and b.second==d.second

>>> counter
{0: 1000000}

Upvotes: 1

jfs
jfs

Reputation: 414795

It is a known Python 3.4 issue:

>>> from datetime import datetime
>>> local = datetime(2014, 1, 30, 23, 59, 40, 1999)
>>> datetime.fromtimestamp(local.timestamp())
datetime.datetime(2014, 1, 30, 23, 59, 40, 1998)

Note: the microsecond is gone. The .timestamp() already returns result that is slightly less than 1999 microseconds:

>>> from decimal import Decimal
>>> local.timestamp()
1391126380.001999
>>> Decimal(local.timestamp())
Decimal('1391126380.0019989013671875')

The rounding is fixed in the next 3.4, 3.5, 3.6 releases:

>>> from datetime import datetime
>>> local = datetime(2014, 1, 30, 23, 59, 40, 1999)
>>> datetime.fromtimestamp(local.timestamp())
datetime.datetime(2014, 1, 30, 23, 59, 40, 1999)

To workaround the issue, you could use the explicit formula:

>>> from datetime import datetime, timedelta
>>> local = datetime(2014, 1, 30, 23, 59, 40, 1999)
>>> datetime.utcfromtimestamp(local.timestamp())
datetime.datetime(2014, 1, 30, 23, 59, 40, 1998) # UTC time
>>> datetime(1970, 1, 1) + timedelta(seconds=local.timestamp())
datetime.datetime(2014, 1, 30, 23, 59, 40, 1999) # UTC time

Note: the input in all examples is the local time but the result is UTC time in the last one.

Upvotes: 4

nigel222
nigel222

Reputation: 8222

That last number is microseconds ... are the internals that accurate? Let's find out.

counter={}
for i in range(0,1000000,43): 
   # fuzz up some random-ish dates
   d = datetime.datetime( 1990+(i%20), 1+(i%12), 1+(i%28), i%24, i%60, i%60, i)
   ts=datetime.datetime.timestamp( d)
   b = b=datetime.datetime.fromtimestamp(ts)
   msdif=d.microsecond-b.microsecond 
   if msdif in counter:
     counter[msdif] += 1
   else:
     counter[msdif]=1
   assert b.day==d.day and b.hour==d.hour and b.minute==d.minute and b.second=d.second

  >>>
  >>> counter
  {1: 23256}
  >>>

I do believe you have found an off-by-one-microsecond error in the datetime library, unless there is something perverse buried in the specifications.

(I was expecting a spread around zero, reflecting rounding errors of some sort)

Upvotes: 3

Related Questions