Is a difference in timezone also taken into account while comparing two dates on linux

Question

I have a couple of timestamps that I am trying to compare.

1. startDate: 2017-04-04T21:27:37.991Z
2. endDate: 2017-04-05

I am comparing the two timestamps in bash by converting them to seconds and then putting them in an if condition like so:

startDateSecs=$(date -d ${startDate} +%s)
endDateSecs=$(date -d ${endDate} +%s)

The values I get are:
startDateSecs = 1491341257
endDateSecs = 1491330600

My timezone is currently IST (India Standard Time). I understand that the Z notation on the first timestamp means that it is offset by 0 from UTC.
When I compare the two timestamps, 2017-04-05 is lower because it takes into account my present timezone too?

Answer 1

Bash doesn’t do date comparisons.

If you want to compare dates in bash you have to convert them either into seconds since the epoch (as you are doing) or into suitably formatted strings: I often use the format +%Y%m%d:%H%M%S. The latter is somewhat faster, because when you do +%s, bash produces a string which gets further converted to a number when numerical comparison is performed; it’s better to produce a string which can be compared lexicographically as is, and is readable too.

In any case, as other answers have already pointed out, the answer to your question is that the local timezone is implied when there is no explicit timezone in the timestamp, and midnight is implied for pure dates.

Some useful examples follow:

# Choose a format
datefmt='%Y%m%d:%H%M%S'

# Prepend timezone definition to force all 
# unqualified times to be interpreted as UTC
date +"$datefmt" -d "TZ=UTC $pure_date"

# or prepend it to the command for the output to be UTC 
TZ=UTC date +"$datefmt" -d "TZ=Asia/Kolkata $pure_date"

# Convert a date from seconds since the epoch (generally output with -%s)
# to a readable format
TZ=UTC date -R -d@1491425174
Wed, 05 Apr 2017 20:46:14 +0000

# Loop for ten minutes
end_time="$(TZ=UTC date +"$datefmt" -d 'now + 10 minutes')"
while [[ "$(TZ=UTC date +"$datefmt")" < "$end_time" ]]; do
  echo Keep going...
  sleep 60
done

And so on...

Answer 2

Q: is lower because it takes into account my present timezone too?

In one word: Sometimes?.

But the correct answer is quite a bit longer. Read on.

TL;DR Simple solution, use:

endDateSecs=$(date -d "${endDate}Z" +%s)

Force an input time in UTC0. The output time is, by force of seconds, Also UTC0.

Apparently equal:

This two give the same string as a result:

$ TZ=America/New_York date -d '2017-04-05' +'%FT%T'
2017-04-05T00:00:00

$ TZ=Asia/Kolkata     date -d '2017-04-05' +'%FT%T'
2017-04-05T00:00:00

But: Are those the same point in time? No

$ TZ=America/New_York date -d '2017-04-05' +'%FT%T%z'
2017-04-05T00:00:00-0400

$ TZ=Asia/Kolkata     date -d '2017-04-05' +'%FT%T%z'
2017-04-05T00:00:00+0530

Those two "points in time" are: +0530-(-0400) 9:30 (nine and a half hours apart).

That's why it is very important to fully qualify a point in time by using a time zone definition or, if the time zone definition is missing, understand that the time will be expressed in local (whichever value of TZ is in effect when the output time value is calculated).

One very useful option of date is the use of --debug. That will clarify which TZ (there must be one chosen) is in effect for the input and the output:

$ TZ=Asia/Kolkata     date --debug -d '2017-04-05' +'%FT%T%z'
date: parsed date part: (Y-M-D) 2017-04-05
date: input timezone: +05:30 (set from TZ="Asia/Kolkata" environment value)
date: warning: using midnight as starting time: 00:00:00
date: starting date/time: '(Y-M-D) 2017-04-05 00:00:00 TZ=+05:30'
date: '(Y-M-D) 2017-04-05 00:00:00 TZ=+05:30' = 1491330600 epoch-seconds
date: output timezone: +05:30 (set from TZ="Asia/Kolkata" environment value)
date: final: 1491330600.000000000 (epoch-seconds)
date: final: (Y-M-D) 2017-04-04 18:30:00 (UTC0)
date: final: (Y-M-D) 2017-04-05 00:00:00 (output timezone TZ=+05:30)
2017-04-05T00:00:00+0530

It can be clearly seen that the input and output timezone are the same.
That is the reason why the output string is midnight (00:00:00).

Crossing Timezones:

It should be obvious now that if the input TZ is different than the output TZ, the resulting string should reflect a time offset equal to the timezones offset.

Compare:

$ TZ=Asia/Kolkata     date -d '2017-04-05T00:00:00+0530' +'%FT%T%z' ; \
> TZ=Asia/Kolkata     date -d '2017-04-05T00:00:00-0400' +'%FT%T%z'
2017-04-05T00:00:00+0530
2017-04-05T09:30:00+0530

Both values are expressed in local time (India time): The resulting indicator: +0530.
But each "point in time" is 09:30 hours apart from the other.

The key question here is: Are input TZ and output TZ equal?

Start date:

That should affect the startDate: 2017-04-04T21:27:37.991Z

It is a Zulu "point in time" (zero difference to UTC), but the output TZ is your locale value: Asia/Kolkata. This is what results:

$ TZ=Asia/Kolkata     date -d '2017-04-04T21:27:37.991Z' +'%FT%T%z'
2017-04-05T02:57:37+0530

A point in time 21:27 zulu is 02:57 ITC (next day).

date -u

Crossing several timezones as shown above would be avoided for any local timezone in effect if the date option -u (UTC0) is used:

$ TZ=Asia/Kolkata     date -ud '2017-04-04T21:27:37.991Z' +'%FT%T%z'
2017-04-04T21:27:37+0000

Note that the result is zulu time, as was the input.
As input and output are equal, the date string is equal to the source date.

Seconds

And, now, for the final twist to explain the whole issue in detail: +'%s'

The epoch timestamp (seconds) is defined in zulu time, always.

By asking for a time in seconds, the output will be referenced to UTC0 (zulu).

If the input is somewhere else, a time difference will appear:

$ TZ=Asia/Kolkata     date -d '2017-04-04T21:27:37.991-0400' +'%s%z'; \
> TZ=UTC0             date -d '2017-04-04T21:27:37.991-0400' +'%s%z'; \
> TZ=UTC0             date -d '2017-04-04T21:27:37.991+0530' +'%s%z';
1491355657+0530
1491355657+0000
1491321457+0000

The first two are the same "point in time", which expressed as epoch yields 1491355657 in UTC0, even if the timezone printed is different.

The third value is some other "point in time": 1491321457 seconds since epoch.

Maybe this will be difficult to process, but it is the exact truth.

Final

Now we can say that:

$ date -d "$startDate" +'%s'
1491341257

Will always print 1491341257 as the value is fully qualified with a timezone in zulu.

However:

$ TZ=America/New_York date -d "$endDate" +'%s%z' ; \
> TZ=Asia/Kolkata     date -d "$endDate" +'%s%z'
1491364800-0400
1491330600+0530

will change with the TZ in efect.

You can avoid that by adding the TimeZone time or by using -u

TZ=America/New_York date  -d "${endDate}IST"  +'%s%z'
TZ=America/New_York date  -d "${endDate}Z"    +'%s%z'
TZ=America/New_York date -ud "${endDate}Z"    +'%s%z'
TZ=America/New_York date -ud "${endDate}"     +'%s%z'
TZ=Asia/Kolkata     date  -d "${endDate}Z"    +'%s%z'
TZ=Asia/Kolkata     date -ud "${endDate}"     +'%s%z'

Will print:

1491330600-0400            # a different value: Crossing TZ's
1491350400-0400
1491350400+0000
1491350400+0000
1491350400+0530
1491350400+0000

Answer 3

Instead of printing as seconds +%s you can print it in local time (-R to include the offset):

$ env TZ=Asia/Kolkata date -R -d '2017-04-04T21:27:37.991Z'
Wed, 05 Apr 2017 02:57:37 +0530
$ env TZ=Asia/Kolkata date -R -d '2017-04-05'
Wed, 05 Apr 2017 00:00:00 +0530

When converted to local time, the first date is after midnight. The second date is interpreted as midnight, local time.

You can also see what midnight UTC would be in local time:

$ env TZ=Asia/Kolkata date -R -d '2017-04-05Z'
Wed, 05 Apr 2017 05:30:00 +0530