Indy Header Last-Modified - Invalid argument to date encode

Question

I am using TIdHTTP from Indy. I am issuing 2 different requests, one header contains the 'Last-Modified' tag, the other one does not. The header with the tag, throws an exception:

'Invalid Argument to date encode'

I already came across this question where Remy Lebeau said, that TIdHttp is now able to parse ISO8601 dates, but it doesnt seem to work for me. As you can see below, i am not doing anything with the component other than changing the UserAgent. Am I missing something?

url :=  'https://api.priceapi.com/v2/jobs/' + JobID+ '?token=' + Token;
http := TIdHTTP.Create(nil);
http.Request.UserAgent := 'XXXXX'; //Some UserAgent
try
  ssl := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
  try
    http.IOHandler := ssl;
    try
      jo := TJsonObject.ParseJSONValue(http.get(url)) as TJSONObject;
      result := jo.GetValue('status').Value;
    finally
    end;
  finally
    ssl.Free;
  end;
finally
  http.Free;
end;

Header with Last-Modified:

Cache-Control: no-cache
Content-Disposition: attachment; 
filename="20181025145103_google_shopping_de_5bd1d857bbd7e520c12841d7.json"
Content-Transfer-Encoding: binary
Content-Type: application/json
Last-Modified: 2018-10-25 14:51:23 +0000
Vary: Origin
X-Accel-Buffering: no
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: b05aa8fe-7ea9-4152-8470-a75f9816549f
X-Runtime: 0.099212
X-XSS-Protection: 1; mode=block
transfer-encoding: chunked
Connection: keep-alive

Header without Last-Modified:

Cache-Control: max-age=0, private, must-revalidate', nil
Content-Type: application/json; charset=utf-8', nil
ETag: W/"43c4a8865a5ebe565f3920779a962e93"', nil
Vary: Origin', nil
X-Content-Type-Options: nosniff', nil
X-Frame-Options: SAMEORIGIN', nil
X-Request-Id: 344ac82e-0d14-4838-ae7e-627c79b78edc', nil
X-Runtime: 0.062357', nil
X-XSS-Protection: 1; mode=block', nil
Content-Length: 157', nil
Connection: Close', nil

StackTrace:

:744717d2 KERNELBASE.RaiseException + 0x62
HIWBase.System.SysUtils.ConvertError($3B68860)
HIWBase.System.SysUtils.EncodeDate(???,???,???)
HIWBase.IdGlobalProtocols.RawStrInternetToDateTime('07:53:37 +0000',0)
HIWBase.IdGlobalProtocols.GMTToLocalDateTime('07:53:37 +0000')
HIWBase.IdHTTPHeaderInfo.TIdEntityHeaderInfo.ProcessHeaders
HIWBase.IdHTTPHeaderInfo.TIdResponseHeaderInfo.ProcessHeaders
HIWBase.IdHTTP.TIdHTTPProtocol.RetrieveHeaders(255)
HIWBase.IdHTTP.TIdCustomHTTP.DoRequest('GET','My URL',nil,$ADF09E0,(...))
HIWBase.IdHTTP.TIdCustomHTTP.Get('My URL',$ADF09E0,(...))
HIWBase.IdHTTP.TIdCustomHTTP.Get('My URL',(...))
HIWBase.IdHTTP.TIdCustomHTTP.Get('My URL')

I am on Indy version 10.6.2.5311

Answer 1

The Last-Modified header is defined in RFC 2616 Section 14.29 ¹ as:

Last-Modified  = "Last-Modified" ":" HTTP-date

^{1: An equivalent definition appears in RFC 7232 Section 2.2.}

HTTP-date is defined in RFC 2616 Section 3.3 ² as:

HTTP-date    = rfc1123-date | rfc850-date | asctime-date
rfc1123-date = wkday "," SP date1 SP time SP "GMT"
rfc850-date  = weekday "," SP date2 SP time SP "GMT"
asctime-date = wkday SP date3 SP time SP 4DIGIT
date1        = 2DIGIT SP month SP 4DIGIT
               ; day month year (e.g., 02 Jun 1982)
date2        = 2DIGIT "-" month "-" 2DIGIT
               ; day-month-year (e.g., 02-Jun-82)
date3        = month SP ( 2DIGIT | ( SP 1DIGIT ))
               ; month day (e.g., Jun  2)
time         = 2DIGIT ":" 2DIGIT ":" 2DIGIT
               ; 00:00:00 - 23:59:59
wkday        = "Mon" | "Tue" | "Wed"
             | "Thu" | "Fri" | "Sat" | "Sun"
weekday      = "Monday" | "Tuesday" | "Wednesday"
             | "Thursday" | "Friday" | "Saturday" | "Sunday"
month        = "Jan" | "Feb" | "Mar" | "Apr"
             | "May" | "Jun" | "Jul" | "Aug"
             | "Sep" | "Oct" | "Nov" | "Dec"

^{2: Equivalent definitions appear in RFC 7231 Section 7.1.1.1.}

The Last-Modified value you have shown does not match any of those formats allowed by HTTP.

TIdHTTP uses Indy's GMTToLocalDateTime() function to parse the Last-Modified (and Date and Expires) header. That function is shared by HTTP, IMAP, NNTP, and email components, so it is a little more flexible in the date/time formats that it supports. For instance, it does parse ISO 8601, which you claim the Last-Modified value is. However, the value you have shown does not actually conform to ISO 8601, either. If it had, it would have looked more like this instead:

Last-Modified: 2018-10-26T08:37:01+00:00

To make matters worse, according to the stack trace you have provided, GMTToLocalDateTime() is being called without any date portion at all:

HIWBase.IdGlobalProtocols.GMTToLocalDateTime('07:53:37 +0000')

The only way that can happen in TIdHTTP is if the HTTP server is sending a Last-Modified (or Date or Expires) header with that exact value, which is also not conformant to the HTTP or ISO 8601 standards, and is not handled as-is by GMTToLocalDateTime().

In short, the API you are querying is sending an illegal date/time format that TIdHTTP does not support parsing (which is ironic, because the main https://www.priceapi.com website does send properly formatted HTTP date/time strings). You should contact the website admin and report that their API server is violating HTTP protocol standards in this regard.

That being said, GMTToLocalDateTime() DOES NOT raise an 'Invalid Argument to date encode' exception when it encounters a malformed date/time string. It returns a TDateTime of 0.0 instead. The only way you could be seeing that exception is if you are running your code inside the debugger. When GMTToLocalDateTime() is given a malformed date/time string, it is possible that it may extract numeric components that it thinks are valid but then fails when it attempts to encode the final TDateTime with them. The exception you are seeing comes from the RTL's EncodeDate() function when it is given an invalid month/day/year as input. But GMTToLocalDateTime() catches that exception internally. Your code will never see it at runtime, only the debugger can see it.