CODEWITHSUNDEEP
windowscode-signingsigntooldigicert
Martin Ba
Martin Ba

Reputation: 38941

Is there any way to get details from signtool.exe for timestamp server errors?

How can I get better timestamp server error messages when signing binaries?

SignTool Error: The specified timestamp server either could not be reached or 
 returned an invalid response. 
 SignTool Error: An error occurred while attempting to sign: ...

Let's start out with a short rant:

As far as error messages go, this is somewhere between but-they-tried, stupid and plain insulting.

signtool.exe needs to talk to the timestamp server via HTTP, so WTF are we not getting told what exactly failed? This is such a lazy error message. "Could not be reached or returned an invalid response." - oh, my.

Maybe I should be content that the tool at least tells me that the error was with the timestamp server, but that does get us only so far.

/rant over

We will get temporary signing errors from http://timestamp.digicert.com from time to time. (Every odd month maybe.)

The thing is, our build scripts already try to circumnavigate this:

Our latest log looks something like this:

[2023-01-24 12:58:11.038] : (INFO ) Sign *all* 1337 binaries at <~redacted~>: This will be SLOW ...
...
[2023-01-24 12:58:11.334] : OK: HEAD request to 'http://timestamp.digicert.com' returned http status code NotFound/404 (any HTTP status code is fine). 
 [2023-01-24 12:58:11.335] : #100 ... 
 [2023-01-24 12:58:11.354] : Invoke-SignTool with 100 files ... 
 [2023-01-24 12:58:11.359] : Executing '&"C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64\signtool.exe" sign /v /sm /s My /n "~certname~" /fd SHA256 /tr http://timestamp.digicert.com /td SHA256 "..."' ... 
 The following certificate was selected: 
     Issued to: ~certname~ 
  
     Issued by: DigiCert Trusted G4 Code Signing RSA4096 SHA384 2021 CA1 
  
     Expires:   ~...
  
     SHA1 hash: ~...
  
  
 Done Adding Additional Store 
 SignTool Error: The specified timestamp server either could not be reached or 
 returned an invalid response. 
 SignTool Error: An error occurred while attempting to sign: ...
  
  x repeat
  ...  
  
 Successfully signed: .\~....dll 
  
 Successfully signed: .\~....dll
 
 x repeat
 ...
 
  
 Number of files successfully Signed: 50 
  
 Number of warnings: 0 
  
 Number of errors: 50 
  
 SignTornadoBinaries error: (ERROR) Exec (exit code <1>): NONZERO exit code
 Waiting for 10 seconds... 
   Retrying...   
 [2023-01-24 13:00:13.022] : (INFO ) Sign *all* 1287 binaries at <~redacted~>: This will be SLOW ... 
 [2023-01-24 13:00:13.319] : OK: HEAD request to 'http://timestamp.digicert.com' returned http status code NotFound/404 (any HTTP status code is fine). 
 [2023-01-24 13:00:13.320] : #100 ... 
 [2023-01-24 13:00:13.338] : Invoke-SignTool with 100 files ... 
 [2023-01-24 13:00:13.343] : Executing '&"C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64\signtool.exe" sign /v /sm /s My /n "~certname~" /fd SHA256 /tr http://timestamp.digicert.com /td SHA256 ...
 The following certificate was selected: 
     Issued to: ~certname~ 
     ....
     
 SignTool Error: The specified timestamp server either could not be reached or 
 returned an invalid response. 
 SignTool Error: An error occurred while attempting to sign: .\x64-vc143\....dll 
 ...
 
Number of files successfully Signed: 50 
  
 Number of warnings: 0 
  
 Number of errors: 50 
  
 SignTornadoBinaries error: (ERROR) Exec (exit code <1>): NONZERO exit code

 Waiting for 10 seconds... 
   Retrying...   
   
[2023-01-24 13:02:16.970] : (INFO ) Sign *all* 1237 binaries at <~redacted~>: This will be SLOW ... 
 [2023-01-24 13:02:17.283] : OK: HEAD request to 'http://timestamp.digicert.com' returned http status code NotFound/404 (any HTTP status code is fine). 
 [2023-01-24 13:02:17.284] : #100 ... 
 [2023-01-24 13:02:17.301] : Invoke-SignTool with 100 files ... 
 [2023-01-24 13:02:17.306] : Executing ...

 
 Number of files successfully Signed: 50 
  
 Number of warnings: 0 
  
 Number of errors: 50

Note that this 100-files-at-once normally works multiple times a week, sometimes multiple times a day (depending on our build schedule -- only production builds are signed.)

As we also can see here, here, the error was reported for exactly the first 50 files, and the second 50 files are then signed. Our retry is 3x with 10 sec wait, so after that the build just aborts: As far as I am concerned, this 50/50 split is a "clear" indication that this is not about a flaky network or some load issue on the digicert server or whatever.

Do note however, that I've also seen logs with 99/1 or maybe 80/20.

So, to get back to an actual question:

Is there any way to get more detailed errors from signtool?

Barring that, is there an alternative toolchain (some powershell commandlets maybe?) that will let me sign my binaries and give me useful error messages?

Aside: If you happen to know if a different number of files to pass to signtool at once may result in a more stable setup, please let me know.

Case in point:

In my case above, the 50/50 failure ratio came from our misbehaving firewall. It would block every odd HTTP Timestamp Protocol request with a HTTP/403.

If signtool would log the HTTP response including the content body, because otherwise I wouldn't have known that the response came from the firewall instead of from the actual target server then analyzing the problem would have been instantaneous!

Upvotes: 0

Views: 1259

Answers (1)

Emmanuel Bourg
Emmanuel Bourg

Reputation: 11058

You can try Jsign, if the timestamp server fails it can retry after a delay and/or fallback to other servers. The syntax looks like this:

jsign --tsaurl http://timestamp.sectigo.com,http://timestamp.digicert.com,...
      --tsretries 10
      --tsretrywait 5
      --keystore keystore.p12
      --alias test
      --storepass password
      file1.exe file2.exe ...

(disclaimer: I'm the author of this project)

Upvotes: 1

Related Questions