Authenticate and sign HTTP request

Question

I'm looking for some guidance/opinion on best practice for both authenticating and signing (integrity checking) an HTTP request. I'm leaving this fairly open for language and technology used, but the basic parameters and requirements are as follows:

• Need both authentication and integrity checking.
• The request contains sensitive data and will be passed over HTTPS, but I cannot assume that the request cannot be sniffed by an attacker, either before it is HTTPS encrypted (at the client end by some illicit piece of installed software) or at the server end, e.g. by a sniffer placed between the HTTPS endpoint and the actual server. The HTTPS encryption/decryption handling has been passed off to an external load balancer so there is a small window for an attacker to insert a sniffer between the load balancer and the server.
• I need to make sure that a transaction cannot be faked.
• I need to make sure that a transaction cannot be replayed.
• Requests can be GET or POST, and so any extra data needs to stay within the maximum size limit of a GET request.

Things that we have considered:

• Username/password on each request. This guarantees authentication I guess, but if an attacker can sniff the username/password pair then it all fails.
• Private key signing of each request. The server will have the public key, only the client will have the private key. This guarantees integrity because only the client could have generated the signature, but the server can check the signature. It generally guarantees authentication as well because the private key is not part of the request data and cannot be sniffed. However by itself it does not stop the transaction being replayed, because 2 transactions with the same data will have the same signature.
• Using a cryptographic client nonce ( https://en.wikipedia.org/wiki/Cryptographic_nonce ) as part of the request data and including that in the data to be signed. We have had issues with these especially when they are generated on the client side because if they aren't sufficiently random then an attacker can find out how they are generated then the attacker can generate a sequence of nonces of their own, in advance of the client generating the same sequence, which can lead to a denial of service attack because the client will be trying to re-use a nonce that has already been used by the attacker. Generating nonces on the server side has been considered but it's an extra transaction and potentially a performance issue.
• Including a date/time in the request data, however this can cause issues where the client's clock and the server's clock drift out of sync.

In case some admin decides to flag this as a duplicate, here are other Qs that I have considered that don't quite address the full scope of this issue:

• What is the current standard for authenticating Http requests (REST, Xml over Http)?
• Authenticity and Integrity of HTTP Requests

Answer 1

Assuming a 1-1 relationship between client and server, a HMAC with counter would solve this problem.

The client and server have a shared secret 128 bit key.

The client sends messages that have an HMAC with the secret key, and a counter. SHA-256 is recommended, as SHA-1 is on its way out (well SHA-1 HMAC is still considered secure, however that's another story).

e.g.

example.com?message=foo&counter=1&hmac=35ed8c76e7b931b06f00143b70307e90f0831682e7bdce2074ebb5c509d16cfb`

(This tool used for this post with secret bar.)

The HMAC is calculated over message=foo&counter=1.

Now, once the HMAC has been authenticated by the server and the counter checked, the server's counter is incremented to 2. Now the server will not accept any authenticated messages with a counter less than 2.

JSON Web Tokens could be used so that you are doing the above in a standard format. You could do the above with multiple clients, however you would need to keep track of the counter server-side for each client, and the client would have to identify itself in the message. Managing the shared secrets is the trickiest bit if you decide on a different key per client.