Inconsistency in HMAC signature generation in Python 3?

Question

Running the create_api_signature() method in a python terminal always return the same value, while it return different values when run in a test.

import hashlib
import hmac
import json

import unittest


def create_api_signature(_method, _url, _body, _timestamp, _secret_key):
    unicode_signature = _method.upper() + _url + json.dumps(_body) + str(_timestamp)

    s = hmac.new(_secret_key.encode(), unicode_signature.encode(), hashlib.sha256).hexdigest()

    return s


class MyTestCase(unittest.TestCase):
    def test_create_signature(self):
        method = 'post'
        url = 'https://api.alpha.example.com/v1/tiers'
        body = {
            "mail": "test@gmail.com",
            "mot_de_passe": "MyComplexPassword",
        }
        timestamp = 1433948791
        secret_key = 'SECRET_KEY'

        signature = create_api_signature(method, url, body, timestamp, secret_key)
        expected_signature = '136b629ac9744258cf558c2d541d563cc3ce647d91ead707ae4d42d49ade50c7'

        self.assertEqual(expected_signature, signature)


if __name__ == '__main__':
    unittest.main()

Error

Failure
Expected :'136b629ac9744258cf558c2d541d563cc3ce647d91ead707ae4d42d49ade50c7'
Actual   :'88a138592ea7eae50040655387a878d15fd4ab4ade5d7d769a36bf9300cb3f9e'
 <Click to see difference>

Traceback (most recent call last):
  File "/home/elopez/projects/portal/tests/test_services.py", line 98, in test_create_signature
    self.assertEqual(expected_signature, signature)
AssertionError: '136b629ac9744258cf558c2d541d563cc3ce647d91ead707ae4d42d49ade50c7' != '88a138592ea7eae50040655387a878d15fd4ab4ade5d7d769a36bf9300cb3f9e'
- 136b629ac9744258cf558c2d541d563cc3ce647d91ead707ae4d42d49ade50c7
+ 88a138592ea7eae50040655387a878d15fd4ab4ade5d7d769a36bf9300cb3f9e

Answer 1

I went to #python's IRC and get the following answer by cdunklau

cdunklau: run this a few times and you'll see why

PYTHONHASHSEED=random python3.2 -c "import json; print(json.dumps({'mail': 'value', 'mot_de_passe': 'othervalue'}))"

cdunklau: you're depending on the order of a dict

Mutability

$ for i in {1..20}; do PYTHONHASHSEED=random python3.4 -c "import json; print(json.dumps({'mail': 'value', 'mot_de_passe': 'othervalue'}))"; done

give the following result (notice the JSON data are not always in the same order):

{"mail": "value", "mot_de_passe": "othervalue"}
{"mot_de_passe": "othervalue", "mail": "value"}
{"mail": "value", "mot_de_passe": "othervalue"}
{"mot_de_passe": "othervalue", "mail": "value"}
{"mail": "value", "mot_de_passe": "othervalue"}
{"mot_de_passe": "othervalue", "mail": "value"}
{"mot_de_passe": "othervalue", "mail": "value"}
…

Solution

I changed from:

 body = {
         "mail": "test@gmail.com",
         "mot_de_passe": "MyComplexPassword",
 }

to a serialized dict as a binary string:

 body = b'{"mail": "test@gmail.com", "mot_de_passe": "MyComplexPassword"}'