Python SimpleCookie and JSON Value

Question

I've run into some problems using Python's SimpleCookie when using a JSON string as a value.

In [1]: from http.cookies import SimpleCookie

In [2]: cookie = SimpleCookie('x=1; json={"myVal":1}; y=2')

In [3]: cookie.keys()
Out[3]: dict_keys(['x'])

In [4]: cookie = SimpleCookie('x=1; y=2')

In [5]: cookie.keys()
Out[5]: dict_keys(['y', 'x'])

Not only the JSON string is missing, but every other value coming afterwards as well. I'm wondering now if this is a Python-related bug as the characters should all be fine to be used in a cookie?

jonrsharpe · Accepted Answer

Currently you're trying to use an object as the value, not a JSON string representation of that object. You need to escape the JSON string within your argument string, as follows:

>>> from http.cookies import SimpleCookie
>>> cookie = SimpleCookie(r'x=1; json="{"myVal":1}"; y=2')
                        # ^ note raw string    ^ and single backslashes
>>> cookie.keys()
dict_keys(['json', 'x', 'y'])

It does seem odd that a malformed string is quietly consumed as far as it can go and the rest is ditched, though; I'd expect a ValueError or something for your input. The parser just stops and returns the result so far when it's run out of things that match the regex.

Given Marius's answer, which seemed to work on 3.3.2 but doesn't work in 3.5.2 (and, to me, doesn't look like it should work anywhere; an implicit JSON package import?!) I went searching for when it changed. Using this script:

from http.cookies import SimpleCookie
import json
from sys import version

print(version)

cookie1 = SimpleCookie('x=1; json=json.dumps({"myVal":1}); y=2')
print('Marius  ', cookie1.keys(), repr(cookie1.get('json')), sep='	')

cookie2 = SimpleCookie(r'x=1; json="{"myVal":1}"; y=2')
print('Jonathan', cookie2.keys(), repr(cookie2.get('json')), sep='	')

cookie3 = SimpleCookie('x=1; json={"myVal":1}; y=2')
print('Bernhard', cookie3.keys(), repr(cookie3.get('json')), sep='	')

and pyenv on Mac OS X gives the following results for 3.3:

3.3.0 (default, Jul  7 2016, 10:47:41)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['x', 'y', 'json'])   
Jonathan    dict_keys(['x', 'y', 'json'])   
Bernhard    dict_keys(['x', 'y', 'json'])   

3.3.1 (default, Jul  7 2016, 10:53:06)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['json', 'x', 'y'])   
Jonathan    dict_keys(['json', 'x', 'y'])   
Bernhard    dict_keys(['json', 'x', 'y'])   

3.3.2 (default, Jul  6 2016, 22:02:23)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['json', 'y', 'x'])   
Jonathan    dict_keys(['json', 'y', 'x'])   
Bernhard    dict_keys(['json', 'y', 'x'])   

# ...loses 'json'

3.3.3 (default, Jul  7 2016, 10:57:02)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['x', 'y'])   None
Jonathan    dict_keys(['x', 'y', 'json'])   
Bernhard    dict_keys(['x', 'y'])   None

3.3.4 (default, Jul  7 2016, 10:59:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['y', 'x'])   None
Jonathan    dict_keys(['y', 'x', 'json'])   
Bernhard    dict_keys(['y', 'x'])   None

3.3.5 (default, Jul  7 2016, 11:01:45)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['y', 'x'])   None
Jonathan    dict_keys(['json', 'y', 'x'])   
Bernhard    dict_keys(['y', 'x'])   None

# ...and now 'y'!

3.3.6 (default, Jul  7 2016, 11:03:40)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['x'])    None
Jonathan    dict_keys(['json', 'x', 'y'])   
Bernhard    dict_keys(['x'])    None

From there it seems to be stable:

3.4.4 (default, Jul  7 2016, 11:13:43)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['x'])    None
Jonathan    dict_keys(['json', 'y', 'x'])   
Bernhard    dict_keys(['x'])    None

3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
Marius      dict_keys(['x'])    None
Jonathan    dict_keys(['x', 'json', 'y'])   
Bernhard    dict_keys(['x'])    None

The version from 3.3.6 onwards was apparently a security fix; see this bug report. This was also applied in 3.2.6, so:

3.2.4 (default, Jul  7 2016, 11:05:33)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['y', 'x', 'json'])   
Jonathan    dict_keys(['y', 'x', 'json'])   
Bernhard    dict_keys(['y', 'x', 'json'])   

3.2.5 (default, Jul  7 2016, 11:07:15)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['y', 'x', 'json'])   
Jonathan    dict_keys(['y', 'x', 'json'])   
Bernhard    dict_keys(['y', 'x', 'json'])   

# ...loses 'y'?!

3.2.6 (default, Jul  7 2016, 11:09:00)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.72)]
Marius      dict_keys(['x', 'json'])    
Jonathan    dict_keys(['y', 'x', 'json'])   
Bernhard    dict_keys(['x', 'json'])

This behaviour is different to 3.3.6 with the same bugfix! In fact, it sets the value of json in the cookie to '{', discarding the rest of that value and y.

I conclude that:

my version is more widely applicable;
Marius's version won't actually help the OP, as it has the same keys as their original version; and
something weird was happening with this functionality in 3.2 and 3.3!

Python SimpleCookie and JSON Value

Answers (2)

Related Questions