CODEWITHSUNDEEP
pythonijson
ballade4op52
ballade4op52

Reputation: 2267

Is it possible to use ijson to create and load JSON objects in memory (not from/to a file)?

I'm trying to use ijson instead of json to be able to efficiently dump/load dictionaries to/from strings (in-memory, not from a file) [1].

Are there any examples for ijson analogous to standard dumping/loading with json? All sources I've seen that use ijson have examples with files only.

[1] -- Dictionaries are being converted to strings in order to store them in a set

Upvotes: 2

Views: 2061

Answers (2)

ballade4op52
ballade4op52

Reputation: 2267

To complement omri_saadon's answer in the vein of performance,

Here is a link (serializer speed comparisons) that measures some lesser known JSON modules, one of which was found to effectively outperform the others (msgpack) and can be used interchangeably with json.

Hence, instead of ijson or standard json, msgpack is presently being used for the dict/str conversions.

To install: pip install msgpack

To use: import msgpack as json # can use dumps/loads functions

Upvotes: 3

omri_saadon
omri_saadon

Reputation: 10631

You can reach both goals (dict to string and string to dict without using the json lib).

Let's look at the above example:

import ast
di = {"a":2, "b": 3}

print (str(di))
>>> "{'a': 2, 'b': 3}"

print (type(str(di)))
>>> <class 'str'>

print (ast.literal_eval(str(di)))
>>> {'a': 2, 'b': 3}

print (type(ast.literal_eval(str(di))))
>>> <class 'dict'>

You have a dictionary, In order to turn it into string, you just need to cast it into str.

If you want to return the str to dict you can use the ast lib.

You can read more about ast.literal_eval

ast.literal_eval(node_or_string)

Safely evaluate an expression node or a Unicode or Latin-1 encoded string containing a Python literal or container display.

EDIT:

After further investigation and following this SO question It seems that using json would get you to a faster results (performance).

Look at abarnert answer at the attached link, specifacally on:

Why is json.loads so much faster ?

Because Python literal syntax is a more complex and powerful language than JSON, it's likely to be slower to parse. And, probably more importantly, because Python literal syntax is not intended to be used as a data interchange format (in fact, it's specifically not supposed to be used for that), nobody is likely to put much effort into making it fast for data interchange.

import json
di = {"a":2, "b": 3}

print (json.dumps(di))
>>> '{"a": 2, "b": 3}'

print (type(json.dumps(di)))
>>> <class 'str'>

print (json.loads(json.dumps(di)))
>>> {'a': 2, 'b': 3}

print (type(json.loads(json.dumps(di))))
>>> <class 'dict'>

Upvotes: 1

Related Questions