CODEWITHSUNDEEP
pythonjson
Kevin Meyer
Kevin Meyer

Reputation: 2966

In python, is there a way to extract a embedded json string inside random text?

So I'm parsing a really big log file with some embedded json.

So I'll see lines like this

foo="{my_object:foo, bar:baz}" a=b c=d

The problem is that since the internal json can have spaces, but outside of the JSON, spaces act as tuple delimiters (except where they have unquoted strings . Huzzah for whatever idiot thought that was a good idea), I'm not sure how to figure out where the end of the JSON string is without reimplementing large portions of a json parser.

Is there a json parser for Python where I can give it '{"my_object":"foo", "bar":"baz"} asdfasdf', and it can return ({'my_object' : 'foo', 'bar':'baz'}, 'asdfasdf') or am I going to have to reimplement the json parser by hand?

Upvotes: 3

Views: 556

Answers (3)

Kevin Meyer
Kevin Meyer

Reputation: 2966

Found a really cool answer. Use json.JSONDecoder's scan_once function

In [30]: import json

In [31]: d = json.JSONDecoder()

In [32]: my_string = 'key="{"foo":"bar"}"more_gibberish'

In [33]: d.scan_once(my_string, 5)
Out[33]: ({u'foo': u'bar'}, 18)

In [37]: my_string[18:]
Out[37]: '"more_gibberish'

Just be careful

In [38]: d.scan_once(my_string, 6)
Out[38]: (u'foo', 11)

Upvotes: 1

Emil Sit
Emil Sit

Reputation: 23562

Use shlex and json.

Something like:

import shlex
import json

def decode_line(line):
    decoded = {}
    fields = shlex.split(line)
    for f in fields:
        k, v = f.split('=', 1)
        if k == "foo":
            v = json.loads(v)
        decoded[k] = v
    return decoded

This does assume that the JSON inside the quotes is quoted properly.

Here's a short example program that uses the above:

import pipes

testdict = {"hello": "world", "foo": "bar"}
line = 'foo=' + pipes.quote(json.dumps(testdict)) + ' a=b c=d'
print line
print decode_line(line)

With output:

foo='{"foo": "bar", "hello": "world"}' a=b c=d
{'a': 'b', 'c': 'd', 'foo': {u'foo': u'bar', u'hello': u'world'}}

Upvotes: 0

Ignacio Vazquez-Abrams
Ignacio Vazquez-Abrams

Reputation: 799370

Match everything around it.

>>> re.search('^foo="(.*)" a=.+ c=.+$', 'foo="{my_object:foo, bar:baz}" a=b c=d').group(1)
'{my_object:foo, bar:baz}'

Upvotes: 0

Related Questions