CODEWITHSUNDEEP
pythonjsonparsing
raeX
raeX

Reputation: 3557

Python/Json:Expecting property name enclosed in double quotes

I've been trying to figure out a good way to load JSON objects in Python. I send this json data:

{'http://example.org/about': {'http://purl.org/dc/terms/title': [{'type': 'literal', 'value': "Anna's Homepage"}]}}

to the backend where it will be received as a string then I used json.loads(data) to parse it.

But each time I got the same exception : 

ValueError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

I googled it but nothing seems to work besides this solution json.loads(json.dumps(data)) which personally seems for me not that efficient since it accept any kind of data even the ones that are not in json format.

Any suggestions will be much appreciated.

Upvotes: 325

Views: 975075

Answers (30)

ElmoVanKielmo
ElmoVanKielmo

Reputation: 11290

This is not JSON:

{
    'http://example.org/about': {
        'http://purl.org/dc/terms/title': [
            {'type': 'literal', 'value': "Anna's Homepage"}
        ]
    }
}

This is JSON:

{
    "http://example.org/about": {
        "http://purl.org/dc/terms/title": [
            {"type": "literal", "value": "Anna's Homepage"}
        ]
    }
}

JSON specification - RFC7159 states that a string begins and ends with quotation mark. That is ".
Single quote ' has no semantic meaning in JSON and is allowed only inside a string.

Upvotes: 332

Akash Ranjan
Akash Ranjan

Reputation: 1074

If you want to convert a json string with single quotes to python dict use ast.literal_eval()

>>> import ast
>>> payload = "{'hello': 'world'}"
>>> ast.literal_eval(payload)
{'hello': 'world'}
>>> type(ast.literal_eval(payload))
<class 'dict'>

This will convert the payload to a python dict.

Upvotes: 11

balaji k
balaji k

Reputation: 1691

import ast
import json

inpt = {'http://example.org/about': {'http://purl.org/dc/terms/title':
                                     [{'type': 'literal', 'value': "Anna's Homepage"}]}}

json_data = ast.literal_eval(json.dumps(inpt))

print(json_data)

this will solve the problem.

Upvotes: 169

Rahul Chouhan
Rahul Chouhan

Reputation: 129

For me the issue was I had to trigger AWS Lambda from Lambda's console. So, the event should be a proper JSON. I had body in event which is suppose to be a stringified object. Now, I want to convert body back to JSON using json.loads. To do this, it also has to be a proper JSON ie string keys and values should be wrapped in double quotes.

// Incorrect JSON
{
    "headers": {"abc": "value123"},
    "body": "{"userId": "12345", "courseId": "09876543"}",
    "httpMethod": "POST",
    "path": "/abc"
}

But problem is if I wrap internal keys and values in double-quotes, then the entire body can be wrapped in single-quotes. But as I said, event in Lambda's console should be a proper JSON, I cannot wrap the body in single-quotes.

So, to resolve this, I will somehow have to use double-quotes at both the places ie while wrapping entire body and while wrapping keys and values of body. So, to do this, I replaced double-quotes with backslash and double-quotes ie \". Somehow str.replace() wasn't working for me so I simply pasted the string in VS Code, I selected a double-quote, pressed Ctrl+Shift+L (to select all double-quotes in the file) and replaced with \".

// Correct JSON
{
    "headers": {"abc": "value123"},
    "body": "{\"userId\": \"12345\", \"courseId\": \"09876543\"}",
    "httpMethod": "POST",
    "path": "/abc"
}

And then it worked.

Upvotes: 1

lav
lav

Reputation: 51

The following worked for me on bash command line: Remember to escape all double quotes and there should be no comma after the last value:

curl --location "http://${MODEL_IP}/v1/models/model:predict" \
--header "Content-Type: application/json" \
--data "{ \"prompt\": \"What is Medicare?\",
          \"temperature\": 0.2,
          \"top_p\": 0.95,
          \"top_k\": 40
}"

Upvotes: 0

learnerer
learnerer

Reputation: 484

You input seems to be a valid (Amazon) ion although not json. In that case, you should be able to parse it as ion, and then output it as json.

single_quote_json = {'http://example.org/about': {'http://purl.org/dc/terms/title': [{'type': 'literal', 'value': "Anna's Homepage"}]}}
    
import amazon.ion.simpleion as ion
from amazon.ion.json_encoder import IonToJSONEncoder
import json

parsed_ion = ion.loads(str(single_quote_json))
json.dumps(parsed_ion, cls=IonToJSONEncoder)

Output:

'{"http://example.org/about": {"http://purl.org/dc/terms/title": [{"type": "literal", "value": "Anna\'s Homepage"}]}}'

Ref: Documentation on Amazon Ion: https://amzn.github.io/ion-docs/guides/cookbook.html#down-converting-to-json

Upvotes: 1

lam vu Nguyen
lam vu Nguyen

Reputation: 629

in my case, error is not in json.loads(data)

I had done some errors with json structure before, so there was a shortage of part in it, such as

{'http://example.org/about': {'http://purl.org/dc/terms/title':

this is also a case which raises this error

Upvotes: 0

elig
elig

Reputation: 3058

as JSON only allows enclosing strings with double quotes you can manipulate the string like this:

s = s.replace("\'", "\"")

if your JSON holds escaped single-quotes (\') then you should use the more precise following code:

import re
p = re.compile('(?<!\\\\)\'')
s = p.sub('\"', s)

This will replace all occurrences of single quote with double quote in the JSON string s and in the latter case will not replace escaped single-quotes.

You can also use js-beautify which is less strict:

$ pip install jsbeautifier
$ js-beautify file.js

Upvotes: 178

Sneil
Sneil

Reputation: 154

I have run into this problem multiple times when the JSON has been edited by hand. If someone was to delete something from the file without noticing it can throw the same error.

For instance, If your JSON last "}" is missing it will throw the same error.

So If you edit you file by hand make sure you format it like it is expected by the JSON decoder, otherwise you will run into the same problem.

Upvotes: -2

Tim Nieradzik
Tim Nieradzik

Reputation: 309

Since your string is a valid JavaScript object, you could use the Js2Py library:

import js2py

content = """x = {'http://example.org/about': {'http://purl.org/dc/terms/title': [{'type': 'literal', 'value': "Anna's Homepage"}]}}"""
content = js2py.eval_js(content)

print(content.to_dict())

Upvotes: 0

Muhammadalive
Muhammadalive

Reputation: 1076

Its easy for example

import json

my_str = '{"message_type": "fixed_price_difference", "message": "Hello hello"}'

print(type(json.loads(my_str)), json.dumps(my_str))

Output:
    <class 'dict'> "{\"message_type\": \"fixed_price_difference\", \"message\": \"Hello hello\"}"

Syntax is very important for example

Bad syntax and not correct: my_str = "{'message_type': 'fixed_price_difference', 'message': 'Hello hello"}'

Correct syntax: my_str = '{"message_type": "fixed_price_difference", "message": "Hello hello"}'

Finally: States that a string begins and ends with quotation mark.

Upvotes: 0

Pigpocket
Pigpocket

Reputation: 458

My problem was that I copy and pasted a JSON snippet and the double quotes that were used were somehow a different, unrecognized double quote character. My file was valid JSON after replacing the double quotes.

Upvotes: 0

rohetoric
rohetoric

Reputation: 354

I would highly recommend usage of json prettify tools like JSON Prettifier for the same as it helped me fix the error of a trailing comma that I had in the JSON file, which yielded the same error.

Upvotes: 1

m7s
m7s

Reputation: 113

I've had this error trying to normalize nested JSON column in Pandas. As noted by @Reihan_amn, replacing all single quotes with double quotes may affect the actual content. Therefore, when getting this error, you should replace only the ' that are where " should be in JSON syntax. You can do it with the following regular expression:

import re
import json

invalid_json = """{'http://example.org/about': {'http://purl.org/dc/terms/title': [{'type': 'literal', 'value': "Anna's Homepage"}]}}"""

valid_json = re.sub( "(?<={)\'|\'(?=})|(?<=\[)\'|\'(?=\])|\'(?=:)|(?<=: )\'|\'(?=,)|(?<=, )\'", "\"", invalid_json)

print(json.loads(valid_json))

This would suffice if the only problem was that there were single quotes (') in places were double quotes (") should be, in your original misformatted JSON document. But you would still get an error, if there are double quotes somewhere in your document that are also not part of JSON syntax. In this case I would suggest a 4 step-solution:

  1. Replace all double quotes that are part of JSON syntax with single quotes (with the regex like above but ' and " swapped).

  2. Replace all (leftover) double quotes with some special character that does not feature in your document, e.g. ``. You can do it with re.sub("\"", "``", x).

  3. Replace all single quotes that are where the double quotes in JSON should be, with double quotes, using the regex given above.

You may now load JSON document and read it into a Pandas DataFrame with pd.json_normalize(df["json_col"].apply(json.loads)).

  1. If you want, you can replace back all `` (or a special character of your choice) with ".

Upvotes: 0

Ujjwal Agrawal
Ujjwal Agrawal

Reputation: 998

Solution 1 (Very Risky)

You can simply use python eval function.

parsed_json = eval(your_json)

Solution 2 (No Risk)

You can use ast library which is included in python by default, it also safely evaluate the expression.

import ast

parsed_json = ast.literal_eval(your_json)

Upvotes: 59

Muhammad Tahir
Muhammad Tahir

Reputation: 434

The below code snippet will help to transform data into JSON. All single-quotes should be converted into double-quotes to jsonify the data.

data = {
"http://example.org/about": {
    "http://purl.org/dc/terms/title": [{
        "type": "literal",
        "value": "Anna's Homepage"
    }]
}}
parsed_data = data.replace("'", '"')
data_json = json.loads(parsed_data)

Upvotes: 2

chrisurf
chrisurf

Reputation: 31

The json syntax requires quotation marks for each "key" and "value". This makes it such a robust data format. In the following example I'm using colors and color as a key:

{"colors":[
  {
     "color":"red",
     "value":"#f00"
  },
  {
     "color":"green",
     "value":"#0f0"
  },
  {
     "color":"blue",
     "value":"#00f"
  },
  {
     "color":"cyan",
     "value":"#0ff"
  },
  {
     "color":"magenta",
     "value":"#f0f"
  },
  {
     "color":"yellow",
     "value":"#ff0"
  },
  {
     "color":"black",
     "value":"#000"
  }]}

Upvotes: 1

Elan Utta
Elan Utta

Reputation: 1134

If you are having a problem transform the dict to string and with double quote, this can help:

json.loads('["foo", {"bar":["baz", null, 1.0, 2]}]')

json.loads documentation

Upvotes: 1

Tobias Senst
Tobias Senst

Reputation: 2830

You can use the json5 package https://pypi.org/project/json5/ instead of json package. This package can deal with single quotes. The decoding function is json5.loads(data) and similar to the json package.

Upvotes: 6

information_interchange
information_interchange

Reputation: 3118

For anyone who wants a quick-fix, this simply replaces all single quotes with double quotes:

import json 

predictions = []

def get_top_k_predictions(predictions_path):
    '''load the predictions'''
    
    with open (predictions_path) as json_lines_file:
        for line in json_lines_file:
            predictions.append(json.loads(line.replace("'", "\"")))
            
    
get_top_k_predictions("/sh/sh-experiments/outputs/john/baseline_1000/test_predictions.jsonl")

Upvotes: 1

Eugen FLOCEA
Eugen FLOCEA

Reputation: 108

I had the same problem and what I did is to replace the single quotes with the double one, but what was worse is the fact I had the same error when I had a comma for the last attribute of the json object. So I used regex in python to replace it before using the json.loads() function. (Be careful about the s at the end of "loads")

import re

with open("file.json", 'r') as f:
     s = f.read()
     correct_format = re.sub(", *\n *}", "}", s)
     data_json = json.loads(correct_format)

The used regex return each comma followed by a newline and "}", replacing it just with a "}".

Upvotes: 0

Anders Solberg
Anders Solberg

Reputation: 53

As the other answers explain well the error occurs because of invalid quote characters passed to the json module.

In my case I continued to get the ValueError even after replacing ' with " in my string. What I finally realized was that some quote-like unicode symbols had found their way into my string:

 “  ”  ‛  ’  ‘  `  ´  ″  ′

To clean all of these you can just pass your string through a regular expression:

import re

raw_string = '{“key”:“value”}'

parsed_string = re.sub(r"[“|”|‛|’|‘|`|´|″|′|']", '"', my_string)

json_object = json.loads(parsed_string)

Upvotes: 1

msamogh
msamogh

Reputation: 147

Use the eval function.

It takes care of the discrepancy between single and double quotes.

Upvotes: -1

Amit Ghosh
Amit Ghosh

Reputation: 1606

x = x.replace("'", '"')
j = json.loads(x)

Although this is the correct solution, but it may lead to quite a headache if there a JSON like this -

{'status': 'success', 'data': {'equity': {'enabled': True, 'net': 66706.14510000008, 'available': {'adhoc_margin': 0, 'cash': 1277252.56, 'opening_balance': 1277252.56, 'live_balance': 66706.14510000008, 'collateral': 249823.93, 'intraday_payin': 15000}, 'utilised': {'debits': 1475370.3449, 'exposure': 607729.3129, 'm2m_realised': 0, 'm2m_unrealised': -9033, 'option_premium': 0, 'payout': 0, 'span': 858608.032, 'holding_sales': 0, 'turnover': 0, 'liquid_collateral': 0, 'stock_collateral': 249823.93}}, 'commodity': {'enabled': True, 'net': 0, 'available': {'adhoc_margin': 0, 'cash': 0, 'opening_balance': 0, 'live_balance': 0, 'collateral': 0, 'intraday_payin': 0}, 'utilised': {'debits': 0, 'exposure': 0, 'm2m_realised': 0, 'm2m_unrealised': 0, 'option_premium': 0, 'payout': 0, 'span': 0, 'holding_sales': 0, 'turnover': 0, 'liquid_collateral': 0, 'stock_collateral': 0}}}}

Noticed that "True" value? Use this to make things are double checked for Booleans. This will cover those cases - 

x = x.replace("'", '"').replace("True", '"True"').replace("False", '"False"').replace("null", '"null"')
j = json.loads(x)

Also, make sure you do not make 

x = json.loads(x)

It has to be another variable.

Upvotes: 5

rohit9786
rohit9786

Reputation: 21

with open('input.json','r') as f:
    s = f.read()
    s = s.replace('\'','\"')
    data = json.loads(s)

This worked perfectly well for me. Thanks.

Upvotes: 2

Michael Elimu
Michael Elimu

Reputation: 71

It is always ideal to use the json.dumps() method. To get rid of this error, I used the following code

json.dumps(YOUR_DICT_STRING).replace("'", '"')

Upvotes: -3

Hamed
Hamed

Reputation: 1156

I used this method and managed to get the desired output. my script

x = "{'inner-temperature': 31.73, 'outer-temperature': 28.38, 'keys-value': 0}"

x = x.replace("'", '"')
j = json.loads(x)
print(j['keys-value'])

output 

>>> 0

Upvotes: 3

greentec
greentec

Reputation: 2280

In my case, double quotes was not a problem.

Last comma gave me same error message.

{'a':{'b':c,}}
           ^

To remove this comma, I wrote some simple code.

import json

with open('a.json','r') as f:
    s = f.read()
    s = s.replace('\t','')
    s = s.replace('\n','')
    s = s.replace(',}','}')
    s = s.replace(',]',']')
    data = json.loads(s)

And this worked for me.

Upvotes: 72

Rahul Bagal
Rahul Bagal

Reputation: 230

I had similar problem . Two components communicating with each other was using a queue .

First component was not doing json.dumps before putting message to queue. So the JSON string generated by receiving component was in single quotes. This was causing error 

 Expecting property name enclosed in double quotes

Adding json.dumps started creating correctly formatted JSON & solved issue.

Upvotes: 2

alexbclay
alexbclay

Reputation: 1429

JSON strings must use double quotes. The JSON python library enforces this so you are unable to load your string. Your data needs to look like this:

{"http://example.org/about": {"http://purl.org/dc/terms/title": [{"type": "literal", "value": "Anna's Homepage"}]}}

If that's not something you can do, you could use ast.literal_eval() instead of json.loads()

Upvotes: 7

Related Questions