CODEWITHSUNDEEP
pythonjsonpython-3.xpython-requests
Matt P
Matt P

Reputation: 147

Requests returns 400 with JSON in payload, string in payload returns 200

Working with an API that returns nested JSON dataset after a POST request. It expects the payload to be:

payload = '{"metrics":[ {"name":"Jobs.2019"}, {"name", "Jobs.2018"], "constraints": ["dimensionName": "Area", "map": {"Adams, OH":["39001"],"Allen, OH":["39003"]} }, {"dimensionName": "Industry", "map": {"Crop Production":["111000"]} } ] }'

Where the {"Adams, OH":["391001"]} puts a label on the FIPS code 391001 for lookup. The issue is there are 88 counties in Ohio, and even more counties in every state of the US.

I thought I could put the counties in as counties.json

{
   "Adams, OH":[
      "39001"
   ],
   "Allen, OH":[
      "39003"
   ]
}

and set the variable in my script

with open('data.json', 'r') as json_file:
    counties = json.load(json_file)

and then call my request

metrics = '[{"name": "Jobs.2019" }, {"name": "Jobs.2018"}, {"name": "Jobs.2017"}, {"name": "Jobs.2016"}, {"name": "Jobs.2015"}, {"name": "Jobs.2020"}, {"name": "Estab.2019" }, {"name": "Estab.2018"}, {"name": "Estab.2017"}, {"name": "Estab.2016"}, {"name": "Estab.2015"}, {"name": "Estab.2014"}, {"name": "EPW.2019"}, {"name": "EPW.2018"}, {"name": "EPW.2017"}, {"name": "EPW.2016"}, {"name": "EPW.2015"}, {"name": "EPW.2014"}]'

payload = f'{{"metrics": {metrics},  "constraints": [{{"dimensionName": "Area", "map": {counties} }}, {{"dimensionName": "Industry", "map": {{"Crop Production":["111000"]}}  ]}}'

result = requests.request("POST", url, headers=headers, data=payload)

however this returns a <400> for bad request and my payload looks like:

{"metrics": [{"name": "Jobs.2019" }, {"name": "Jobs.2018"}, {"name": "Jobs.2017"}, {"name": "Jobs.2016"}, {"name": "Jobs.2015"}, {"name": "Jobs.2020"}, {"name": "Estab.2019" }, {"name": "Estab.2018"}, {"name": "Estab.2017"}, {"name": "Estab.2016"}, {"name": "Estab.2015"}, {"name": "Estab.2014"}, {"name": "EPW.2019"}, {"name": "EPW.2018"}, {"name": "EPW.2017"}, {"name": "EPW.2016"}, {"name": "EPW.2015"}, {"name": "EPW.2014"}],  "constraints": [{"dimensionName": "Area", "map": {'Adams, OH': ['39001'], 'Allen, OH': ['39003']} }, {"dimensionName": "Industry", "map": {"Crop Production":["111000"]}  ]}

However, when I call this:

metrics = '[{"name": "Jobs.2019" }, {"name": "Jobs.2018"}, {"name": "Jobs.2017"}, {"name": "Jobs.2016"}, {"name": "Jobs.2015"}, {"name": "Jobs.2020"}, {"name": "Estab.2019" }, {"name": "Estab.2018"}, {"name": "Estab.2017"}, {"name": "Estab.2016"}, {"name": "Estab.2015"}, {"name": "Estab.2014"}, {"name": "EPW.2019"}, {"name": "EPW.2018"}, {"name": "EPW.2017"}, {"name": "EPW.2016"}, {"name": "EPW.2015"}, {"name": "EPW.2014"}]'

payload = '{"metrics":' + metrics + ',"constraints": [{"dimensionName": "Area", "map": {"Adams, OH":["39001"],"Allen, OH":["39003"]} }, {"dimensionName": "Industry", "map": {"Crop Production":["111000"]} } ] }'

result = requests.request("POST", url, headers=headers, data=payload)

I receive a <200> with the payload looking like below:

{"metrics":[{"name": "Jobs.2019" }, {"name": "Jobs.2018"}, {"name": "Jobs.2017"}, {"name": "Jobs.2016"}, {"name": "Jobs.2015"}, {"name": "Jobs.2020"}, {"name": "Estab.2019" }, {"name": "Estab.2018"}, {"name": "Estab.2017"}, {"name": "Estab.2016"}, {"name": "Estab.2015"}, {"name": "Estab.2014"}, {"name": "EPW.2019"}, {"name": "EPW.2018"}, {"name": "EPW.2017"}, {"name": "EPW.2016"}, {"name": "EPW.2015"}, {"name": "EPW.2014"}],"constraints": [{"dimensionName": "Area", "map": {"Adams, OH":["39001"],"Allen, OH":["39003"]} }, {"dimensionName": "Industry", "map": {"Crop Production":["111000"]} } ] }

From what I'm seeing, everything is exactly the same, the interpreter spits out the JSON as single quotes (') instead of double (") but from what I recall the requests module receives the same payload.

I feel like I'm missing something blatanly dumb and obvious, but I've tried about every trick in my toolbelt at this point.

Upvotes: 0

Views: 70

Answers (1)

Keelan Fadden-Hopper
Keelan Fadden-Hopper

Reputation: 111

The issue is that when you read the JSON file, json.load is returning a Python dict. When you then use this as an input to an f-string, you are just getting the Python representation of that dict, rather than a valid JSON object - as you've noticed, it is presented as single quotes, which are not valid in JSON - JSON is fussy about single vs. double quotes.

Instead of using counties, you could use json.dumps(counties) instead.

However, I'd recommend that you do something different. Instead of manipulating strings to create JSON objects, do that work in Python and then convert to JSON as necessary. That's what the json library is there to help you with!

E.g. you could do the following, which is quite a bit simpler:

metrics = json.loads('[{"name": "Jobs.2019" }, {"name": "Jobs.2018"}, {"name": "Jobs.2017"}, {"name": "Jobs.2016"}, {"name": "Jobs.2015"}, {"name": "Jobs.2020"}, {"name": "Estab.2019" }, {"name": "Estab.2018"}, {"name": "Estab.2017"}, {"name": "Estab.2016"}, {"name": "Estab.2015"}, {"name": "Estab.2014"}, {"name": "EPW.2019"}, {"name": "EPW.2018"}, {"name": "EPW.2017"}, {"name": "EPW.2016"}, {"name": "EPW.2015"}, {"name": "EPW.2014"}]')

payload = {"metrics": metrics,
           "constraints": [{"dimensionName": "Area",
                            "map": counties},
                           {"dimensionName": "Industry",
                            "map": {"Crop Production": ["111000"]}}]}

result = requests.post(url, headers=headers, json=payload)

Upvotes: 1

Related Questions