JSON sub for loop produces KeyError, but key exists

Question

I'm trying to add the JSON output below into a dictionary, to be saved into a SQL database.

{'Parkirisca': [
    {    
    'ID_Parkirisca': 2,
    'zasedenost': {
                   'Cas': '2016-10-08 13:17:00',
                   'Cas_timestamp': 1475925420,
                   'ID_ParkiriscaNC': 9,
                   'P_kratkotrajniki': 350
                  }
    }
]}

I am currently using the following code to add the value to a dictionary:

import scraperwiki
import json
import requests
import datetime
import time
from pprint import pprint

html = requests.get("http://opendata.si/promet/parkirisca/lpt/")

data = json.loads(html.text)

for carpark in data['Parkirisca']:

    zas = carpark['zasedenost']
    free_spaces = zas.get('P_kratkotrajniki')
    last_updated = zas.get('Cas_timestamp')

    parking_type = carpark.get('ID_Parkirisca')
    if parking_type == "Avtomatizirano":
        is_automatic = "Yes"
    else:
        is_automatic = "No"

    scraped = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')

    savetodb = {
    'scraped': scraped,
    'id': carpark.get("ID_Parkirisca"),
    'total_spaces': carpark.get("St_mest"),
    'free_spaces': free_spaces,
    'last_updated': last_updated,
    'is_automatic': is_automatic,
    'lon': carpark.get("KoordinataX_wgs"), 
    'lat': carpark.get("KoordinataY_wgs")
    }

    unique_keys = ['id']

    pprint savetodb

However when I run this, it gets stuck at for zas in carpark["zasedenost"] and outputs the following error:

Traceback (most recent call last):
File "./code/scraper", line 17, in <module>
    for zas in carpark["zasedenost"]:
KeyError: 'zasedenost'

I've been led to believe that zas is in fact now a string, rather than a dictionary, but I'm new to Python and JSON, so don't know what to search for to get a solution. I've also searched here on Stack Overflow for KeyErrror when key exist questions, but they didn't help, and I believe that this might be due to the fact that's a sub for loop.

Update: Now, when I swapped the double quotes for single quotes, I get the following error:

Traceback (most recent call last):
File "./code/scraper", line 17, in <module>
    free_spaces = zas.get('P_kratkotrajniki')
AttributeError: 'unicode' object has no attribute 'get'

Answer 1

I fixed up your code:

1. Added required imports.
2. Fixed the pprint savetodb line which isn't valid Python.
3. Didn't try to iterate over carpark['zasedenost'].

I then added another pprint statement in the for loop to see what's in carpark when the KeyError occurs. From there, the error is clear. (Not all the elements in the array in your JSON contain the 'zasedenost' key.)

Here's the code I used:

import datetime
import json
from pprint import pprint
import time

import requests

html = requests.get("http://opendata.si/promet/parkirisca/lpt/")

data = json.loads(html.text)

for carpark in data['Parkirisca']:
    pprint(carpark)
    zas = carpark['zasedenost']
    free_spaces = zas.get('P_kratkotrajniki')
    last_updated = zas.get('Cas_timestamp')

    parking_type = carpark.get('ID_Parkirisca')
    if parking_type == "Avtomatizirano":
        is_automatic = "Yes"
    else:
        is_automatic = "No"

    scraped = datetime.datetime.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')

    savetodb = {
        'scraped': scraped,
        'id': carpark.get("ID_Parkirisca"),
        'total_spaces': carpark.get("St_mest"),
        'free_spaces': free_spaces,
        'last_updated': last_updated,
        'is_automatic': is_automatic,
        'lon': carpark.get("KoordinataX_wgs"), 
        'lat': carpark.get("KoordinataY_wgs")
    }

    unique_keys = ['id']

    pprint(savetodb)

And here's the output on the iteration where the KeyError occurs:

{u'A_St_Mest': None,
 u'Cena_dan_Eur': None,
 u'Cena_mesecna_Eur': None,
 u'Cena_splosno': None,
 u'Cena_ura_Eur': None,
 u'ID_Parkirisca': 7,
 u'ID_ParkiriscaNC': 72,
 u'Ime': u'P+R Studenec',
 u'Invalidi_St_mest': 9,
 u'KoordinataX': 466947,
 u'KoordinataX_wgs': 14.567929171694901,
 u'KoordinataY': 101247,
 u'KoordinataY_wgs': 46.05457609543313,
 u'Opis': u'2,40 \u20ac /dan',
 u'St_mest': 187,
 u'Tip_parkirisca': None,
 u'U_delovnik': u'24 ur (ponedeljek - petek)',
 u'U_sobota': None,
 u'U_splosno': None,
 u'Upravljalec': u'JP LPT d.o.o.'}
Traceback (most recent call last):
  File "test.py", line 14, in <module>
    zas = carpark['zasedenost']
KeyError: 'zasedenost'

As you can see, the error is quite accurate. There's no key 'zasedenost' in the dictionary. If you look through your JSON, you'll see that's true for a number of the elements in that array.

I'd suggest a fix, but I don't know what you want to do in the case where this dictionary key is absent. Perhaps you want something like this:

zas = carpark.get('zasedenost')
if zas is not None:
    free_spaces = zas.get('P_kratkotrajniki')
    last_updated = zas.get('Cas_timestamp')
else:
    free_spaces = None
    last_updated = None