Python JSON to CSV with variable fields

Question

I've a very large json file ( like 1,5gb ) and i need to transform it into csv.

The problem is that sometimes there's an extra field like:

[
    {
        "item": {
            "name": "something",
            "colors": {
                "color_of_something": "something",
                "color_of_something2": "something",
                "color_of_something3": "something"
            },
            "dimensions": {
                "dimensions1": "something",
                "dimensions2": "something",
                "dimensions3": "something"
            },
            "This_field_appears_sometimes": "something",
            "description": {
                "text": "something"
            }
        }
    }]

I've this code to transform the json file into csv file:

# -*- coding: utf-8 -*-
import json, csv
with open("items.json") as file:
    data = json.load(file)
    csv_data = csv.writer(open('items.csv','wb+'))
    csv_data.writerow(['item_name','item_color','item_dimension','item_random_field','item_description')
    for json_parsed in data:
        csv_data.writerow([
            json_parsed['item']['name'],
            json_parsed['item']['colors']['color_of_something'],
            json_parsed['item']['dimensions']['dimensions1'],
            json_parsed['item']['This_field_appears_sometimes'],
            json_parsed['item']['description']['text']
        ])

When i run the task i'm getting this error:

KeyError: 'This_field_appears_sometimes'

Need some tip or advice to fix this, meanwhile i'll try if a len checkup works on this code.

Answer 1

You can use a "safe get" like this:

json_parsed['item'].get('This_field_appears_sometimes', '')

or check with a condition if that key is inside item

if 'This_field_appears_sometimes' in json_parsed['item'].keys()

Answer 2

The reason is no key 'This_field_appears_sometimes' in some item.

you can use json_parsed['item'].get('This_field_appears_sometimes') or check the json file