CODEWITHSUNDEEP
pythonjson
Pete
Pete

Reputation: 797

Mass creation of JSON config files

I have code that uses a JSON file as an input - each entry in the JSON file is a unique configuration for a simulation run. Below is a simplified sample of one row (expanded) of the JSON file. You can have thousands of these rows each with different unique values.

{
  "1": {
    "description": "unique configuration 1",
    "attribute to change": 1750,
    "object type 1": {
      "object name": {
        "number": 10,
        "object attribute 1": 5
      }
    },
    "object type 2": {
      "object name": {
        "number": 5,
        "object attribute 1": 50
      }
    }
  }
}

It works well. However whenever I wish to make a change to the configuration files I need to do it manually, which if you have thousands of entries, can be tedious. I wish to be able to load a default JSON structure (which is the above) and automatically create the variations required.

i have created the below code which is nearly there.

def generate_config_file(input_data, attribute, new_value):

    for key, value in input_data.items():
        if isinstance(value, dict):
            if attribute in value:
                value[attribute] = new_value
            else:
                generate_config_file(value, attribute, new_value)
        elif key == attribute:
            input_data[attribute] = new_value

file_name = input('Enter file name: ')
if len(file_name) < 1:
    file_name = 'default structure.JSON'

    id_num = 1
    out_file = open('new config file.JSON', "a")

    # so here create a new file with multiple rows 
    # where "attribute to change" is the attribute modified
    # here  between 5 and 95 in steps of 5
    for i in range(5, 100, 5):
        with open(file_name) as data_file:
            data = json.load(data_file) 

        # give the new row a unique id_num 
        data[str(id_num)] = data.pop('1')
        generate_config_file(data[str(id_num)], 'attribute to change', i)
        json.dump(data, out_file, sort_keys=True, indent=4)
        id_num += 1

    out_file.close()

I would like the output to look like below (except you would have 19 rows). I have collapsed to the top level but within each row (1,2,3....19) the structure should match the default above. The only difference between the rows being the value associated with the attribute to be changed.

{
  "1": {},
  "2": {},
  "3": {} 
}

However it produces:

{
  "1": {}
}{
  "2": {}
}{
  "3": {} 
}

I've tried various things. Such as converting the output to a string and trying to strip out the extra {} and replacing with a '. I have also tried when dumping each section of output o remove the outer {} and replace. Neither worked and I am now not sure what to try.

Any help appreciated.

Upvotes: 1

Views: 272

Answers (2)

tmms
tmms

Reputation: 116

You are talking about "rows" but you expect a dictionary structure like this, (that's a valid JSON file):

{
  "1": {},
  "2": {},
  "3": {} 
}

So I think is better to forget about "rows" and think always in terms of dictionary key value pairs, mainly because "rows" aren't part of JSON standard, check also the validator.

Using inside a loop:

json.dump(data, out_file, sort_keys=True, indent=4)

having opened the output file in incremental mode here:

out_file = open('new config file.JSON', "a")`)

translates to stacking multiple objects into the output.json text file, that creates an invalid JSON file, like the one you pointed out.

In order to avoid this you may write your dictionary structure to file all at once, to do this you can change the second part of your example code like this:

# parse old config
with open(file_name, "r") as data_file:
    data = json.load(data_file)

# set the new value of attribute_to_change for the first object in json
i = 5

# loop through top level object or what you call rowss
for key in sorted(data.keys()):

    # update the attribute with recursive function on each top level
    # object with an increasing value i
    generate_config_file(data[key], 'attribute to change', i)

    i += 5
    # if you have 19 objects inside your root object in input json 
    # the value will span from 5 to 95

# save the whole modified "data" dictionary in one shot 
out_file_name = 'new config file.JSON'
with open(out_file_name, "w") as out_file:
    json.dump(data, out_file, sort_keys=True, indent=4)

Upvotes: 0

dnit13
dnit13

Reputation: 2496

What you are doing is dumping the json data in the for loop, which will always dump a dictionary, which will always produce the data you are getting now, to get around this. I suggest you create a new dictionary ( new_data ) like this, 

new_data = {}
# so here create a new file with multiple rows 
# where "attribute to change" is the attribute modified
# here  between 5 and 95 in steps of 5
for i in range(5, 100, 5):
    with open(file_name) as data_file:
        data = json.load(data_file) 

    # give the new row a unique id_num 
    data[str(id_num)] = data.pop('1')
    generate_config_file(data[str(id_num)], 'attribute to change', i)
    new_data[str(id_num)] = data[str(id_num)]
    #json.dump(data, out_file, sort_keys=True, indent=4)
    id_num += 1
json.dump(new_data, out_file, sort_keys=True, indent=4)

and dump it afterwards

Upvotes: 1

Related Questions