CODEWITHSUNDEEP
pythonlistdictionary
Mark
Mark

Reputation: 19

Looping though a list trying to create an dict

I have a list of data example:

res = { 'results': [
{'consumption': 0.025, 'interval_start': '2021-06-27T00:00:00+01:00', 'interval_end': '2021-06-27T00:30:00+01:00'},
{'consumption': 0.043, 'interval_start': '2021-06-26T23:30:00+01:00', 'interval_end': '2021-06-27T00:00:00+01:00'},
{'consumption': 0.053, 'interval_start': '2021-06-26T23:00:00+01:00', 'interval_end': '2021-06-26T23:30:00+01:00'},
{'consumption': 0.056, 'interval_start': '2021-06-26T22:30:00+01:00', 'interval_end': '2021-06-26T23:00:00+01:00'},
{'consumption': 0.031, 'interval_start': '2021-06-26T22:00:00+01:00', 'interval_end': '2021-06-26T22:30:00+01:00'},
{'consumption': 0.129, 'interval_start': '2021-06-26T21:30:00+01:00', 'interval_end': '2021-06-26T22:00:00+01:00'},
{'consumption': 0.19,  'interval_start': '2021-06-26T21:00:00+01:00', 'interval_end': '2021-06-26T21:30:00+01:00'},
{'consumption': 0.164, 'interval_start': '2021-06-26T20:30:00+01:00', 'interval_end': '2021-06-26T21:00:00+01:00'},
{'consumption': 0.145, 'interval_start': '2021-06-26T20:00:00+01:00', 'interval_end': '2021-06-26T20:30:00+01:00'},
{'consumption': 0.213, 'interval_start': '2021-06-26T19:30:00+01:00', 'interval_end': '2021-06-26T20:00:00+01:00'},
{'consumption': 0.167, 'interval_start': '2021-06-26T19:00:00+01:00', 'interval_end': '2021-06-26T19:30:00+01:00'},
{'consumption': 0.333, 'interval_start': '2021-06-26T18:30:00+01:00', 'interval_end': '2021-06-26T19:00:00+01:00'},
{'consumption': 0.133, 'interval_start': '2021-06-26T18:00:00+01:00', 'interval_end': '2021-06-26T18:30:00+01:00'},
{'consumption': 0.211, 'interval_start': '2021-06-26T17:30:00+01:00', 'interval_end': '2021-06-26T18:00:00+01:00'},
{'consumption': 0.135, 'interval_start': '2021-06-26T17:00:00+01:00', 'interval_end': '2021-06-26T17:30:00+01:00'},
{'consumption': 0.158, 'interval_start': '2021-06-26T16:30:00+01:00', 'interval_end': '2021-06-26T17:00:00+01:00'},
{'consumption': 0.073, 'interval_start': '2021-06-26T16:00:00+01:00', 'interval_end': '2021-06-26T16:30:00+01:00'},
{'consumption': 0.077, 'interval_start': '2021-06-26T15:30:00+01:00', 'interval_end': '2021-06-26T16:00:00+01:00'},
{'consumption': 0.125, 'interval_start': '2021-06-26T15:00:00+01:00', 'interval_end': '2021-06-26T15:30:00+01:00'},
{'consumption': 0.201, 'interval_start': '2021-06-26T14:30:00+01:00', 'interval_end': '2021-06-26T15:00:00+01:00'},
{'consumption': 0.043, 'interval_start': '2021-06-26T14:00:00+01:00', 'interval_end': '2021-06-26T14:30:00+01:00'},
] }

What I would like to do is loop though the data above and creating a dictionary data structure, an example of what Im trying to create is:

{
  "2021": {
    "06": {
      "01": [
        {
          "interval_start": "23:00",
          "interval_end": "23:30",
          "consumption": "0.021"
        },
        {
          "interval_start": "22:30",
          "interval_end": "23:00",
          "consumption": "0.021"
        }
      ],
      "02": [
        {
          "interval_start": "23:00",
          "interval_end": "23:30",
          "consumption": "0.021"
        },
        {
          "interval_start": "22:30",
          "interval_end": "23:00",
          "consumption": "0.021"
        }
      ]
    }
  }
}

The code which I have written to do this is:

main_obj = {}
for i in res['results']:    
    
    date = i["interval_start"].split("T")[0].split("-")

    
    insert_obj = {
        "interval_start" : i['interval_start'],
        "interval_end": i["interval_end"],
        "consumption": i["consumption"]
        
    }
    
    main_obj[date[0]] = {}
    main_obj[date[0]][date[1]] = {}
    main_obj[date[0]][date[1]][date[2]] = []
    
    main_obj[date[0]][date[1]][date[2]].append(insert_obj)
    
     
print(main_obj)

Where res['results'] is the list of Dicts above. When I print this out I am getting:

{
    '2021': {
        '06': {
            '26': [{
                'interval_start': '2021-06-26T14:00:00+01:00',
                'interval_end': '2021-06-26T14:30:00+01:00',
                'consumption': 0.043
            }]
        }
    }
}

The question I am stuck on is why when I looping through each dict is this not getting added to the list main_obj[date[0]][date[1]][date[2]]? Also, as dicts have unique keys why am I only seeing an insert for the 26 and not the 27? Which is at index 0?

Any help would be appreicated as I have been scratching my head with this for a while now!

Upvotes: 1

Views: 65

Answers (2)

Alex Waygood
Alex Waygood

Reputation: 7559

As @chepner said, the issue here is that in every iteration of the loop you're overriding the existing value associated with a certain dictionary key, if there is an existing value associated with that key.

Here's a funky solution to this, using functools.partial and collections.defaultdict instead of the setdefault method of a regular dictionary.

from collections import defaultdict
from functools import partial
from pprint import pprint

results_list = [
    {'consumption': 0.025, 'interval_start': '2021-06-27T00:00:00+01:00', 'interval_end': '2021-06-27T00:30:00+01:00'},
    {'consumption': 0.043, 'interval_start': '2021-06-26T23:30:00+01:00', 'interval_end': '2021-06-27T00:00:00+01:00'},
    {'consumption': 0.053, 'interval_start': '2021-06-26T23:00:00+01:00', 'interval_end': '2021-06-26T23:30:00+01:00'},
    {'consumption': 0.056, 'interval_start': '2021-06-26T22:30:00+01:00', 'interval_end': '2021-06-26T23:00:00+01:00'},
    {'consumption': 0.031, 'interval_start': '2021-06-26T22:00:00+01:00', 'interval_end': '2021-06-26T22:30:00+01:00'},
    {'consumption': 0.129, 'interval_start': '2021-06-26T21:30:00+01:00', 'interval_end': '2021-06-26T22:00:00+01:00'},
    {'consumption': 0.19, 'interval_start': '2021-06-26T21:00:00+01:00', 'interval_end': '2021-06-26T21:30:00+01:00'},
    {'consumption': 0.164, 'interval_start': '2021-06-26T20:30:00+01:00', 'interval_end': '2021-06-26T21:00:00+01:00'},
    {'consumption': 0.145, 'interval_start': '2021-06-26T20:00:00+01:00', 'interval_end': '2021-06-26T20:30:00+01:00'},
    {'consumption': 0.213, 'interval_start': '2021-06-26T19:30:00+01:00', 'interval_end': '2021-06-26T20:00:00+01:00'},
    {'consumption': 0.167, 'interval_start': '2021-06-26T19:00:00+01:00', 'interval_end': '2021-06-26T19:30:00+01:00'},
    {'consumption': 0.333, 'interval_start': '2021-06-26T18:30:00+01:00', 'interval_end': '2021-06-26T19:00:00+01:00'},
    {'consumption': 0.133, 'interval_start': '2021-06-26T18:00:00+01:00', 'interval_end': '2021-06-26T18:30:00+01:00'},
    {'consumption': 0.211, 'interval_start': '2021-06-26T17:30:00+01:00', 'interval_end': '2021-06-26T18:00:00+01:00'},
    {'consumption': 0.135, 'interval_start': '2021-06-26T17:00:00+01:00', 'interval_end': '2021-06-26T17:30:00+01:00'},
    {'consumption': 0.158, 'interval_start': '2021-06-26T16:30:00+01:00', 'interval_end': '2021-06-26T17:00:00+01:00'},
    {'consumption': 0.073, 'interval_start': '2021-06-26T16:00:00+01:00', 'interval_end': '2021-06-26T16:30:00+01:00'},
    {'consumption': 0.077, 'interval_start': '2021-06-26T15:30:00+01:00', 'interval_end': '2021-06-26T16:00:00+01:00'},
    {'consumption': 0.125, 'interval_start': '2021-06-26T15:00:00+01:00', 'interval_end': '2021-06-26T15:30:00+01:00'},
    {'consumption': 0.201, 'interval_start': '2021-06-26T14:30:00+01:00', 'interval_end': '2021-06-26T15:00:00+01:00'},
    {'consumption': 0.043, 'interval_start': '2021-06-26T14:00:00+01:00', 'interval_end': '2021-06-26T14:30:00+01:00'},
]

main_obj = defaultdict(partial(defaultdict, partial(defaultdict, list)))

for i in results_list:    
    
    date = i["interval_start"].split("T")[0].split("-")    
    
    insert_obj = {
        "interval_start" : i['interval_start'],
        "interval_end": i["interval_end"],
        "consumption": i["consumption"]
        
    }

    main_obj[date[0]][date[1]][date[2]].append(insert_obj)

pprint(main_obj)  # Expected result (I think!)

The documentation for defaultdict is here, and you can read more about how it works here. When instantiating a defaultdict, the function passed to create the default has to take 0 arguments, so it's necessary to use functools.partial to modify the functions here so that they take fewer arguments than they would usually. The documentation for functools.partial is here, and you can read more about how it works here: How does functools partial do what it does?.

Upvotes: 0

chepner
chepner

Reputation: 531480

You are overwriting any existing dicts/lists with the unconditional assignments like main_obj[date[0]] = {}; if date[0] has already been seen, you are erasing whatever previous data there was.

Use the setdefault method instead. (I'm not sure what the PEP-8-approved line-splitting for this looks like.)

(main_obj
  .setdefault(date[0], {})
  .setdefault(date[1], {})
  .setdefault(date[2], [])
).append(insert_obj)

Upvotes: 2

Related Questions