CODEWITHSUNDEEP
pythonjsonscrapy
Alex16237
Alex16237

Reputation: 199

Python - iterate through nested json and save values

I have a nested JSON (API) webstie which i want to parse and save items to file (using Scrapy framework).

I want to access each subelement of given elements, those are in following format

0   {…}
1   {…}
2   {…}
3   {…}
4   {…}
5   {…}
6   {…}
7   {…}
8   {…}
9   {…}
10  {…}

If I expand element 0 i get following values, where {...} exapnds further

id  6738
date    "2018-06-14T09:38:51"
date_gmt    "2018-06-14T09:38:51"
guid    
     rendered   "https:example.com"
modified    "2019-03-19T20:43:50"
modified_gmt    "2019-03-19T20:43:50"

How does it look like in reality

How do I access, consecutively, each element, first 0, then 1, then 2 ... up to total of 350 and grab value of, for example 

guid   
    rendered "https//:example.com"

and save it to item.

What I have:

       results = json.loads(response.body_as_unicode())
       item = DataItem()
       for var in results:
           item['guid'] = results["guid"]
       yield item

This fails with

TypeError: list indices must be integers, not str

I know that i can access it with

item['guid'] = results[0]["guid"]

But this only gives me [0] index of the whole list and I want to iterate through all of indexes. How do I pass index number inside of the list?

Upvotes: 0

Views: 352

Answers (1)

Mojtaba Kamyabi
Mojtaba Kamyabi

Reputation: 3610

Replace results["guid"] in your for loop to var["guid"]:

for var in results:
    item['guid'] = var["guid"]
    # do whatever you want with item['guid'] here

when you can access guid like results[0]["guid"] it means that you have list of dictionaries that every dictionary contains key named guid. in your for loop you use results (that is list) instead of var (that contain every dictionary in each iteration) that throws TypeError because list indices must be integers not strings (like "guid").

UPDATE: if you want to save each var["guid"] you can save them in a dictionary like this:

guid_holder = {"guid": []}
for var in results:
    guid_golder["guid].append(var["guid"])
for guid in guid_holder["guid"]:
    print(guid)

now guid_holder holds all elements.

Upvotes: 1

Related Questions