CODEWITHSUNDEEP
pythonjsonpython-3.xpython-requests
Hayden
Hayden

Reputation: 498

Iterate JSON in Python

I am attempting to pull data from an API that returns data as a JSON object. I have the API call which returns data that looks like this (shortened here for brevity's sake)


   "status":"OK",
   "copyright":"Copyright (c) 2017 Pro Publica Inc. All Rights Reserved.",
   "results":[
      {
         "num_results": 10,
         "offset": 0,
         "bills": [
              {
                 "bill_id": "hr2739-113",
                 "bill_type": "hr",
                 "number": "H.R.2739",
                 "bill_uri": "https://api.propublica.org/congress/v1/113/bills/hr2739.json",
                 "title": "Efficient Use of Government Spectrum Act of 2013",
                 "sponsor_title": "Rep.",
                 "sponsor_id": "M001163",
                 "sponsor_name": "Doris Matsui",
                 "sponsor_state": "CA",
                 "sponsor_party": "D",
                 "sponsor_uri": "https://api.propublica.org/congress/v1/members/M001163.json",
                 "gpo_pdf_uri": "http://www.gpo.gov/fdsys/pkg/BILLS-113hr2739ih/pdf/BILLS-113hr2739ih.pdf",
                 "congressdotgov_url": "https://www.congress.gov/bill/113th-congress/house-bill/2739",
                 "govtrack_url": "https://www.govtrack.us/congress/bills/113/hr2739",
                 "introduced_date": "2013-07-18",
                 "committees": "House Armed Services Committee",
                 "committee_codes": ["HSAS","HSIF"],
                 "subcommittee_codes": ["HSAS26","HSIF16"],
                 "primary_subject": "Science, Technology, Communications",
                 "summary_short": "Efficient Use of Government Spectrum Act of 2013 - Directs the Federal Communications Commission (FCC), within three years after enactment of the Middle Class Tax Relief and Job Creation Act of 2012, to: (1) reallocate electromagnetic spectrum between the frequencies from 1755 to 1780 megahertz (currently, such frequencies are occupied by the Department of Defense [DOD] and other federal agencies); and (2) as part of the competitive bidding auctions required by such Act, grant new initial lic...",
                 "latest_major_action_date": "2013-08-29",
                 "latest_major_action": "Referred to the Subcommittee on Intelligence, Emerging Threats & Capabilities."
              },
                           {
                 "bill_id": "hr3355-113",
                 "bill_type": "hr",
                 "number": "H.R.3355",
                 "bill_uri": "https://api.propublica.org/congress/v1/113/bills/hr3355.json",
                 "title": "Reducing Employer Burdens, Unleashing Innovation, and Labor Development Act of 2013",
                 "sponsor_title": "Rep.",
                 "sponsor_id": "G000558",
                 "sponsor_name": "Brett Guthrie",
                 "sponsor_state": "KY",
                 "sponsor_party": "R",
                 "sponsor_uri": "https://api.propublica.org/congress/v1/members/G000558.json",
                 "gpo_pdf_uri": "http://www.gpo.gov/fdsys/pkg/BILLS-113hr3355ih/pdf/BILLS-113hr3355ih.pdf",
                 "congressdotgov_url": "https://www.congress.gov/bill/113th-congress/house-bill/3355",
                 "govtrack_url": "https://www.govtrack.us/congress/bills/113/hr3355",
                 "introduced_date": "2013-10-28",
                 "committees": "House Armed Services Committee",
                 "primary_subject": "Economics and Public Finance",
                 "summary_short": "Reducing Employer Burdens, Unleashing Innovation, and Labor Development Act of 2013 - Expresses the sense of Congress that increasing the competitiveness of U.S. manufacturers will strengthen the national economy. Title I: Investing in America's Workforce - Investing in America's Workforce Act - Amends the Workforce Investment Act of 1998 to require state or local workforce investment systems to use youth activities funds allocated to a local area for programs that provide training, which may...",
                 "latest_major_action_date": "2014-01-24",
                 "latest_major_action": "Referred to the Subcommittee on Intelligence, Emerging Threats & Capabilities."
              },

Using Python, I am trying to loop through the data and return all the values like so:

import requests
import json

r = requests.get({url and credentials here}).text

resp = json.loads(r)


for bill in resp['results']['bills']:
    name = bill['results']['bills']['title']
    type = item['results']['bills']['bill_type']

print(name, type)

However, whenever I try to run this, I get 

TypeError: list indices must be integers or slices, not str

Why can't I use the list indices as str? There exists plenty of examples that use str.

Upvotes: 0

Views: 159

Answers (5)

elembie
elembie

Reputation: 640

The 'bills' in the json is a list of objects (e.g. "bills" : [ - the opening bracket is square which implies a list).

You have to use an integer to access an element of a list, so you can say;

resp['results'][0]['bills'][0]

To access the first bill, for example.

However, your code is a bit confused in your loop, the bill variable will contain the information for each bill, so you can reference the keys directly, e.g. to loop over the bills in the first result

for bill in resp['results'][0]['bills']:
    name = bill['title']
    type = bill['bill_type']

    print(name, type)

The variables 'name' and 'type' will only contain the details of each bill during the loop block.

You can nest loops to loop through all results 

for result in resp['results']:
  for bill in result['bills']:
      name = bill['title']
      type = bill['bill_type']

      print(name, type)

Upvotes: 4

Shakeel
Shakeel

Reputation: 2015

If you are receiving JSON response please find below two methods of parsing it.

Method 1: Normal way - this is something which you are trying

import requests


resp = requests.get(url="url",auth="cred").json()

for bill in resp['results'][0]['bills']:
    bill_name = bill['title']
    bill_type = bill['bill_type']

print(bill_name, bill_type)

Method 2 : Cleaner approach of accessing json response

for bill_dict in resp.get("results", []):  # returns None if "results" field is not found
    for bill in bill_dict.get('bills', []): 
        bill_name = bill.get('title')  # None is assigned to bill_name if key 'title' is not found
        bill_type = bill.get('bill_type')  # None is assigned to bill_name if key 'title' is not found error OR you can assign default value if key is not found Ex: bill.get('bill_type', 'bill type unavailable')

print(bill_name, bill_type)

There are many other ways too.

Upvotes: 0

lanartri
lanartri

Reputation: 41

also in your code, name equals:

name = bill['results']['bills']['title'] -> resp['results']['bills']['results']['bills']['title']

Upvotes: 0

abhilb
abhilb

Reputation: 5757

In json data results is a list of dicts. So first iterate over results. Then for each result iterate over bills to retrieve the name and type of bill. 

for result in resp['results']:
    for bill in result['bills']:
        name = bill['title']
        type = item['bill_type']
        print(name, type)

Upvotes: 0

Tanvir Sajed
Tanvir Sajed

Reputation: 1

bill is a json object that does not have results or bills as a string index. Try the following :

for bill in resp['results']['bills']:
    name = bill['title']
    type = item['bill_type']

    print(name, type)

Upvotes: 0

Related Questions