CODEWITHSUNDEEP
pythonjson
Frendom
Frendom

Reputation: 558

Converting list of JSON pages to one object

I want to create program which uses api to generate some data/statistics.

Api is presented on 32 pages, so I have to iterate, add page to list and after getting all of them I want to convert elements in list to one huge object and operate on him.

So, to get data from API and store in the list I have:

response = []

for i in range(1,32):
    url = "https://api.dane.gov.pl/resources/17201/data?page="+str(i)
    response.append(requests.get(url).json)

I was trying using json.loads(), json.dumps() but nothing worked. Any solution idea hot to solve this?

EDIT: when I collect my response and use:

valid = json.dumps(response)
print(valid)

I got error: Object of type 'method' is not JSON serializable

Upvotes: 1

Views: 220

Answers (1)

bergerg
bergerg

Reputation: 995

Try:

import json

def collect_responses(num_of_pages):
    response = []

    for i in range(1, num_of_pages):
        url = "https://api.dane.gov.pl/resources/17201/data?page="+str(i)
        response.append(requests.get(url).json())
    return response

valid = json.dumps({"responseList": collect_responses(32)}

Or in a more general way:

def paginate_while_200(url_template_missing_pagenum):
    list_of_json_response = []
    i = 1
    while True:
        response = requests.get(url % i)
        if response.status == 200:
            try:
                list_of_json_response.append(response.json())
                i += 1
            finally:
                return list_of_json_response
        else:
            return list_of_json_response

url = "https://api.dane.gov.pl/resources/17201/data?page=%d"
json.dumps({"responseList": paginate_while_200(url)})

Upvotes: 1

Related Questions