CODEWITHSUNDEEP
pythonpaginationpython-requests
user291699
user291699

Reputation: 45

Python client to a paginated HTTP API

I'm using requests to query a paginated API.

This answer already seems pretty un-pythonic to me but my case is even worse because the API doesn't have a last page parameter, it only has a last URL parameter, so I have to do

url = 'http://my_url'
result = request(url)
data = result['datum']
while not result['last_url'] == url:
  url = result['next_page']
  result = request(url)
  data += result['datum']

This doesn't seem pythonic, is there a more elegant way?

Upvotes: 2

Views: 1348

Answers (1)

Open AI - Opting Out
Open AI - Opting Out

Reputation: 24133

I would write a generator function which yields pages:

def pages(url):
    page = {'last_url': None,
            'next_page': url}
    while page['last_url'] != url:
        url = page['next_page']
        page = request(url)
        yield page

Then you can use it as so:

data = ''.join([page['datum'] for page in pages(url)])

Upvotes: 2

Related Questions