CODEWITHSUNDEEP
rapi
obrunt
obrunt

Reputation: 122

Paginated API - R

With a paginated oAuth2.0 API, how do I use the response headers to view the next page?

So far I have successfully created an endpoint, app, token, and response:

# here's the key and username
API_KEY = 'my key' 
API_USER = 'my username'

# set up the endpoint
api_endpoint= httr::oauth_endpoint(
  authorize =  "https://www.hydrovu.com/public-api/oauth/authorize",
  access = "https://www.hydrovu.com/public-api/oauth/token"
)

# heres the app we will use to download data to
App = httr::oauth_app("xxx", key = API_USER, secret = API_KEY)

# heres the token we will use as a password
API.token = httr::oauth2.0_token(api_endpoint, 
                                 App, 
                                 scope = "https://www.hydrovu.com/public-api/oauth/token", 
                                 client_credentials=TRUE, 
                                 cache = FALSE,
                                 config_init = user_agent("Testing Oauth with httr"))

# this is the response or the data
res <- httr::GET(url = "https://www.hydrovu.com/public-api/v1/locations/{id}/data?startTime=0", 
                 user_agent("Testing Oauth with httr"),
                 config(token = API.token))

The response headers from res include next-page, which has a long string as the value. E.g. 6haGnbeLO09jlwe6poy7erT43qp

How to I use this next-page information to actually get the next page?

I am using the HydroVu API: https://www.hydrovu.com/public-api/docs/index.html

Let me know if you need more information!

Upvotes: 0

Views: 377

Answers (1)

obrunt
obrunt

Reputation: 122

After speaking to the excellent Hadley Wickham from httr maintenance, I have found the solution.

I was originally trying to pass x-isi-next-page to x-isi-next-page through add_headers() like so:

# original GET
res <- httr::GET(url, 
                 user_agent("Testing Oauth with httr"),
                 config(token = API.token))

# write x-isi-next-page to object
nextpage = res$headers$`x-isi-next-page`

# pass to add_headers()
res <- httr::GET(url, 
                 user_agent("Testing Oauth with httr"),
                 config(token = API.token),
                 add_headers(`x-isi-next-page` = nextpage))

When in fact I should have been passing it to x-isi-start-page

resp <- httr::GET(url, 
                 user_agent("Testing Oauth with httr"),
                 config(token = API.token),
                 add_headers(`x-isi-start-page` = nextpage))

Thus, giving me the next page of results.

Now, to decode the json and create a compiled csv...

Upvotes: 1

Related Questions