CODEWITHSUNDEEP
postmanazure-data-factory
VRam
VRam

Reputation: 41

azure Data Factory Pagination

I am calling an API using Azure Data Factory and this requires pagination. The source is the REST endpoint and the output is CSV in Data Lake Storage Gen2.

I tried this API in Postman and this is what i am looking for: enter image description here

The Response Header from the API consists of the Link Key which has the rel="first", rel="current", rel="next" and rel="last".

How can i do the same in Azure ADF call where i implement pagination for rel="next"?

Upvotes: 1

Views: 503

Answers (2)

Curt Martin
Curt Martin

Reputation: 1

Setting RFC9588 to TRUE in the Pagination rules worked for me. According to Microsoft it's True by default, but I wasn't able to get more than 30 records at a time until I set it to True explicitly.

Image of RFC9588 to TRUE in ADF

Upvotes: 0

KarthikBhyresh-MT
KarthikBhyresh-MT

Reputation: 5044

You can refer to official doc for Pagination support

enter image description here

Upvotes: 1

Related Questions