CODEWITHSUNDEEP
pythonaws-lambdaboto3aws-api-gatewayamazon-athena
Tien Vu
Tien Vu

Reputation: 97

how do i paginate my query in Athena using Lambda and Boto3

I am querying my data in Athena from lambda using Boto3. My result is json format. when I run my lambda function I get the whole record. Now how can I paginate this data. I only want to get fewer data per page and send that small dataset to the UI to display.

Here is my Python code:

def lambda_handler(event, context):
    athena = boto3.client('athena')
    s3 = boto3.client('s3')
    query = event['query']
    # Execution  
    query_id = athena.start_query_execution(
        QueryString=query,
        QueryExecutionContext={'Database': DATABASE},
        ResultConfiguration = {'OutputLocation': output}
    )['QueryExecutionId']

I use postman to pass my query to get data and I am aware of the SQl query LIMIT and OFFSET but want to know if there is any other better way to pass LIMIT and OFFSET parameter in my function. Please help me in this case.

Thanks.

Upvotes: 0

Views: 3256

Answers (1)

Sidah Merzouk
Sidah Merzouk

Reputation: 594

A quick google search and found this answer in the Athena docs, which seems to be promising. Example from the docs

response_iterator = paginator.paginate(
QueryExecutionId='string',
PaginationConfig={
    'MaxItems': 123,
    'PageSize': 123,
    'StartingToken': 'string'
})

I hope this helps!

Upvotes: 1

Related Questions