CODEWITHSUNDEEP
pythonamazon-web-servicesbotomechanicalturk
Niko
Niko

Reputation: 111

How can iterate through results in boto3 with mturk

I am a newbie at boto and I try to iterate through the results of the I can get.

In particular, I want to count all workers with a given qualification. However, the limit is 100 and I do not understand how it works with NextToken. Could anybody help me?

# next_token = 1
qualification_count = 0
while True:
    response = s3.list_workers_with_qualification_type(
        QualificationTypeId=qualification_id,
        Status='Granted',
        MaxResults=100,
        NextToken=next_token
    )
    next_token = response['NextToken']
    qualification_count += response['NumResults']

clearly the next_token is not correct, but I don't know what it should be

Upvotes: 2

Views: 1358

Answers (1)

jrb
jrb

Reputation: 578

There are a few things that might be screwing this up for you. The first, which is minor, is that the client you're using is named s3. This might just be a variable name you've chosen for MTurk, but worth making sure that you aren't trying to call this against the AWS S3 client.

The second is that you're referencing next_token (the variable) on your first call into the While loop. The problem is that it won't be initialized your first time through, thus is destined to fail. Again, that might just be a relic of the brief code snippet you've shown and not actually the issue.

But either way, the following code should work. Note that you can configure the size of page to return (up to 100, I believe). But the important part is that it's never passing in a NextToken that's uninitialized and it's setting up the MTurk client correctly. This code works on my end. Let me know if you run into any issues with it. Happy to help further. 

import boto3

region_name = 'us-east-1'
aws_access_key_id = 'YOUR_ACCESS_KEY'
aws_secret_access_key = 'YOUR_SECRET_KEY'

PAGE_SIZE = 20

endpoint_url = 'https://mturk-requester-sandbox.us-east-1.amazonaws.com'

client = boto3.client('mturk',
    endpoint_url = endpoint_url,
    region_name = region_name,
    aws_access_key_id = aws_access_key_id,
    aws_secret_access_key = aws_secret_access_key,
)

qualification_id='9W4ZQKNWM3FZ5HGM2070'

response = client.list_workers_with_qualification_type(
        QualificationTypeId=qualification_id,
        Status='Granted',
        MaxResults=PAGE_SIZE
    )
next_token = response['NextToken']

qualification_count = response['NumResults']
while (response['NumResults'] == PAGE_SIZE):
    print "Using next token of {}".format(next_token)
    response = client.list_workers_with_qualification_type(
            QualificationTypeId=qualification_id,
            Status='Granted',
            MaxResults=PAGE_SIZE,
            NextToken=next_token
    )
    next_token = response['NextToken']
    qualification_count += response['NumResults']

print "There are {} Workers in Qualification {}".format(qualification_count, qualification_id)

Upvotes: 3

Related Questions