CODEWITHSUNDEEP
python-3.xamazon-web-servicesaws-lambdaboto3
Vivek Sable
Vivek Sable

Reputation: 10223

TooManyRequestsException for Boto3 Client Organization

I am fetch all child account from the Master AWS Account by boto3 Organization. Code is working fine. I am able to get child account list. But if you run my AWS Lambda function again then it fail to get Child Accounts.

Getting following error:

Error while getting AWS Accounts : An error occurred (TooManyRequestsException) when calling the ListAccounts operation: AWS Organizations can't complete your request because another request is already in progress. Try again later

After 20 to 30 minutes, I can see my code work for once and again raise above exception.

I am Run this code by AWS Gateway + AWS Lambda.

Any idea?

Code:

import boto3
class Organizations(object):
    """AWS Organization"""
    def __init__(self, access_key, secret_access_key, session_token=None):
        self.client = boto3.client('organizations',
                                   aws_access_key_id=access_key,
                                   aws_secret_access_key=secret_access_key,
                                   aws_session_token=session_token
                                  )

    def get_accounts(self, next_token=None, max_results=None):
        """Get Accounts List"""
        if next_token and max_results:
            result = self.client.list_accounts(NextToken=next_token,
                                               MaxResults=max_results)
        elif next_token:
            result = self.client.list_accounts(NextToken=next_token)
        elif max_results:
            result = self.client.list_accounts(MaxResults=max_results)
        else:
            result = self.client.list_accounts()

        return result

class AWSAccounts(object):
    """ Return AWS Accounts information. """    
    def get_aws_accounts(self, access_key, secret_access_key, session_token):
        """ Return List of AWS account Details."""
        org_obj = Organizations(access_key=access_key,
                                secret_access_key=secret_access_key,
                                session_token=session_token)

        aws_accounts = []
        next_token = None
        next_result = None
        while True:
            response = org_obj.get_accounts(next_token, next_result)
            for account in response['Accounts']:
                account_details = {"name": account["Name"],
                                   "id": account["Id"],
                                   "admin_role_name": self.account_role_name
                                  }
                aws_accounts.append(account_details)

            if "NextToken" not in response:
                break
            next_token = response["NextToken"]

        return aws_accounts

Upvotes: 4

Views: 11925

Answers (2)

Prescode
Prescode

Reputation: 116

Configure your boto3 client to use the built-in standard retry mode:

import boto3
from botocore.config import Config

config = Config(
   retries = {
      'max_attempts': 10,
      'mode': 'standard'
   }
)

ec2 = boto3.client('ec2', config=config)

Per the documentation, the default mode is 'legacy' which doesn't handle TooManyRequestsException.

See boto3 documentation about retry configuration here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html

Upvotes: 10

Vivek Sable
Vivek Sable

Reputation: 10223

By Exception Handling, my code is running successfully.

Catch TooManyRequestsException exception by ClientError exception and retry to call AWS list_accounts API by boto3.

We can add time sleep of 0.1 seconds.

Code:

class AWSAccounts(object):
    """ Return AWS Accounts information. """  
    def get_accounts(self, next_token=None, max_results=None):
        """Get Accounts List"""
        # If Master AWS account contain more child accounts(150+) then
        # Too-Many-Request Exception is raised by the AWS API(boto3).
        # So to fix this issue, we are calling API again by Exception Handling.
        result = None
        while True:
            try:
                if next_token and max_results:
                    result = self.client.list_accounts(NextToken=next_token,
                                                       MaxResults=max_results)
                elif next_token:
                    result = self.client.list_accounts(NextToken=next_token)
                elif max_results:
                    result = self.client.list_accounts(MaxResults=max_results)
                else:
                    result = self.client.list_accounts()
            except botocore.exceptions.ClientError as err:
                response = err.response
                print("Failed to list accounts:", response)
                if (response and response.get("Error", {}).get("Code") ==
                        "TooManyRequestsException"):
                    print("Continue for TooManyRequestsException exception.")
                    continue

            break

        return result

Upvotes: 3

Related Questions