CODEWITHSUNDEEP
boto3boto
khomkovova
khomkovova

Reputation: 63

How to filter AWS roles by tags in boto3?

I'm trying to get list of roles with special tags

Python version 3.9

Boto3 version 1.16.25

There is my code

iam = boto3.client('iam', region_name='us-east-1')
roles = iam.list_roles()["Roles"]
print(roles)

Result:

{
   "Path":"/",
   "RoleName":"aws-***-delivery-role",
   "RoleId":"AROA****LOIO",
   "Arn":"arn:aws:iam::448*****770:role/aws-***-delivery-role",
   "CreateDate":datetime.datetime(2017,   11,   28,   21,   13,   3, "tzinfo=tzutc())",
   "AssumeRolePolicyDocument":{
      "Version":"2012-10-17",
      "Statement":[
         {
            "Sid":"",
            "Effect":"Allow",
            "Principal":{
               "Service":"firehose.amazonaws.com"
            },
            "Action":"sts:AssumeRole",
            "Condition":{
               "StringEquals":{
                  "sts:ExternalId":"448***770"
               }
            }
         }
      ]
   },
   "MaxSessionDuration":3600
}

I don't event see a tags

I have tried this code too

iam = boto3.resource('iam', region_name='us-east-1')
roles = iam.roles.all()

But the same those roles don't have tags

Only when I run this code I could see a tag and then filtrate it, but role.load() each time do API call to AWS and I have 3k roles without role.load() it doesn't work. In result it is so long

iam = boto3.resource('iam', region_name='us-east-1')
roles = iam.roles.all()
for role in roles:
    role.load()
    print(role.tags)

Please, give me some advice about it or share your experience, how can I filtrate list of roles by tags?

I sow this documentation https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.list_roles

list_roles should return JSON with tags but this doesn't work

Upvotes: 1

Views: 1904

Answers (1)

Jack Higgins
Jack Higgins

Reputation: 63

The boto3 documentation incorrectly reports that list_roles() returns tags, see this GitHub issue for updates: https://github.com/boto/boto3/issues/2126.

To get the tags we need to call get_role() on each role and then filter from there. This can cause throttling so I also added a retry with exponential backoff.

import logging
from typing import Dict, Iterator, List

import botocore.client
from botocore.exceptions import ClientError
from retry import retry

# Type aliases to make it more clear what type of AWS resource is being returned
Policy = Dict
PolicyDocument = Dict
Role = Dict

logger = logging.getLogger(__package__)
logger.setLevel(logging.INFO)

RETRY_ARGS = {
    "exceptions": ClientError,
    "tries": 5,
    "delay": 5,
    "backoff": 2,
    "jitter": (2, 15)
}

class IAMHelper:
    def __init__(self, client: botocore.client):
        self.client = client

    @retry(**RETRY_ARGS)
    def get_role_details(self, role_name: str) -> Role:
        """
        Since we are making a client.get_role() call for each role to filter on tags, this method is prone to
        throttling. To counter this this method uses an exponential backoff.
        """

        logger.info("Attempting to get details for role %s", role_name)

        return self.client.get_role(RoleName=role_name).get("Role")

    def describe_roles(self) -> Iterator[Role]:
        """
        The IAM client has no `describe_roles()` method, only `list_roles()`. The output of `list_roles()` does not
        contain tags, so we need to call `get_role()` on each.

        The boto3 documentation incorrectly reports that `list_roles()` returns tags:
            https://github.com/boto/boto3/issues/2126
        """

        for page in self.client.get_paginator('list_roles').paginate():
            for role in page["Roles"]:
                yield self.get_role_details(role["RoleName"])

    def find_roles_by_tag(self, tag_key: str, tag_value: str) -> List[Role]:
        roles = []

        for role in self.describe_roles():
            for tag in role.get("Tags", []):
                if tag["Key"] == tag_key and tag["Value"] == tag_value:
                    logger.info("Found %s role with tags %s: %s", role.get("RoleName"), tag_key, tag_value)
                    roles.append(role)

        return roles

Upvotes: 1

Related Questions