Issue parsing JSON file-Python

Question

Have this section in one large JSON file

"UserDetailList": [
        {
            "UserName": "citrix-xendesktop-ec2-provisioning", 
            "GroupList": [], 
            "CreateDate": "2017-11-07T14:20:14Z", 
            "UserId": "AIDAI2YJINPRUEM3XHKXO", 
            "Path": "/", 
            "AttachedManagedPolicies": [
                {
                    "PolicyName": "AmazonEC2FullAccess", 
                    "PolicyArn": "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
                }, 
                {
                    "PolicyName": "AmazonS3FullAccess", 
                    "PolicyArn": "arn:aws:iam::aws:policy/AmazonS3FullAccess"
                }
            ], 
            "Arn": "arn:aws:iam::279052847476:user/citrix-xendesktop-ec2-provisioning"
        }, 

Need to extract AttachedManagedPolicy.Policy name for user

Desired output:

"citrix-xendesktop-ec2-provisioning","AmazonEC2FullAccess"
"citrix-xendesktop-ec2-provisioning","AmazonS3FullAccess"

Some users don't have any policy at all so need some checking mechanism to avoid errors

with open('1.json') as file:
        data = json.load(file)
        for element in data['UserDetailList']:
            s = element['UserName'], element['AttachedManagedPolicies']
            print s

And getting

(u'citrix-xendesktop-ec2-provisioning', [{u'PolicyName': u'AmazonEC2FullAccess', u'PolicyArn': u'arn:aws:iam::aws:policy/AmazonEC2FullAccess'}, {u'PolicyName': u'AmazonS3FullAccess', u'PolicyArn': u'arn:aws:iam::aws:policy/AmazonS3FullAccess'}])

When added element['AttachedManagedPolicies']['PolicyName']

got: TypeError: list indices must be integers, not str

Answer 1

You are getting error because element['AttachedManagedPolicies'] is list not dictionary you need to iterate over element['AttachedManagedPolicies'] and then access key as below:

[i['PolicyName'] for i in element['AttachedManagedPolicies']]

this will construct list of values for key PolicyName

As you said you have very large JSON structure you might have empty values or not values and for that you can proceed as below:

d = {
"UserDetailList": [
        {
            "UserName": "citrix-xendesktop-ec2-provisioning", 
            "GroupList": [], 
            "CreateDate": "2017-11-07T14:20:14Z", 
            "UserId": "AIDAI2YJINPRUEM3XHKXO", 
            "Path": "/", 
            "AttachedManagedPolicies": [
                {
                    "PolicyName": "AmazonEC2FullAccess", 
                    "PolicyArn": "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
                }, 
                {
                    "PolicyName": "AmazonS3FullAccess", 
                    "PolicyArn": "arn:aws:iam::aws:policy/AmazonS3FullAccess"
                }
            ], 
            "Arn": "arn:aws:iam::279052847476:user/citrix-xendesktop-ec2-provisioning"
        } 
    ]
}

user_list = d.get("UserDetailList", None)  # if unable to fetch key then it will return None
if user_list:
    for user_detail in user_list:
        username = user_detail.get("UserName", None)
        policies = [i.get('PolicyName') for i in user_detail.get('AttachedManagedPolicies', []) if i.get('PolicyName', None)]  # empty list constructed if no policy exist
        print(username, policies)