CODEWITHSUNDEEP
pythonamazon-web-serviceselasticsearchamazon-dynamodb
user1297406
user1297406

Reputation: 1341

Storing List of Dict in a DynamoDB Table

I want to store a list of Tags of an Elasticsearch domain in a DynamoDB and i'm facing some errors.

I'm getting the list of tags using list_tags() function : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/es.html#ElasticsearchService.Client.list_tags

response = client.list_tags(
    ARN='string'
)

It returns that : 

{
    'TagList': [
        {
            'Key': 'string',
            'Value': 'string'
        },
    ]
}

Here's what they say in the doc : Response Structure

(dict) -- The result of a ListTags operation. Contains tags for all requested Elasticsearch domains. TagList (list) -- List of Tag for the requested Elasticsearch domain. (dict) -- Specifies a key value pair for a resource tag.

Now i tried to insert the list in DynamoDB using various ways but i'm always getting errors :

 ':TagList': {
                'M': response_list_tags['TagList']
            },

Invalid type for parameter ExpressionAttributeValues.:TagList.M, value: [{'Key': 'Automation', 'Value': 'None'}, {'Key': 'Owner', 'Value': 'owner'}, {'Key': 'BU', 'Value': 'DS'}, {'Key': 'Support', 'Value': 'teamA'}, {'Key': 'Note', 'Value': ''}, {'Key': 'Environment', 'Value': 'dev'}, {'Key': 'Creator', 'Value': ''}, {'Key': 'SubProject', 'Value': ''}, {'Key': 'DateTimeTag', 'Value': 'nodef'}, {'Key': 'ApplicationCode', 'Value': ''}, {'Key': 'Criticity', 'Value': '3'}, {'Key': 'Name', 'Value': 'dev'}], type: , valid types: : ParamValidationError

Tried with L instead of M and got this :

Unknown parameter in ExpressionAttributeValues.:TagList.L[11]: "Value", must be one of: S, N, B, SS, NS, BS, M, L, NULL, BOOL: ParamValidationError

Upvotes: 4

Views: 10263

Answers (2)

user1297406
user1297406

Reputation: 1341

Thank you Mike, i eneded up with a similar solution. I stored the Tag List as String like that : 

':TagList': {
   'S': str(response_list_tags['TagList'])
}

Then to convert the string to a list for a later use i did this : 

import ast
...
TagList= ast.literal_eval(db_result['Item']['TagList']['S'])

Upvotes: 1

Mike Dinescu
Mike Dinescu

Reputation: 55760

The specific error you are getting is because you are using the native DynamoDB document item JSON format which requires that any attribute value (including key-values in a map, nested in a list) to be fully qualified with a type as a key-value.

There are two ways you can do that and from your question I'm not sure if you wanted to store those key-value tag objects as a list, or you wanted to store that as an actual map in Dynamo.

Either way, I recommend you JSON encode you list and just store it in DynamoDB as a string value. There's no really good reason why you would want to go through the trouble of storing that as a map or list.

However, if you really wanted to you could do the conversion to the DynamoDB native JSON and store as a map. You will end up with something like this:

 ':TagList': {
      'M': {
         'Automation': { 'S': 'None' },
         'Owner': {'S': 'owner'},
         'BU': {'S': 'DS'},
         'Support': {'S': 'teamA'}
                ...  
      }
  }

Another possibility would be using a list of maps:

  ':TagList': {
      'L': [
         'M': {'Key': {'S': 'Automation'}, 'Value': { 'S': 'None' }},
         'M': {'Key': {'S': 'Owner'}, 'Value' : {'S': 'owner'}},
         'M': {'Key': {'S': 'BU'}, 'Value': {'S': 'DS'}},
         'M': {'Key': {'S': 'Support'}, 'Value': {'S': 'teamA'}}
                ...  
      ]
  }

But in my experience I have never gotten any real value out of storing data like this in Dynamo. Instead, storing those tags as a JSON string is both easier and less error prone. You end up with this:

  ':TagList': {
      'S': '{\'Key\': \'Automation\', \'Value\': \'None\'}, {\'Key\': \'Owner\', \'Value\': \'owner\'}, {\'Key\': \'BU\', \'Value\': \'DS\'}, {\'Key\': \'Support\', \'Value\': \'teamA\'}, ... }'
  }

And all you have to do is writhe the equivalent of:

 ':TagList': {
     'S': json.dumps(response_list_tags['TagList'])
 }

Upvotes: 7

Related Questions