CODEWITHSUNDEEP
insertamazon-dynamodbboto3
user10404489
user10404489

Reputation: 33

how to insert a nested dict to dynamodb with boto3?

we construct some data which is in nested format, it contains 3 keys. the time, id and others where id contains another nested dict. We have no idea how to create table for this or just how to insert such data to that specific table. We have tried to create a table with hashkey called id, but when we use

dynamodb.batch_write_item(RequestItems={
    'table': [{ 'PutRequest': { 'Item':whatweget}}
    }]
})

it shows thatbotocore.exceptions.ClientError: An error occurred (Validation Exception) when calling the BatchWriteItem operation: The prov ided key element does not match the schema can anyone offer some help?

Upvotes: 0

Views: 1482

Answers (2)

danialk
danialk

Reputation: 1443

The first thing to clarify is how you are interacting with DynamoDB. When using boto3, there are two approaches to making requests:

Generally, the Service Resource is easier to use and is preferable unless you have specific needs.

The other thing to consider is that storing nested data makes querying and filtering difficult, you will likely need to handle this after retrieving the data.

In the following, we'll create a table with a single index and add additional data alongside, including nested data, without defining the structure for the other fields.

Create table

Assuming you use the CLI, create a table with:

aws dynamodb create-table \
    --table-name my-table \
    --attribute-definitions AttributeName=id,AttributeType=S \
    --key-schema AttributeName=id,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST

This creates a table with a single indexed id attribute.

Insert items

The following will insert 2 items (max 25), and will automatically handle converting Python data types to DynamoDB attribute types - one of the benefits of the Service Resource over the low-level client.

import boto3
dynamodb = boto3.resource('dynamodb')
dynamodb.batch_write_item(
  RequestItems={
    'my-table': [
      {
        'PutRequest': {
          'Item': {
            'id': 'id1',
            'newAttribute': 'some new value 1',
            'anotherAttribute': 'another new value 1',
            'someDict': {
              'a': 1,
              'b': 2
            }
          }
        },
      },
      {
        'PutRequest': {
          'Item': {
            'id': 'id2',
            'newAttribute': 'some new value 2',
            'anotherAttribute': 'another new value 2',
          }
        },
      },
    ]
  }
)

In what follows, we see automatic creation of the new attributes and auto inference of the data type, and are therefore able to query and filter. The map, not so. Consider carefully your querying strategy if you decide to store nested data.

DynamoDB table browser showing list of inserted items

DynamoDB table browser showing details of nested inserted item

Upvotes: 1

Hossam26644
Hossam26644

Reputation: 54

You can create a table using DynamoDB console at your aws dashboard, you can view detailed instructions through dynamodb documentations.

For the nested dict you have multiple options, first you can just insert the values in id as xmls or jsons, since dynamodb will accept it as a string, if you want to have more strict rules regarding table operations you can just create a field for id and other fields for the supposed to be nested inside id fields, they don’t have to be present physically inside id

You can insert data to your created table using many ways, simplest is to add them manually through the aws dynamoDB consol, or you can use boto3, but you have to have a table first of course.

Upvotes: 1

Related Questions