Type mismatch for key data expected: S actual: L Dynamodb insert error with boto3

Question

I have the following script for inserting the json files to DynamoDb table:

import os
import boto3
import json

region = '<>'
image = 'ami-<>'
ubuntu_image = 'ami-<>'
keyname = '<>'
AWS_ACCESS_KEY_ID = '<>'
AWS_SECRET_ACCESS_KEY = '<>'


def dynamo(path):
    session = boto3.Session(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
    dynamo = session.resource('dynamodb')
    table = dynamo.Table('<>')

    for filename in os.listdir(path):
        with open(path + '/' + filename) as data_file:
            data = json.loads(data_file.read())
            print data
            table.put_item(
                           Item={
                                'filename': filename,
                                'data': data,
                            }
                        )      

        print filename



print dynamo('<>')

I am getting the following error, when attempting to do so:

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: Type mismatch for key data expected: S actual: L

One of the json files is:

[{"text": "Vasovagal syncope", "type": "DX_LSTM", "end": 103, "assertion": 1, "start": 86}, {"text": "Traumatic arthritis", "type": "DX_LSTM", "end": 145, "assertion": 1, "start": 126}, {"text": "right knee", "type": "DX_LSTM", "end": 157, "assertion": 1, "start": 147}, {"text": "Hypertension", "type": "DX_LSTM", "end": 174, "assertion": 1, "start": 162}, {"text": "urinary tract infection", "type": "DX_LSTM", "end": 223, "assertion": 1, "start": 200}, {"text": "renal carcinoma", "type": "DX_LSTM", "end": 254, "assertion": 1, "start": 239}, {"text": "obstructive pulmonary disease", "type": "DX_LSTM", "end": 315, "assertion": 1, "start": 286}, {"text": "stroke", "type": "DX_LSTM", "end": 442, "assertion": 1, "start": 436}, {"text": "hypertension", "type": "DX_LSTM", "end": 456, "assertion": 1, "start": 444}, {"text": "COPD", "type": "DX_LSTM", "end": 462, "assertion": 1, "start": 458}, {"text": "renal carcinoma", "type": "DX_LSTM", "end": 487, "assertion": 1, "start": 472}, {"text": "syncope", "type": "DX_LSTM", "end": 533, "assertion": 1, "start": 526}, {"text": "fell to her knees", "type": "DX_LSTM", "end": 584, "assertion": 1, "start": 567}, {"text": "hit her head on the ground, near", "type": "DX_LSTM", "end": 625, "assertion": 1, "start": 593}, {"text": "loss of consciousness", "type": "DX_LSTM", "end": 725, "assertion": 0, "start": 704}, {"text": "falls", "type": "DX_LSTM", "end": 804, "assertion": 1, "start": 799}, {"text": "hip fracture", "type": "DX_LSTM", "end": 845, "assertion": 1, "start": 833}, {"text": "bruising around the left eye", "type": "DX_LSTM", "end": 967, "assertion": 1, "start": 939}, {"text": "decreased mobility of", "type": "DX_LSTM", "end": 1085, "assertion": 1, "start": 1064}, {"text": "left", "type": "DX_LSTM", "end": 1094, "assertion": 1, "start": 1090}, {"text": "syncope", "type": "DX_LSTM", "end": 1175, "assertion": 0, "start": 1168}, {"text": "stroke", "type": "DX_LSTM", "end": 1195, "assertion": 0, "start": 1189}, {"text": "fractures", "type": "DX_LSTM", "end": 1348, "assertion": 0, "start": 1339}, {"text": "left humeral head", "type": "DX_LSTM", "end": 1405, "assertion": 0, "start": 1388}, {"text": "neck fracture", "type": "DX_LSTM", "end": 1423, "assertion": 0, "start": 1410}, {"text": "baseline anterior dislocation", "type": "DX_LSTM", "end": 1458, "assertion": 1, "start": 1429}, {"text": "changes", "type": "DX_LSTM", "end": 1500, "assertion": 0, "start": 1493}, {"text": "left periorbital soft tissue swelling", "type": "DX_LSTM", "end": 1539, "assertion": 0, "start": 1502}, {"text": "soft tissue", "type": "DX_BLME", "end": 1530, "assertion": 0, "start": 1519}, {"text": "facial bone fracture", "type": "DX_LSTM", "end": 1600, "assertion": 0, "start": 1580}, {"text": "ventricular function", "type": "DX_LSTM", "end": 1656, "assertion": 1, "start": 1636}, {"start": 1774, "end": 1782, "text": "syncopal", "type": "DX_LSTM", "assertion": 0}, {"text": "orthostatic", "type": "DX_LSTM", "end": 1865, "assertion": 1, "start": 1854}, {"text": "walk", "type": "DX_LSTM", "end": 2021, "assertion": 1, "start": 2017}, {"text": "traumatic injury of her knee", "type": "DX_LSTM", "end": 2072, "assertion": 1, "start": 2044}, {"text": "injury", "type": "DX_Exact", "end": 2060, "assertion": 1, "start": 2054}, {"text": "pain", "type": "DX_LSTM", "end": 2098, "assertion": 1, "start": 2094}, {"text": "swelling", "type": "DX_LSTM", "end": 2111, "assertion": 1, "start": 2103}, {"text": "fractures", "type": "DX_LSTM", "end": 2154, "assertion": 0, "start": 2145}, {"text": "frail", "type": "DX_LSTM", "end": 2175, "assertion": 0, "start": 2170}]

Answer 1

Worth mentioning that in my case I had a duplicate attribute fruitID

When I was building up the object to be inserted into DynamoDB I create a template with fruitID

const template = {
  fruitID: ulib()
}

// Where fruitData is an incoming payload
const newFruit = Object.assign(template, fruitData)

That incoming payload was also containing an unexpected fruitID prop.

That incoming payload fruitData.fruitID prop was actually an L instead of an S, hence the issue above.

Even though my code here is JS my pipelines use boto3.

Answer 2

Firstly, it looks like you are trying to persist the List object into String data type of DynamoDB.

Second, the String attribute data - I think it is defined as Key Attribute. The key attribute can't be List data type. So, you can't change the key attribute to List.

I am not sure what you would like to achieve. However, I guess you would like to store the entire JSON present in the file on one of the attributes.

The above code should work if you try to store the data in some non-key attribute.

The output will be a List attribute with all the objects (i.e. individual occurrences) stored as Map.

See example below:-