CODEWITHSUNDEEP
pythonamazon-web-serviceslambdaamazon-dynamodb
Haha
Haha

Reputation: 1009

Insert multiple lines of json in DynamoDB

I have the following json file 

{"columnwithoutname":"structureet","nofinesset":810001792,"nofinessej":810001784}
{"columnwithoutname":"structureet","nofinesset":670797117,"nofinessej":670010339}

I want to insert it in DynamoDB using Lambda. This is what I did: 

def lambda_handler(event, context):
    bucket=event['b']
    file_key=event['c']
    table=event['t']
    recList=[]
    s3 = boto3.client('s3')
    dynamodb = boto3.client('dynamodb')
    obj= s3.get_object(Bucket=bucket, Key=file_key)
    recList=obj['Body'].read().split('\n')
    for row in recList:
        response = dynamodb.put_item(TableName='test-abe', Item=row)

But I have this error: 

 "errorMessage": "Parameter validation failed:\nInvalid type for parameter Item

Apparently I also need to precise the type of each column so it could be accepted. Anyway to do it automatically? I want all columns to be strings. thank you

Upvotes: 1

Views: 2794

Answers (2)

alexis-donoghue
alexis-donoghue

Reputation: 3387

DynamoDB client expects Item parameter to be a dict (per https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Client.put_item)

When you do recList = obj['Body'].read().split('\n') what you get is a list of str, so passing str to a param expecting dict will obviously fail.

One more thing to consider is that DynamoDB client expects item in a very specific format, with explicitly specified attribute datatypes. If you want to read JSON and simply write it, I suggest using DynamoDB resource, something like this:

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(table_name)
table.put_item(item)

Table.put_item() accepts simple dict with no need to specify data type for every attribute, so you can simply read from file, convert it to dict and send it away (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Table.put_item).

Upvotes: 1

Horatiu Jeflea
Horatiu Jeflea

Reputation: 7404

You need to manipulate each row in order to have the 'S' format:

import json

for row in recList:
    row_dict = json.loads(row)
    ddb_row_dict = {k:{"S": v} for (k,v) in row_dict.items()}
    response = dynamodb.put_item(TableName='test-abe', Item=ddb_row_dict)

Upvotes: 1

Related Questions