CODEWITHSUNDEEP
javascriptcsvamazon-web-servicesamazon-dynamodb
FF5Ninja
FF5Ninja

Reputation: 525

Empty String Validation Exception - DynamoDB

I have scoured the internet on how to fix this with little success...

I am parsing a CSV and putting the data into a table using DynamoDB. Anytime there is a blank entry, I receive the error:

One or more parameter values were invalid: An AttributeValue may not contain an empty string

For example:

Header: "Name","Age","Birth date"
Entry:  Brandon,22  <------ THROWS ERROR

The problem is, I will never know if there are blanks or not in the CSV. But I still need to parse it even if there is.

I have attempted to re-assign the value of the empty string to something like "N/A" in an effort to subvert this error to no avail. Any suggestions?

EDIT: Code to add context.

var file = process.argv[2];
console.log("File: " + file);

var csv = require("fast-csv");

csv.fromPath(file, {
        headers: true,
        ignoreEmpty: true
    })
    .on("data", function(data) {
        // Uncomment to see CSV data
        // console.log(data);

        params = {
            TableName: tableName,
            Item: {
                RefID: {
                    S: data["Ref-ID"]
                },
                //lots more items
            }
        };

        //Validation loops to make sure the items are defined
        for (var key in params.Item) {
            for (var items in params.Item[key]) {
                var value = params.Item[key][items];
                if (value === undefined || value === "") {
                    value = "N/A";
            }
        }

        dynamodb.putItem(params, function(err, info) {
            if (err) {
                console.error("Unable to add item. Error JSON:", JSON.stringify(err, null, 2));
            }
            else {
                console.log("Added item:", JSON.stringify(data, null, 2));
            }
        });
        // }

    })
    .on("end", function() {
        console.log("done");
    });

Upvotes: 9

Views: 8463

Answers (6)

extractor
extractor

Reputation: 37

Adding this note as it may be helpful for others who use partiQL or use empty string in params.

The error still appears if you pass an empty param in where condition. For example , If you run below query in partiql ,you can see "ValidationException: One or more key attribute values are not valid: The AttributeValue for a key attribute cannot contain an empty string value."

SELECT  "mypartitionkey","my_sort_key","fullName" FROM "dynamodbtable"  WHERE "mypartitionkey"= 'profile' and my_sort_key in ('12345', '' )

but it will work if you pass run below

SELECT  "mypartitionkey","my_sort_key","fullName" FROM "dynamodbtable"  WHERE "mypartitionkey"= 'profile' and my_sort_key in ('12345' )

Upvotes: 0

Aaron_H
Aaron_H

Reputation: 1683

2020 May 18 Update:

DynamoDB now supports empty string values for non-key attributes: https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-dynamodb-now-supports-empty-values-for-non-key-string-and-binary-attributes-in-dynamodb-tables/

You should no longer see this Exception when writing to DynamoDB.

Upvotes: 1

Abdelrahman Salem
Abdelrahman Salem

Reputation: 9

Or you can pass nullify_invalid new Marshaler(['nullify_invalid'=>true]);

Upvotes: 0

Khalid T.
Khalid T.

Reputation: 10547

Until now, lack of support for empty strings in DynamoDB's DocumentClient is still a limitation. However, you should know that since July 2017, AWS SDK v2.7.16 added a constructor option (called convertEmptyValues) for document client to convert empty strings, sets, and binary strings to a DynamoDB NULL typed field.

If this client-side workaround works out for you, you'll need to set to true if you would like the document client to convert empty values (0-length strings, binary buffers, and sets) to be converted to NULL types when persisting to DynamoDB.

Something like this:

const AWS = require("aws-sdk");

const docClient = new AWS.DynamoDB.DocumentClient({
    convertEmptyValues: true
});

Upvotes: 13

gronaz
gronaz

Reputation: 399

These functions work like a charm, just have to execute storageSerialize on your object before storing and storageDeserialize when you get an item.

You might want to get your object in the same state as in input.

Note: the code above is in Typescript, but you can easily adapt it

private storageSerialize(data: any) {
    Object.keys(data).forEach(function (key) {
        let val = data[key];
        if (val == null) {
            data[key] = "PLACEHOLDER_NULL"
        }
        if (val == "") {
            data[key] = "PLACEHOLDER_EMPTYSTRING"
        }
    });
    return data;
}

private storageDeserialize(data: any) {
    if (data !== undefined) {
        Object.keys(data).forEach(function (key) {
            let val = data[key];
            if (val == "PLACEHOLDER_NULL") {
                data[key] = null
            }
            if (val == "PLACEHOLDER_EMPTYSTRING") {
                data[key] = ""
            }
        });
    }
    return data;
}

Upvotes: 0

Mark B
Mark B

Reputation: 200562

Nowhere in your code are you assigning the values of name, age, etc. into your params object. Are you still not showing all the code? Disregarding that for a minute and looking at your validation loop, you are just throwing away the "N/A" value. You need to modify the actual params object you are using to insert records. Change the validation loop to this:

    //Validation loops to make sure the items are defined
    for (var key in params.Item) {
        for (var items in params.Item[key]) {
            var value = params.Item[key][items];
            if (value === undefined || value === "") {
                params.Item[key][items] = "N/A";
        }
    }

There's probably an easier way to do this as you are taking the values from data and copying that to the params object, but you don't seem to be showing that code.

Upvotes: 3

Related Questions