CODEWITHSUNDEEP
javaspring-bootamazon-dynamodb
curious
curious

Reputation: 291

How to handle mixed data types in dynamodb?

We have a dynamoDb table, and one column name "createdAt" is created sometimes as S (String) data type and sometimes with N (Number) data type.

In my code if i define as String, it fails when i want to fetch data and it is number:

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@ToString
@EqualsAndHashCode
@DynamoDBTable(tableName = "SomeTable")
public class SomeTable {
    @DynamoDBAttribute
    @DynamoDBTyped(DynamoDBMapperFieldModel.DynamoDBAttributeType.S)
    private Long createdAt;
}

and if i define as Number , it fails when i want to fetch data and it is string in table:

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
@ToString
@EqualsAndHashCode
@DynamoDBTable(tableName = "SomeTable")
public class SomeTable {
    @DynamoDBAttribute
    @DynamoDBTyped(DynamoDBMapperFieldModel.DynamoDBAttributeType.N)
    private Long createdAt;
}

do any of you had the same issue before maybe? there should be a way to fix it right? and it is not option to have only one datatype :(

Upvotes: 2

Views: 1719

Answers (2)

entimaniac
entimaniac

Reputation: 163

I had the same issue. I acknowledge it is a sign of something very wrong and there were several things that lead to our situation, but we ended up with two processes that were writing the to the same table. The new process wrote longs for the dateModified field while the old process wrote a timestamp string.

In order to do get the job done, I wrote a custom converter:

import java.time.Instant;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverter;
import com.amazonaws.services.dynamodbv2.model.AttributeValue;

public class StringOrLongConverter implements DynamoDBTypeConverter<AttributeValue, Long> {

    @Override
    public AttributeValue convert(Long timeInMilli) { // convert from Java Long to AttributeValue (Number) to save in Dynamo
        try {
            return new AttributeValue().withN(String.valueOf(timeInMilli));
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    @Override
    public Long unconvert(AttributeValue av) {

        try {
            return Long.valueOf(av.getN()); // try to parse as a number
        } catch (NumberFormatException e) {
            String dateString = av.getS(); // try to parse as a timestamp
            Instant instant = Instant.parse(dateString);
            return instant.toEpochMilli();
        }

    }
}

and then I just utilized that converter in my model:

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConverted;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "my-table")
public class User {
    @DynamoDBHashKey(attributeName = "id")
    private String id;

    @DynamoDBTypeConverted(converter = StringOrLongConverter.class) // utilize the converter here
    @DynamoDBAttribute(attributeName = "dateModified")
    private Long dateModified;
}

There is probably a more elegant way to check the type of the AttributeValue of the field in question other than a try...catch but it got the job done for me.

Upvotes: 0

Borislav Stoilov
Borislav Stoilov

Reputation: 3697

Saving the same attribute with different types sounds like a bad design if you can use two attributes.

However, if you absolutely need to store it like that you can use a custom converter.

    @DynamoDBAttribute
    @DynamoDBTypeConverted(converter = CustomConverter.class)
    private Long createdAt;

and then the converter

class CustomConverter<Object> implements AttributeConverter<Object> {
... implement the methods here handling both cases String and Long

Upvotes: 2

Related Questions