CODEWITHSUNDEEP
amazon-dynamodbaws-sdkaws-sdk-jsaws-sdk-nodejs
Justin Grant
Justin Grant

Reputation: 46683

How many bits of integer data can be stored in a DynamoDB attribute of type Number?

DynamoDB's Number type supports 38 digits of decimal precision. This is not big enough to store a 128-bit integer which would require 39 digits. The max value is 340,282,366,920,938,463,463,374,607,431,768,211,455 for unsigned 128-bit ints or 170,141,183,460,469,231,731,687,303,715,884,105,727 for signed 128-bit ints. These are both 39-digit numbers.

If I can't store 128 bits, then how many bits of integer data can I store in a Number?

Upvotes: 7

Views: 7390

Answers (2)

AnonBird
AnonBird

Reputation: 656

It is possible to store numbers up to 9.9999999999999999999999999999999999999E+125. As stated by the documentation:

Positive range: 1E-130 to 9.9999999999999999999999999999999999999E+125

Negative range: -9.9999999999999999999999999999999999999E+125 to -1E-130

They will effectively be stored as Number:

aws dynamodb update-item --table-name "BIG_NUMBERS" --key '{"pk": {"S": "test"}' --update-expression "SET num = :big" --expression-attribute-values '{":big": {"N": "1e125"}}' --return-values ALL_NEW

{
    "Attributes": {
        "pk": {
            "S": "test"
        },
        "num": {
            "N": "1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
        }
    }
}

However you won't be able to perform operation on it:

aws dynamodb update-item --table-name "BIG_NUMBERS" --key '{"pk": {"S": "test"}}' --update-expression "ADD num :one" --expression-attribute-values '{":one": {"N": "1"}}' --return-values ALL_NEW

An error occurred (ValidationException) when calling the UpdateItem operation: Number overflow. Attempting to store a number with magnitude larger than supported range

A similar error will happen when trying to substract.

Operations can only be performed up to 10e38:

aws dynamodb update-item --table-name "BIG_NUMBERS" --key '{"pk": {"S": "test"}}' --update-expression "SET num = :big" --expression-attribute-values '{":big": {"N": "99999999999999999999999999999999999999"}}' --return-values ALL_NEW
{
    "Attributes": {
        "pk": {
            "S": "test"
        },
        "num": {
            "N": "99999999999999999999999999999999999999"
        }
    }
}

Adding one:

aws dynamodb update-item --table-name "BIG_NUMBERS" --key '{"pk": {"S": "test"}}' --update-expression "ADD num :one" --expression-attribute-values '{":one": {"N": "1"}}' --return-values ALL_NEW
{
    "Attributes": {
        "pk": {
            "S": "test"
        },
        "num": {
            "N": "100000000000000000000000000000000000000"
        }
    }
}

Adding one more fails:

aws dynamodb update-item --table-name "BIG_NUMBERS" --key '{"pk": {"S": "test"}}' --update-expression "ADD num :one" --expression-attribute-values '{":one": {"N": "1"}}' --return-values ALL_NEW

An error occurred (ValidationException) when calling the UpdateItem operation: Number overflow. Attempting to store a number with magnitude larger than supported range

Upvotes: 1

Justin Grant
Justin Grant

Reputation: 46683

DynamoDB attribute of type Number can store 126-bit integers (or 127-bit unsigned integers, with serious caveats).

According to Amazon's documentation:

Numbers can have up to 38 digits precision. Exceeding this results in an exception.

This means (verified by testing in the AWS console) that the largest positive integer and smallest negative integers, respectively, that DynamoDB can store in a Number attribute are:

99,999,999,999,999,999,999,999,999,999,999,999,999 (aka 10^38-1) -99,999,999,999,999,999,999,999,999,999,999,999,999 (aka -10^38+1)

These numbers require 126 bits of storage, using this formula:

bits = floor (ln(number) / ln (2))
     = floor (87.498 / 0.693)
     = floor (126.259)
     = 126

So you can safely store a 126-bit signed int in a DynamoDB.

If you want to live dangerously, you can store a 127-bit unsigned int too, but there are some caveats:

  • You'd need to avoid (or at least be very careful) using such a number as a sort key, because values with a most-significant-bit of 1 will sort as negative numbers.
  • Your app will need to convert unsigned ints to signed ints when storing them or querying for them in DynamoDB, and will also need to convert them back to unsigned after reading data from DynamoDB.

If it were me, I wouldn't take these risks for one extra bit without a very, very good reason.

One logical question is whether 126 (or 127 given the caveats above) is good enough to store a UUID. The answer is: it depends. If you are in control of the UUID generation, then you can always shave a bit or two from the UUID and store it. If you shave from the 4 "version" bits (see format here) then you may not be losing any entropy at all if you are always generating UUIDs with the same version.

However, if someone else is generating those UUIDs AND is expecting lossless storage, then you may not be able to use a Number to store the UUID. But you may be able to store it if you restrict clients to a whitelist of 4-8 UUID versions. The largest version now is 5 out of a 0-15 range, and some of the older versions are discouraged for privacy reasons, so this limitation may be reasonable depending on your clients and whether they adhere to the version bits as defined in RFC 4122.

BTW, I was surprised that this bit-limit question wasn't already online... at least not in an easily-Google-able place. So contributing this Q&A pair so future searchers can find it.

Upvotes: 20

Related Questions