CODEWITHSUNDEEP
amazon-web-servicesamazon-timestream
rabejens
rabejens

Reputation: 8162

AWS TimeStream: Records that are older than one day are rejected

I created a database table in AWS TimeStream which has a memory retention period of 7 days:

Retention

However, when I try to upload data that is 3 days old, it will be rejected. It seems that only data newer than 24 hours is accepted:

2020-10-26T13:43:33.244+01:00: Rejected a record: The record timestamp is outside the time range [2020-10-25T12:38:33.242Z, 2020-10-26T13:18:33.242Z) of the memory store.

Is this desired, or is there a configuration option I am missing?

Upvotes: 10

Views: 7302

Answers (4)

Azat
Azat

Reputation: 43

Just enable Enable magnetic storage writes. under db=>table=>Edit

Upvotes: 1

santamanno
santamanno

Reputation: 636

I'm a bit late to the party, but I had the same issue with one of my tables on AWS TimeStream. Changing memory retention policy after creation of the table was useless, it always kept the initial retention policy of 3 months.

I had to delete and recreate the table with the new memory retention policy in order to be able to insert the data accordingly.

Upvotes: 4

Maarten
Maarten

Reputation: 51

Timestream has an issue when using unix epoch timestamps instead of milliseconds. I got around this by multiplying by 1000. I read this on https://medium.com/perlego/amazon-timestream-101-3b097db680cf

Upvotes: 5

Jason Capriotti
Jason Capriotti

Reputation: 2060

Here's the relevant documentation: https://docs.aws.amazon.com/timestream/latest/developerguide/storage.html

Timestream loads data into the memory store only if the timestamp is within the timespan of its retention period. So if the retention period is 1 day, the timestamp can't be more than 1 day ago.

The way I understand how changes to the memory store retention period works... it takes effect from the point of when the change is made, so if you increase the retention period from 1 day to 7 days, you can't immediately insert data with a timestamp of 7 days ago.

I'm not sure if you had increased the memory store retention period and then tried to insert old data, but it sounds like that may have been the case?

Upvotes: 8

Related Questions