CODEWITHSUNDEEP
amazon-s3replicationfailover
Artem
Artem

Reputation: 833

How do I work around inconsistency in AWS S3 replica bucket after file restore?

I had deleted file from source bucket. Source and replica are synchronized, both have delete marker. I delete the delete marker in source bucket to restore the file. But the file is deleted (has delete marker) in replica. It is by design:

If a DELETE request specifies a particular object version ID to delete, Amazon S3 deletes that object version in the source bucket, but it does not replicate the deletion in the destination bucket (in other words, it does not delete the same object version from the destination bucket). This behavior protects data from malicious deletions. https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-what-is-isnot-replicated.html

I need a consistent replica for failover. How do I prevent inconsistency in replica?

Upvotes: 3

Views: 853

Answers (2)

Paul Jowett
Paul Jowett

Reputation: 6581

I understand the problem and thought it worth adding this info even though quite a while after your question. In a disaster recovery scenario where the source bucket is not available and you want to restore from the replica, the "old" files still hanging around in the replica (that should have been deleted) are a real problem. This is different from the recovery scenario where someone has done a big delete by accident (and it would have been good if the delete hadn't happened).

Currently the bucket replication is at V2. You can enable V1 replication which does allow the replication of deletion. It seems AWS wanted to protect us from ourselves by disabling the delete replication, but sometimes it is important and correct (esp when version control is in place anyway).

Related AWS resources to enable V1:

https://docs.aws.amazon.com/AmazonS3/latest/dev/crr-add-config.html#crr-backward-compat-considerations

https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-replication.html

Hope that helps someone.

Upvotes: 0

Artem
Artem

Reputation: 833

I see the only one way to save consistency:

Never delete object versions, including delete markers.

Reupload the object (create a new version) to restore it instead removing delete marker.

Upvotes: 2

Related Questions