CODEWITHSUNDEEP
amazon-web-servicesamazon-s3versioning
user3606089
user3606089

Reputation: 313

Is it possible to limit the number of versions stashed by Amazon S3's versioning?

I'm currently working on migrating the file backup system for my office to Amazon S3. The basic backup is working like a charm but I'm looking to make it a little more robust. Specifically I am looking to add version control for the files in the bucket (using Amazon "Versioning") but there is no mention (that I can find) of a way to limit the number of old versions stashed per file (ie: "File x" has a maximum of 5 previous versions at any given time).

Is it possible to place limits on the number of versions stored per file? Or am I stuck with an unlimited propagation of file versions over time if I go this route?

I've been digging through the AWS forums (as well as anywhere else I look) and have yet to find anything. Any info would be greatly appreciated. Thanks!

Upvotes: 31

Views: 23545

Answers (4)

Shivam Anand
Shivam Anand

Reputation: 1611

Yes, it is practically possible.

We can create a lifecycle policy, which will delete non-concurrent versions (previous versions) 1 day after it has become non-concurrent version.
1 day is the hard limit, you cannot change it to 0 days.

Run the below commands to keep the latest 3 non-concurrent versions.

cat << EOF > lifecycle_policy.json
{
   "Rules": [
       {
           "ID": "Keep only the latest 4 versions of a File",
           "Filter": {},
           "Status": "Enabled",
           "NoncurrentVersionExpiration": {
               "NoncurrentDays": 1,
               "NewerNoncurrentVersions": 3
           }
       }
   ]
}
EOF

BUCKET_NAME=<bucket-name>

aws s3api put-bucket-lifecycle-configuration \
    --bucket $BUCKET_NAME \
    --lifecycle-configuration file://lifecycle_policy.json

This policy will keep 3 non-concurrent versions + 1 active version = 4 versions in Total.

The bucket policy will take some time to take effect if you have a large bucket.
Changes in the s3 Storage Lens might take time to reflect

Upvotes: 3

Antonio Aga Rossi
Antonio Aga Rossi

Reputation: 321

YES is's possible @paul-littlefield @user3606089

creating a lifecycle rule, you can specify when non current version of the objects will expire and "Number of newer versions to retain - Optional"

If you want to keep a certain number of versions, despite how old they are, set the expiration to 1 day and the number of versions you want to keep in the second field.

settings

behaviour

Upvotes: 18

Bernie Habermeier
Bernie Habermeier

Reputation: 2598

While I don't believe Amazon added the option to limit the versions to a specific count of revisions (say "5" previous revisions), you can certainly use S3 lifecycle management, which does support rules based on time. You can use NoncurrentVersionTransition to transition the old version to a different storage class, and you can use NoncurrentVersionExpiration to finally delete an old revision.

So if you backup your data at a predefined time interval (say once a week), then if you set your NoncurrentVersionExpiration to 6 weeks, you'll only keep the last 5 (maybe 6?) versions.

Reference: AWS S3 Lifecycle

Upvotes: 44

Julio Faerman
Julio Faerman

Reputation: 13501

Currently you would need to list and erase old versions manually, as there is no user defined version limit.

Upvotes: 5

Related Questions