CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-cli
mark86v1
mark86v1

Reputation: 312

How to move s3 objects from one bucket to another based on time of creation using AWS CLI?

I have a version enabled bucket "A" with many objects. I need to move all the objects from bucket "A" to another version enabled bucket "B". The condition is "I need to move the objects based on time of creation. The older objects in bucket "A" should be moved first"

How to do this using aws-cli?

Upvotes: 0

Views: 1009

Answers (1)

Floyd
Floyd

Reputation: 367

Here it is the single command:

aws s3 ls s3://<from-bucket> --recursive | awk '{d=substr($0,1,10)}(d < "2019-10-25")' | cut -c 32- | awk '{print "--include "$0}' ORS=' ' | xargs aws s3 mv s3://<from-bucket> s3://<to-bucket> --recursive --exclude '*'
  1. first we read all the files from the source bucket and the first column in the output would be the latest change date
  2. then using awk we subtract the date and compare with a target date you specify by your own
  3. using cut subtract file names
  4. using awk prepare include arguments for aws s3 mv command
  5. using xargs pass the arguments to aws s3 mv

Upvotes: 3

Related Questions