CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3
alan_blk
alan_blk

Reputation: 193

How can i automatically delete AWS S3 files using python?

I want delete some files from S3 after certain time. i need to set a time limit for each object not for the bucket. is that possible?

I am using boto3 to upload the file into S3.

region = "us-east-2"
    bucket = os.environ["S3_BUCKET_NAME"]
    credentials = {
    'aws_access_key_id': os.environ["AWS_ACCESS_KEY"],
    'aws_secret_access_key': os.environ["AWS_ACCESS_SECRET_KEY"]
        }
    client = boto3.client('s3', **credentials)
    transfer = S3Transfer(client)

    transfer.upload_file(file_name, bucket, folder+file_name,
                         extra_args={'ACL': 'public-read'})

Above is the code i used to upload the object.

Upvotes: 1

Views: 1225

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269340

You have many options here. Some ideas:

  • You can automatically delete files are a given time period by using Amazon S3 Object Lifecycle Management. See: How Do I Create a Lifecycle Policy for an S3 Bucket?
  • If you requirements are more-detailed (eg different files after different time periods), you could add a Tag to each object specifying when you'd like the object deleted, or after how many days it should be deleted. Then, you could define an Amazon CloudWatch Events rule to trigger an AWS Lambda function at regular periods (eg once a day or once an hour). You could then code the Lambda function to look at the tags on objects, determine whether they should be deleted and delete the desired objects. You will find examples of this on the Internet, often called a Stopinator.
  • If you have an Amazon EC2 instance that is running all the time for other work, then you could simply create a cron job or Scheduled Task to run a similar program (without using AWS Lambda).

Upvotes: 1

Related Questions