CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3boto3python-s3fs
Edy Bourne
Edy Bourne

Reputation: 6187

How to use Python's S3FS to delete a file that contains dashes and brackets in the filename?

I have a file named data_[2022-10-03:2022-10-23].csv.gzip in S3, inside a bucket and folder named s3://<bucket_name>/data/cache/

I am attempting to delete this file using S3FS. When I attempt to do so with:

filename = 'data_[2022-10-03:2022-10-23].csv.gzip'
data_filepath = 's3://<bucket_name>/data/cache'
fs.rm(f"{data_filepath}/{filename}")

But the rm call throws exception:

error('bad character range 2-1 at position 54')

It seems it is thinking this is a regex.

I attempted escaping it with:

filename = filename.replace('[', '\\[').replace(']', '\\]').replace('-', '\\-')

but I get the same error:

('bad character range \\\\-1 at position 57',)

I also tried re.escape():

fs.rm(f"{data_filepath}/{re.escape(filename)}")

But I get:

error('bad character range \\\\-1 at position 57')

The same thing.

I tried the delete method:

fs.delete(f"{data_filepath}/{filename}")

Same error.

I tried accessing the underlying boto3 and deleting directly:

bucket_name = '<bucket_name>'
path = 'data/cache'
filename = 'data_[2022-10-03:2022-10-23].csv.gzip'
fs.s3.delete_object(Bucket=bucket_name, Key=f'{path}/{filename}')

This method executes with no errors, but nothing is deleted.

How can I delete this file?

Upvotes: 0

Views: 185

Answers (1)

Halod
Halod

Reputation: 606

Tested this with boto3 as below to see it deleting the file as it is:

import boto3

def s3_delete():
    fs = boto3.client('s3', aws_access_key_id="########", aws_secret_access_key="########")
    bucket_name = '<bucket-name>'
    path = 'data/cache'
    filename = 'data_[2022-10-03:2022-10-23].csv.gzip'

    fs.delete_object(Bucket=bucket_name, Key=f'{path}/{filename}')


s3_delete()

Upvotes: 0

Related Questions