CODEWITHSUNDEEP
pythonamazon-s3boto3
chowpay
chowpay

Reputation: 1687

boto3 download with file path

Can't seem to figure out how to translate what I can do with the cli to boto3 python.

I can run this fine:

aws s3 ls s3://bucket-name-format/folder1/folder2/
aws s3 cp s3://bucket-name-format/folder1/folder2/myfile.csv.gz

Trying to do this with boto3:

import boto3
s3  = boto3.client('s3', region_name='us-east-1', aws_access_key_id=KEY_ID, aws_secret_access_key=ACCESS_KEY)


bucket_name = "bucket-name-format"
bucket_dir = "/folder1/folder2/"
bucket = '{0}{1}'.format(bucket_name,bucket_dir)
filename = 'myfile.csv.gz'
s3.download_file(Filename=final_name,Bucket=bucket,Key=filename)

I get this error :

invalid bucket name "bucket-name-format/folder1/folder2/": Bucket name must match the regex "^[a-zA-Z0-9.-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).:(s3|s3-object-lambda):[a-z-0-9]:[0-9]{12}:accesspoint[/:][a-zA-Z0-9-.]{1,63}$|^arn:(aws).:s3-outposts:[a-z-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9-]{1,63}$"*

I know the error is because the bucket name "bucket-name-format/folder1/folder2/" is indeed invalid.

Question: how do I add the path? All the examples Ive seen just list the base bucket name

Upvotes: 1

Views: 14232

Answers (1)

Anon Coward
Anon Coward

Reputation: 10833

Take the following command:

aws s3 cp s3://bucket-name-format/folder1/folder2/myfile.csv.gz

That S3 URI can be broken down into

  • Bucket Name: bucket-name-format
  • Object Prefix: folder1/folder2/
  • Object Suffix: myfile.csv.gz

Really the prefix and suffix are a bit artificial, the object name is really folder1/folder2/myfile.csv.gz

This means to download the same object with the boto3 API, you want to call it with something like:

bucket_name = "bucket-name-format"
bucket_dir = "folder1/folder2/"
filename = 'myfile.csv.gz'
s3.download_file(Filename=final_name,Bucket=bucket_name,Key=bucket_dir + filename)

Note that the argument to download_file for the Bucket is just the bucket name, and the Key does not start with a forward slash.

Upvotes: 3

Related Questions