CODEWITHSUNDEEP
django-modelsamazon-s3boto3
Yaswanth Kumar
Yaswanth Kumar

Reputation: 469

Upload to Amazon S3 using Boto3 and return public url

Iam trying to upload files to s3 using Boto3 and make that uploaded file public and return it as a url.

class UtilResource(BaseZMPResource):
class Meta(BaseZMPResource.Meta):
    queryset = Configuration.objects.none()
    resource_name = 'util_resource'
    allowed_methods = ['get']

def post_list(self, request, **kwargs):

    fileToUpload = request.FILES
    # write code to upload to amazone s3
    # see: https://boto3.readthedocs.org/en/latest/reference/services/s3.html


    self.session = Session(aws_access_key_id=settings.AWS_KEY_ID,
                  aws_secret_access_key=settings.AWS_ACCESS_KEY,
                  region_name=settings.AWS_REGION)

    client = self.session.client('s3')
    client.upload_file('zango-static','fileToUpload')


    url = "some/test/url"
    return self.create_response(request, {
        'url': url // return's public url of uploaded file 
    })

I searched whole documentation I couldn't find any links which describes how to do this can someone explain or provide any resource where I can find the soultion?

Upvotes: 44

Views: 45342

Answers (7)

Ankit Shubham
Ankit Shubham

Reputation: 3589

Going through the existing answers and their comments, I did the following and works well for special cases of file names like having whitespaces, having special characters (ASCII), corner cases. E.g. file names of the form: "key=value.txt"

import boto3
import botocore

config = botocore.client.Config(signature_version=botocore.UNSIGNED)
object_url = boto3.client('s3', config=config).generate_presigned_url('get_object', ExpiresIn=0, Params={'Bucket': s3_bucket_name, 'Key': key_name})
print(object_url)

Upvotes: 1

Dheeraj Inampudi
Dheeraj Inampudi

Reputation: 1457

I used an f-string for the same

import boto3    
#s3_client = boto3.session.Session(profile_name='sssss').client('s3')
s3_client=boto3.client('s3')
s3_bucket_name = 'xxxxx'
s3_website_URL= f"http://{s3_bucket_name}.s3-website.{s3_client.get_bucket_location(Bucket=s3_bucket_name)['LocationConstraint']}.amazonaws.com"

Upvotes: 0

Jay Lim
Jay Lim

Reputation: 421

Somebody who wants to build up a direct URL for the public accessible object to avoid using generate_presigned_url for some reason.

Please build URL with urllib.parse.quote_plus considering whitespace and special character issue.

  • My object key: 2018-11-26 16:34:48.351890+09:00.jpg please note whitespace and ':'
  • S3 public link in aws console: https://s3.my_region.amazonaws.com/my_bucket_name/2018-11-26+16%3A34%3A48.351890%2B09%3A00.jpg

Below code was OK for me

import boto3    
s3_client = boto3.client
bucket_location = s3_client.get_bucket_location(Bucket='my_bucket_name')
url = "https://s3.{0}.amazonaws.com/{1}/{2}".format(bucket_location['LocationConstraint'], 'my_bucket_name', quote_plus('2018-11-26 16:34:48.351890+09:00.jpg')
print(url)

Upvotes: 3

wonton
wonton

Reputation: 8247

The best solution I found is still to use the generate_presigned_url, just that the Client.Config.signature_version needs to be set to botocore.UNSIGNED.

The following returns the public link without the signing stuff.

config = Config(signature_version=botocore.UNSIGNED)
config.signature_version = botocore.UNSIGNED
boto3.client('s3', config=config).generate_presigned_url('get_object', ExpiresIn=0, Params={'Bucket': bucket, 'Key': key})

The relevant discussions on the boto3 repository are:

Upvotes: 28

Sebastian Brestin
Sebastian Brestin

Reputation: 503

For Django, if you use Django storages with boto3 the code below does exactly what you want:

default_storage.url(name=f.name)

Upvotes: 0

Eron Castro
Eron Castro

Reputation: 233

I had the same issue. Assuming you know the bucket name where you want to store your data, you can then use the following:

import boto3
from boto3.s3.transfer import S3Transfer

credentials = { 
    'aws_access_key_id': aws_access_key_id,
    'aws_secret_access_key': aws_secret_access_key
}

client = boto3.client('s3', 'us-west-2', **credentials)
transfer = S3Transfer(client)

transfer.upload_file('/tmp/myfile', bucket, key,
                     extra_args={'ACL': 'public-read'})

file_url = '%s/%s/%s' % (client.meta.endpoint_url, bucket, key)

Upvotes: 22

Sam Keen
Sam Keen

Reputation: 578

I'm in the same situation. Not able to find anything in the Boto3 docs beyond generate_presigned_url which is not what I need in my case since I have public readable S3 Objects.

The best I came up with is:

bucket_location = boto3.client('s3').get_bucket_location(Bucket=s3_bucket_name)
object_url = "https://s3-{0}.amazonaws.com/{1}/{2}".format(
    bucket_location['LocationConstraint'],
    s3_bucket_name,
    key_name)

You might try posting on the boto3 github issues list for a better solution.

Upvotes: 44

Related Questions