CODEWITHSUNDEEP
pythonamazon-s3
Екатерина Яцкив
Екатерина Яцкив

Reputation: 65

How can I extract file from the public AWS S3 bucket with Python

I have bucket URL, name and files list object key, and need to download the file.

What have I tried:

import boto3
import botocore

BUCKET_NAME = 'my-bucket' 
KEY = 'my_file' 

s3 = boto3.resource('s3')

try:
    s3.Bucket(BUCKET_NAME).download_file(KEY, 'my_local_file')
except botocore.exceptions.ClientError as e:
    if e.response['Error']['Code'] == "404":
        print("The object does not exist.")
    else:
        raise

And caught errors:

Traceback (most recent call last):
  File "D:\me\work\Mackpaw Data Engeneering test tsk\main.py", line 10, in <module>
    s3.Bucket(BUCKET_NAME).download_file(KEY, 'data.json')
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\boto3\s3\inject.py", line 244, in bucket_download_file
    return self.meta.client.download_file(
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\boto3\s3\inject.py", line 170, in download_file
    return transfer.download_file(
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\boto3\s3\transfer.py", line 307, in download_file
    future.result()
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\s3transfer\futures.py", line 106, in result
    return self._coordinator.result()
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\s3transfer\futures.py", line 265, in result
    raise self._exception
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\s3transfer\tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\s3transfer\download.py", line 340, in _submit
    response = client.head_object(
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\client.py", line 662, in _make_api_call
    http, parsed_response = self._make_request(
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\client.py", line 682, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\endpoint.py", line 132, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\endpoint.py", line 115, in create_request
    self._event_emitter.emit(event_name, request=request,
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\signers.py", line 90, in handler
    return self.sign(operation_name, request)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\signers.py", line 162, in sign
    auth.add_auth(request)
  File "C:\Users\Katherine\AppData\Local\Programs\Python\Python39\lib\site-packages\botocore\auth.py", line 373, in add_auth
    raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials

Seems that I'm trying to download the file as if the bucket is private and it needs some configs. But this bucket is public and I haven't found info how to get access to the file in that situation.

Upvotes: 0

Views: 614

Answers (2)

jrchew
jrchew

Reputation: 255

If the bucket and its objects are public, you can just use the object's public URL to retrieve the required object. The public URL is in the following format:

https://<image_name>.s3-.amazonaws.com/<file_path_in_s3>

Upvotes: 1

Vikash Kumar
Vikash Kumar

Reputation: 197

I guess you have not setup the aws credentials - secret and access key. You can follow the below link for the same:

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

Upvotes: 0

Related Questions