CODEWITHSUNDEEP
pythonpython-2.7boto3
pcjtse
pcjtse

Reputation: 353

Does boto3 support VPC endpoint connection to S3

We have a Python client that connects to the Amazon S3 via a VPC endpoint. Our code uses boto and we are able to connect and download from S3.

After migration from boto to boto3, we noticed that the VPC endpoint connection no longer works. Below is a copy snippet that can reproduce the problem.

python -c "import boto3;
s3 = boto3.resource('s3',
       aws_access_key_id='foo',
       aws_secret_access_key='bar');
s3.Bucket('some-bucket').download_file('hello-remote.txt', 
                                       'hello-local.txt')"

got the below error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\site-packages\boto3-1.4.0-py2.7.egg\boto3\s3\inject.py",
line 163, in bucket_download_file
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "C:\Python27\lib\site-packages\boto3-1.4.0-py2.7.egg\boto3\s3\inject.py",
line 125, in download_file
    extra_args=ExtraArgs, callback=Callback)
  File "C:\Python27\lib\site-packages\boto3-1.4.0-py2.7.egg\boto3\s3\transfer.py
", line 269, in download_file
    future.result()
  File "build\bdist.win32\egg\s3transfer\futures.py", line 73, in result
  File "build\bdist.win32\egg\s3transfer\futures.py", line 233, in result
botocore.vendored.requests.exceptions.ConnectionError: ('Connection aborted.', e
rror(10060, 'A connection attempt failed because the connected party did not pro
perly respond after a period of time, or established connection failed because c
onnected host has failed to respond'))

Does anyone know if boto3 support connection to S3 via VPC endpoint and/or was able to get it to work? We are using boto3-1.4.0.

Upvotes: 25

Views: 11059

Answers (2)

Vivs
Vivs

Reputation: 485

It depends on your AWS policies and roles defined. Shortest way to make your code run is to make the S3 bucket Public [ not recommended] else add your IP in the security policies and then re-run the code. Details of it can be found here.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/authorizing-access-to-an-instance.html

Use IP whitelisting to secure your AWS Transfer for SFTP servers https://aws.amazon.com/blogs/storage/use-ip-whitelisting-to-secure-your-aws-transfer-for-sftp-servers/

Upvotes: -1

pgpb.padilla
pgpb.padilla

Reputation: 2418

This is most likely a configuration error in your VPC endpoint policies. If your policies are correct, then Boto3 never knows exactly how it's able to reach the S3 location, it really is up to the policies to allow/forbid this type of traffic.

Here's a quick walkthrough of what you can do for troubleshooting: https://aws.amazon.com/premiumsupport/knowledge-center/connect-s3-vpc-endpoint/

Other relevant docs:

Upvotes: 2

Related Questions