CODEWITHSUNDEEP
pythonamazon-s3boto
daydreamer
daydreamer

Reputation: 91939

Python: Amazon S3 cannot get the bucket: says 403 Forbidden

I have a bucket for my organization in Amazon S3 which looks like mydev.orgname

I do the following in oder to get the bucket

>>> import boto
>>> from boto.s3.connection import S3Connection
>>> from boto.s3.key import Key
>>> 
>>> conn = S3Connection('xxxxxxxxxxx', 'yyyyyyyyyyyyyyyyyyyyyy')
>>> conn
S3Connection:s3.amazonaws.com

Now when I try to get bucket I see error

>>> b = conn.get_bucket('mydev.myorg')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/boto/s3/connection.py", line 389, in get_bucket
    bucket.get_all_keys(headers, maxkeys=0)
  File "/Library/Python/2.7/site-packages/boto/s3/bucket.py", line 367, in get_all_keys
    '', headers, **params)
  File "/Library/Python/2.7/site-packages/boto/s3/bucket.py", line 334, in _get_all
    response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>EEC05E43AF3E00F3</RequestId><HostId>v7HHmhJaLLQJZYkZ7sL4nqvJDS9yfrhfKQCgh4i8Tx+QsxKaub50OPiYrh3JjQbJ</HostId></Error>

But from the Java application everything seems to work.

Am I doing anything wrong here?

Upvotes: 15

Views: 28334

Answers (4)

Ajeet Verma
Ajeet Verma

Reputation: 3031

Read files from Amazon S3 bucket using Python

import boto3
import csv

# get a handle on s3
session = boto3.Session(
                aws_access_key_id='XXXXXXXXXXXXXXXXXXXXXXX',
                aws_secret_access_key='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
                region_name='XXXXXXXXXX')

s3 = session.resource('s3')

# get a handle on the bucket that holds your file
bucket = s3.Bucket('bucket name') # example: energy_market_procesing

# get a handle on the object you want (i.e. your file)
obj = bucket.Object(key='file to read') # example: market/zone1/data.csv

# get the object
response = obj.get()

# read the contents of the file
lines = response['Body'].read()

# saving the file data in a new file test.csv
with open('test.csv', 'wb') as file:
    file.write(lines)

Upvotes: 0

Pavel Repin
Pavel Repin

Reputation: 30953

Giving the user a "stronger role" is not the correct solution. This is simply a problem with boto library usage. Clearly, you don't need extra permissions when using Java S3 library.

Correct way to use boto in this case is:

b = conn.get_bucket('my-bucket', validate=False)
k = b.get_key('my/cool/object.txt') # will send HEAD request to S3
...

Basically, boto by default (which is a mistake on their part IMHO), assumes you want to interact with S3 bucket. Granted, sometimes you do want that, but then you should use credentials that have permissions for S3 bucket operations. But a more popular use case is to interact with S3 objects, and in this case you don't need any special bucket-level permissions, hence the use of validate=False kwarg.

Upvotes: 41

Wonjun Hwang
Wonjun Hwang

Reputation: 51

this answer work for me :)

I did

  • S3 bucket policy setting
  • time setting
  • bucket = conn.get_bucket(BUCKET_NAME, validate=False)

Upvotes: 4

daydreamer
daydreamer

Reputation: 91939

After giving my "User" much stronger role, this error was gone. Means User given the permission to get_bucket

Upvotes: 0

Related Questions