CODEWITHSUNDEEP
pythonamazon-s3botoboto3
neelshiv
neelshiv

Reputation: 6385

Unable to retrieve known keys in python/boto

I'm attempting to connect to a coworkers S3 bucket using python and boto. I was previously able to do this without any issues, but they recently added prefixes/directories to the bucket and I can no longer access the keys.

Any attempts to get a key with bucket.get_key(keyname) return a NoneType. If I get all keys with bucket.get_all_keys(), I get a long list of keys.

>>> s3 = boto.connect_s3()
>>> bucket = s3.get_bucket('bucket_name')
>>> keys=bucket.get_all_keys()
>>> len(keys)
1000
>>> keys[5:7]
[<Key: bucketname,/prod/keyname>, <Key: bucketname,/prod/keyname>]
>>> keys[6].get_contents_as_string()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/myusername/.pyenv/versions/2.7.8/lib/python2.7/site-packages/boto/s3/key.py", line 1782, in get_contents_as_string
    response_headers=response_headers)
  File "/home/myusername/.pyenv/versions/2.7.8/lib/python2.7/site-packages/boto/s3/key.py", line 1650, in get_contents_to_file
    response_headers=response_headers)
  File "/home/myusername/.pyenv/versions/2.7.8/lib/python2.7/site-packages/boto/s3/key.py", line 1482, in get_file
    query_args=None)
  File "/home/myusername/.pyenv/versions/2.7.8/lib/python2.7/site-packages/boto/s3/key.py", line 1514, in _get_file_internal
    override_num_retries=override_num_retries)
  File "/home/myusername/.pyenv/versions/2.7.8/lib/python2.7/site-packages/boto/s3/key.py", line 343, in open
    override_num_retries=override_num_retries)
  File "/home/myusername/.pyenv/versions/2.7.8/lib/python2.7/site-packages/boto/s3/key.py", line 303, in open_read
    self.resp.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 404 Not Found
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.

I obscured some information, but they key names and prefixes take the form of /prod/word-123-123, where there are dashes and numbers in the names.

Is there any reason why this might be happening? Do I have to treat things differently now that there are prefixes/directories?

edit: I just got a coworker to add an item that has no prefix, and I was able to get it successfully.

>>> bucket.get_key('test-bucket-item')
<Key: bucketname,test-bucket-item>
>>> key=bucket.get_key('test-bucket-item')
>>> key.get_contents_as_string()
'test contents'

Upvotes: 1

Views: 1020

Answers (1)

neelshiv
neelshiv

Reputation: 6385

As mootmoot suggested, I installed boto3.

http://boto3.readthedocs.io/en/latest/guide/migration.html

boto3 works very differently from boto, so I had to make some changes to my code. The documentation can be found here.

https://boto3.readthedocs.io/en/latest/

Upvotes: 2

Related Questions