CODEWITHSUNDEEP
pythonpython-2.7amazon-s3boto
Endophage
Endophage

Reputation: 21483

Boto doubles all get request cycles by design

To get an object out of S3 using Boto, you have to call something like (at least this is the only documented way I can find):

key = bucket.get_key(some_id)
data = key.get_contents_as_string()

However, get_key does a HEAD, then get_contents_as_string does a GET. This is painfully inefficient. I know the things I'm requesting from S3 will almost always exist (eventual consistency aside, my ratio of there to almost there is enormous; millions:1). Is there some way I can bypass the apparent boto requirement of calling get_key and just jump straight to "get me the data associated with this id"?

Upvotes: 0

Views: 141

Answers (2)

Derek Litz
Derek Litz

Reputation: 10897

The API shows the class used to instantiate a Key object directly.

key = Key(bucket=bucket, name=my_key_id)
data = key.get_contents_as_string()

In my usage of boto so far I've found the docs a useful introduction but very incomplete compared to the generated API documentation.

Upvotes: 1

garnaat
garnaat

Reputation: 45916

Sure.

If you know the key is there, you can simply create a Key object like this:

key = bucket.new_key(some_id)
data = key.get_contents_as_string()

This will allow you to skip the HEAD request to check for the key's existence. Of course, if the key isn't actually there this will produce a 404 error.

Upvotes: 1

Related Questions