CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3boto3
Chris Chambers
Chris Chambers

Reputation: 1365

S3Transfer download_file errors out, but client.download_file works fine

I'm using Boto in Python to download a file from my S3 bucket.

This works fine:

import boto3
s3_client = boto3.resource('s3')
s3_client.meta.client.download_file('mybucket', 'db/file.00.txt', '/work/testing/file.00.txt')

I would like to use S3Transfer to automatically use multipart, as I will be working with some pretty massive files (900 MB+).

However, when I try the following, it fails:

import boto3
from boto3.s3.transfer import S3Transfer
s3_client = boto3.resource('s3')
transfer = S3Transfer(s3_client)
transfer.download_file('mybucket', 'db/file.00.txt', '/work/testing/file.00.txt')

The error I get is as follows:

Traceback (most recent call last):
  File "/work/sparkrun/CommonBlast.py", line 126, in <module>
    transfer.download_file('mybucket', 'db/file.00.txt', '/work/testing/file.00.txt')
  File "/usr/local/lib/python2.7/dist-packages/boto3/s3/transfer.py", line 658, in download_file
    object_size = self._object_size(bucket, key, extra_args)
  File "/usr/local/lib/python2.7/dist-packages/boto3/s3/transfer.py", line 723, in _object_size
    return self._client.head_object(
AttributeError: 's3.ServiceResource' object has no attribute 'head_object'

The parameters to the download_file method are identical. I'm using the latest version of boto (1.2.3). What is going on?

Upvotes: 2

Views: 4964

Answers (1)

Jordon Phillips
Jordon Phillips

Reputation: 16003

You're passing in a resource when S3Transfer expects a client.

You'll also never need to create your own S3Transfer object anyway because its methods are added to both the client and the resource already.

import boto3
client = boto3.client('s3')
client.download_file('bucket', 'key', 'filename.txt')

resource = boto3.resource('s3')
bucket = resource.Bucket('bucket')
bucket.download_file('key', 'filename.txt')

obj = bucket.Object('key')
obj.download_file('filename.txt')

docs

Upvotes: 7

Related Questions