CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3
Jesse McMullen-Crummey
Jesse McMullen-Crummey

Reputation: 3865

Using Boto to automate upload to AWS S3 bucket Yields Expired Token Error

Very new to AWS. I'm trying to upload a directory of files to AWS using python and Boto3

I have used terminal to set the various tokens provided from the console and then can use AWS command line to successfully transfer a file to my bucket using the command:

> aws s3 cp C://dog.jpg s3://mybucketname/dog.jpg

Now I want to automate a whole slew of files, so I run the following code in python 3.7

import boto3
from botocore.exceptions import ClientError

os.environ['AWS_ACCESS_KEY_ID'] = "provided access key id"
os.environ['AWS_SECRET_ACCESS_KEY'] = "provided secret access key"
os.environ['AWS_SESSION_TOKEN'] = "provided session token"

#create the client
s3 = boto3.client(
's3',
region_name = 'ca-central-1',
aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY'),
aws_session_token=os.environ.get('AWS_SESSION_TOKEN'),
)

def upload_file(s3_client, file_name, bucket, object_name=None):
    """Upload a file to an S3 bucket
    :param s3_client: client to upload with
    :param file_name: File to upload
    :param bucket: Bucket to upload to
    :param object_name: S3 object name. If not specified then file_name is used
    :return: True if file was uploaded, else False
    """

    # If S3 object_name was not specified, use file_name
    if object_name is None:
        object_name = file_name

    #try the upload
    try:
        response = s3_client.upload_file(file_name, bucket, object_name)
    except ClientError as e:
        logging.error(e)
        return False
    return True

upload_file(s3,'c:\\dog.jpg','mys3bucket','doggy.jpg')

I get the following error

An error occurred (ExpiredToken) when calling the PutObject operation: The provided token has expired.

I've used the exact same keys/credentials as moments before. What am I missing?

Thanks.

Upvotes: 1

Views: 9904

Answers (2)

Jaffer
Jaffer

Reputation: 2968

AWS_SESSION_TOKEN - This is only needed when you are using temporary credentials.

Can you make sure you are using temporary credentials ? Like you are assuming a role or you are doing a web identity federation.

If not, Please remove it from your code

s3 = boto3.client(
  's3',
  region_name = 'ca-central-1',
  aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
  aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY')
)

Upvotes: 1

Ashutosh gupta
Ashutosh gupta

Reputation: 447

I faced the same issues few months back. I solved it via following method.

One easy hack for this is to write the files in a temp folder. And then use os python library to execute "aws s3 cp recursive" command in your script to upload it to S3.

Upvotes: 0

Related Questions