CODEWITHSUNDEEP
pythonsessionamazon-s3boto3was
user1315621
user1315621

Reputation: 3372

Boto3 - How to keep session alive

I have a process that is supposed to run forever and needs to updates data on a S3 bucket on AWS. I am initializing the session using boto3:

        session = boto3.session.Session()
        my_s3 = session.resource(
            "s3",
            region_name=my_region_name,
            aws_access_key_id=my_aws_access_key_id,
            aws_secret_access_key=my_aws_secret_access_key,
            aws_session_token=my_aws_session_token,
        )

Since the process is supposed to run for days, I am wondering how I can make sure that the session is kept alive and working. Do I need to re-initialize the session sometimes?

Note: not sure if it is useful, but I have actually multiple threads each using its own session.

Thanks!

Upvotes: 3

Views: 4710

Answers (2)

ddelange
ddelange

Reputation: 1577

As of botocore#2766, you can use the tcp_keepalive parameter (docs):

import botocore, boto3, environs

env = environs.Env()

S3_CLIENT = boto3.session.Session().client(  # this init style makes the client thread-safe
    "s3",
    config=botocore.client.Config(
        tcp_keepalive=env.bool("AWS_TCP_KEEPALIVE", True),
        max_pool_connections=env.int("AWS_S3_CONNECTIONS", 100),
        retries={
            "max_attempts": env.int("AWS_MAX_ATTEMPTS", 6),
            "mode": env("AWS_RETRY_MODE", "adaptive"),
        },
    ),
)

Upvotes: 1

John Rotenstein
John Rotenstein

Reputation: 269081

There is no concept of a 'session'. The Session() is simply an in-memory object that contains information about how to connect to AWS.

It does not actually involve any calls to AWS until an action is performed (eg ListBuckets). Actions are RESTful, and return results immediately. They do not keep open a connection.

A Session is not normally required. If you have stored your AWS credentials in a config file using the AWS CLI aws configure command, you can simply use:

import boto3

s3_resource = boto3.resource('s3')

The Session, however, is useful if you are using temporary credentials returned by an AssumeRole() command, rather than permanent credentials. In such a case, please note that credentials returned by AWS Security Token Service (STS) such as AssumeRole() have time limitations. This, however, is unrelated to the concept of a boto3 Session.

Upvotes: 1

Related Questions