CODEWITHSUNDEEP
pythonamazon-s3boto3
lf_celine
lf_celine

Reputation: 673

Select second to last file in S3 bucket with boto3

I try to select the second to last file of a S3 bucket. The code is okay for the last modified file.

client = boto3.client(
    's3',
    # Hard coded strings as credentials, not recommended.
    aws_access_key_id=AWS_ACCESS_KEY,
    aws_secret_access_key=AWS_SECRET_KEY
)
#get S3 file
# navigate through dir
response = client.list_objects_v2(
    Bucket=_BUCKET_NAME, Prefix=_PREFIX_TEST)
allContent = response.get('Contents', [])
def lastFile():
    # get latest file path
    latestContent = max(allContent, key=lambda x: x['LastModified'])
    objectFile = latestContent["Key"]
    # get json file
    fileObj = client.get_object(
        Bucket=_BUCKET_NAME,
        Key=objectFile
    )
    fileData = fileObj['Body'].read().decode('utf-8')
    lastData = json.loads(fileData)
    return lastData
lastData = lastFile()

But how to replicate this with the second to last modified file ?

Upvotes: 0

Views: 405

Answers (1)

Josh Clark
Josh Clark

Reputation: 1012

Change this line:

latestContent = max(allContent, key=lambda x: x['LastModified'])

to

secondLatestContent = sorted(
    allContent,
    key=lambda x: x['LastModified']
)[-2]

sorted() will put the items in ascending order. [-2] will take the second-to-last item in the list.

You can also use this code to get the nth-to-last item in the list:

def get_nth_to_last_file(n):
    # -1 * len(all_content) < n (int) <= len(all_content) to avoid an error
    nth_to_last_content = sorted(
        all_content,
        key=lambda x: x['LastModified']
    )[-1 * n]
    ...

Upvotes: 2

Related Questions