CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
rayman
rayman

Reputation: 21596

How to get the last X modified files from s3 bucket

I have in our bucket lots of keys (zipped).

we want to request from s3 only the last X keys that were created.

We use Java API. However, I couldn't find a way how to get only last modified/created.

We have sorted the list ourselves but still, we don't get the last X items. that's problematic as this bucket can be huge and we don't want to retrieve all results and sort them.

public List<MyObject> getResults(int numberOfResults) {
        ObjectListing listing = s3Client.listObjects(new ListObjectsRequest()
                .withBucketName(bucketName)
                .withMaxKeys(numberOfResults));
        List<S3ObjectSummary> list = listing.getObjectSummaries();

// our internal sorting logic:

        List<MyObject> myObjects = list.stream().map(item ->
              MyObject.builder().resultKey(item.getKey()).createdDate(item.getLastModified()).build()).sorted(Comparator.comparing(MyObject::getCreatedDate))
                .collect(Collectors.toList());
        return myObjects;
    }

Any idea?

Upvotes: 2

Views: 2576

Answers (2)

fagiani
fagiani

Reputation: 2341

I've faced this same challenge and was able to use the following command to get information from S3 API:

aws s3api list-objects-v2 --max-items 3 --query "reverse(sort_by(Contents,&LastModified))" --bucket <bucketName>

You could adjust --max-items to the X you need.

I hope that becomes helpful to others in the future!

Upvotes: 0

Vaisakh PS
Vaisakh PS

Reputation: 1201

I think Aws is not offering a way to do this in server side. You can use the client side filtering. By CLI you can do this like aws s3api list-objects --bucket bucketName --query "sort_by(keys,LastModified)"

Upvotes: 0

Related Questions