CODEWITHSUNDEEP
javaamazon-web-servicesamazon-s3
backslashN
backslashN

Reputation: 2875

listObjects() doesnt give full list of objects in a bucket in s3

I am trying to get the commonPrefixes of a bucket from amazon s3. I am using following code to get the list of all objects:

ObjectListing listing = s3Client.listObjects(new ListObjectsRequest().withBucketName(bucket).withPrefix("used/").withDelimiter("/"));
for (String name : listing.getCommonPrefixes()) 
{
    System.out.println(name);
    objectNames.add(name);
}
System.out.println("\n\n\nSize: " + objectNames.size());

But it just prints some of the commonPrefixes. There are more than 2000 prefixes, but it just prints 950. How can I get all the prefixes?

Upvotes: 2

Views: 3065

Answers (1)

Mark B
Mark B

Reputation: 201008

Your ObjectListing is only going to contain at most 1000 objects at a time. When you call getCommonPrefixes it is only returning the common prefixes for those 1000 objects. You need to check the ObjectListing's isTruncated() method to determine if there are more records, and then use the getNextMarker() method along with subsequent listObjects calls to get the remaining object records, and the common prefixes for those objects.

Upvotes: 6

Related Questions