CODEWITHSUNDEEP
javaamazon-web-servicesamazon-s3
user2403149
user2403149

Reputation:

AWS S3 - List only root level objects

I have an S3 bucket with the following folder structure:

<bucket-name>
    folder1/
        ....
    folder2/
        ....
    foldern/
        ....

Each of these folders has files in it, I only want the top level folders listed folder1, folder2 etc.

I have found a lot of solutions suggesting I use "/" as a delimiter and an empty prefix, which is exactly what I'm trying to do in the Java snippet below:

ListObjectsRequest listObjects = new ListObjectsRequest()
            .withDelimiter("/")
            .withPrefix("")
            .withBucketName(s3BucketName);
    ObjectListing objects = s3.listObjects(listObjects);
    for (S3ObjectSummary summ : objects.getObjectSummaries()) {
        System.out.println(summ.getKey());
    }

The for loop still prints no keys. Is there something I am doing wrong, or is this not done that way at all?

Upvotes: 5

Views: 4865

Answers (3)

Jaco Theron
Jaco Theron

Reputation: 41

If you need to list the top level directories:

public List<String> listTopLevelDirectories(AmazonS3 amazonS3, String bucketName) {

    ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
            .withBucketName(bucketName)
            .withDelimiter("/");
    return amazonS3.listObjects(listObjectsRequest).getCommonPrefixes();

}

Upvotes: 3

Youness
Youness

Reputation: 2070

I managed to list the top level folders with the following (in Java):

ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                                       .withBucketName(bucketName)
                                       .withDelimiter("/");
ObjectListing objectListing = client.listObjects(listObjectsRequest);
List<String> commonPrefixes = objectListing.getCommonPrefixes();

You only need to set the delimiter and read from common prefixes.

The SDK version I am currently using is 1.10.14

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.10.14</version>
    </dependency>

Upvotes: 6

Gina Marano
Gina Marano

Reputation: 1803

Here is a c# version. Should be very easy to port:

    public List<string> GetBucketRootItems(string bucketName, string regionName)
    {
        List<string> result = new List<string>();
        try
        {
            using (var client = new AmazonS3Client(RegionEndpoint.GetBySystemName(regionName)))
            {
                ListObjectsV2Request request = new ListObjectsV2Request
                {
                    BucketName = bucketName,
                    MaxKeys = 10000,
                    Delimiter = "/",
                    Prefix = ""
                };

                ListObjectsV2Response response;
                do
                {
                    response = client.ListObjectsV2(request);
                    result.AddRange(response.CommonPrefixes);
                    request.ContinuationToken = response.NextContinuationToken;
                }
                while (response.IsTruncated == true);
            }
        }
        catch (AmazonS3Exception amazonS3Exception)
        {
            if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
                || amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
            {
                log.Error($"Check the provided AWS Credentials.");
            }
            else
            {
                log.Error($"Error occurred. Message:'{amazonS3Exception.Message}' when listing objects", amazonS3Exception);
            }
        }
        return result;
    }

This appears to work. I got just under 10000 entries

Upvotes: 2

Related Questions