CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-ec2
i.am.jabi
i.am.jabi

Reputation: 678

S3 doesn't return the subfolder keys. It's just returning all the files inside the subfolder

I'm using Java AmazonS3Client to connect to a bucket in S3. My bucket structure is as below.

Bucket --> folder_1 --> sub_folder_1 --> file1,file2,file3

When I queried the listObjects from s3 client, I'm getting all the folders and files as objects, which is what I'm expecting.

When I have deployed my code in EC2, the same code is returning only files as objects. It doesn't return folder objects.

Only difference I'm doing is, in local I'm using ACCESS_KEY & SECRET_KEY and in EC2, I have created IAM role. Below are the actions allowed as part of my IAM role.

"Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads"
            ],
"Resource": [
                "arn:aws:s3:::my-bucket-name/*",
                "arn:aws:s3:::my-bucket-name"
            ]

In local listObjects returns the below object keys.

folder_1
folder_1/sub_folder_1
folder_1/sub_folder_1/file1
folder_1/sub_folder_1/file2
folder_1/sub_folder_1/file3

But in EC2, I'm only getting the below keys. 

folder_1/sub_folder_1/file1
folder_1/sub_folder_1/file2
folder_1/sub_folder_1/file3

I want to get the same object keys that I'm getting in local also in EC2 as well. What could be the issue.

Upvotes: 1

Views: 783

Answers (1)

jarmod
jarmod

Reputation: 78850

The symptoms suggest that you are actually listing objects in two different S3 buckets.

One of those buckets was likely populated from the AWS S3 console and you used the S3 console to explicitly create folders to hold your files.

The other bucket was populated programmatically by simply uploading objects with keys, and there are no folders (because folders are not required in S3, and in the typical case do not actually exist). You can simply upload folder_1/sub_folder_1/file1 and that process does not need, and will not create, a folder structure of folder_1/sub_folder_1/.

Upvotes: 1

Related Questions