CODEWITHSUNDEEP
node.jsamazon-s3aws-sdkaws-sdk-nodejs
appy
appy

Reputation: 609

GET list of objects located under a specific S3 folder

I am trying to GET a list of objects located under a specific folder in an S3 bucket using a query-string which takes the foldername as the parameter and list all objects which match that specific folder using Node JS aws-sdk

For example: http://localhost:3000/listobjects?foldername=xxx

Please suggest how to implement this functionality.

Upvotes: 45

Views: 92266

Answers (4)

Aurangzaib Rana
Aurangzaib Rana

Reputation: 4252

AWS S3 gives a maximum of 1000 files list. In order to get more than 1000 count use this approach:

export function getListingS3(prefix) {
  return new Promise((resolve, reject) => {
    try {
      let params = {
        Bucket: AWS_S3.BUCKET_NAME,
        MaxKeys: 1000,
        Prefix: prefix,
        Delimiter: prefix
      };
      const allKeys = [];
      listAllKeys();
      function listAllKeys() {
        s3.listObjectsV2(params, function (err, data) {
          if (err) {
            reject(err)
          } else {
            var contents = data.Contents;
            contents.forEach(function (content) {
              allKeys.push(content.Key);
            });

            if (data.IsTruncated) {
              params.ContinuationToken = data.NextContinuationToken;
              console.log("get further list...");
              listAllKeys();
            } else {
              resolve(allKeys);
            }
          }
        });
      }
    } catch (e) {
      reject(e);
    }

  });
}```

Upvotes: 14

Amit Shakya
Amit Shakya

Reputation: 994

You forget to mention folder into s3 bucket, anyways this code works for me

var params = {
  Bucket: 'Bucket_Name',
  Delimiter: '/',
  Prefix: 'foldername/'
};

s3Bucket.listObjects(params, function(err, data) {
  if (err) {
    return 'There was an error viewing your album: ' + err.message
  } else {
    console.log(data.Contents,"<<<all content");
                
    data.Contents.forEach(function(obj,index) {
      console.log(obj.Key,"<<<file path")
    })
  }
})

Upvotes: 29

Tobi
Tobi

Reputation: 1902

Starting with index = 1 in the loop excludes the folder itself + just lists the files in the folder:

const s3 = new AWS.S3();

const params = {
    Bucket: bucketname,
    Delimiter: '/',
    Prefix: s3Folder + '/'
};

const data = await s3.listObjects(params).promise();

for (let index = 1; index < data['Contents'].length; index++) {
    console.log(data['Contents'][index]['Key'])        
}

Upvotes: 15

Abhyudit Jain
Abhyudit Jain

Reputation: 3748

You can specify the prefix while calling the getObject or listObjectsV2 in aws-sdk

var params = {
  Bucket: 'STRING_VALUE', /* required */
  Prefix: 'STRING_VALUE'  // Can be your folder name
};
s3.listObjectsV2(params, function(err, data) {
  if (err) console.log(err, err.stack); // an error occurred
  else     console.log(data);           // successful response
});

By the way, S3 doesn't have folders. It is just a prefix. It shows you the folder structure to make it easy for you too navigate and see files.

Source: AWS SDK Javascript

Upvotes: 55

Related Questions