CODEWITHSUNDEEP
javascriptnode.jsamazon-web-servicesamazon-s3
Charlie Fish
Charlie Fish

Reputation: 20496

AWS SDK JS S3 getObject Stream Metadata

I have code similar to the following to pipe an S3 object back to the client as the response using Express, which is working perfectly.

const s3 = new AWS.S3();
const params = {
    Bucket: 'myBucket',
    Key: 'myImageFile.jpg'
};
s3.getObject(params).createReadStream().pipe(res);

Problem is, I want to be able to access some of the properties in the response I get back from S3, such as LastModified, ContentLength, ETag, and more. I want to use those properties to send as headers in the response to the client, as well as for logging information.

Due to the fact that it is creating a stream I can't figure out how to get those properties.

I have thought about doing a separate s3.headObject call to get the data I need, but that seems really wasteful, and will end up adding a large amount of cost at scale.

I have also considered ditching the entire stream idea and just having it return a buffer, but again, that seems really wasteful to be using extra memory when I don't need it, and will probably add more cost at scale due to the extra servers needed to handle requests and the extra memory needed to process each request.

How can I get back a stream using s3.getObject along with all the metadata and other properties that s3.getObject normally gives you back?

Something like the following would be amazing:

const s3 = new AWS.S3();
const params = {
    Bucket: 'myBucket',
    Key: 'myImageFile.jpg',
    ReturnType: 'stream'
};
const s3Response = await s3.getObject(params).promise();
s3Response.Body.pipe(res); // where `s3Response.Body` is a stream
res.set("ETag", s3Response.ETag);
res.set("Content-Type", s3Response.ContentType);
console.log("Metadata: ", s3Response.Metadata);

But according to the S3 documentation it doesn't look like that is possible. Is there another way to achieve something like that?

Upvotes: 3

Views: 3840

Answers (1)

jcuypers
jcuypers

Reputation: 1794

I've found and tested the following. it works.

as per: Getting s3 object metadata then creating stream

function downloadfile (key,res) {

  let stream;

  const params = { Bucket: 'xxxxxx', Key: key }
  const request = s3.getObject(params);


    request.on('httpHeaders', (statusCode, httpHeaders) => {
        console.log(httpHeaders);
        stream.pipe(res)
        stream.on('end', () => { 
            console.log('were done')
        })
    })
    stream = request.createReadStream()

}

Upvotes: 2

Related Questions