CODEWITHSUNDEEP
javascriptnode.jsapiamazon-s3pre-signed-url
AVu
AVu

Reputation: 125

Generating multiple pre-signed urls with an API

I've created an API that generates a pre-signed url for files that are in my private s3 bucket. I want to store these urls into an array so that I can call them from another application. ["FILE1 pre-signed url","FILE2 pre-signed url", etc..] However, I'm confused as to how my API is reading in the commands so I came here for help.

  1. Why doesn't the page that hosts the API show the populated array when I enter? It's only until I refresh the page that it finally populates the array.

I want the array to reset every time and have the links replaced instead of adding on to the end of the array. I also need the array to show as soon as the page is loaded instead of refreshing it to see the results.

This is what is shown when I enter the page that the API is served on: enter image description here

When I hit refresh the array is populated with the pre-signed urls:

["https://bucketname.s3.amazonaws.com/file1.pdf?AWSAccessKeyId=ACCESSKEY&Expires=1572669513&Signature=SIGNATURE","https://bucketname.s3.amazonaws.com/file2.pdf?AWSAccessKeyId=ACCESSKEY&Expires=1572669513&Signature=SIGNATURE"]

HOWEVER, when I refresh the page again, it pushes these two files AGAIN making the array four elements instead of two. And if I keep refreshing the page it keeps adding on two more urls and so on. (WHY WOULD IT DO THIS IF THE LOOP STOPS AT 2 ITERATIONS)

const express = require('express');
const bodyParser = require('body-parser');
const morgan = require('morgan');
const uuid = require('uuid');
const AWS = require('aws-sdk');
let config = require('./config');

var s3 = new AWS.S3();
s3.config.update({
    accessKeyId: config.accessKeyId,
    secretAccessKey: config.secretAccessKey,
    region: 'us-east-1'
});

var app = express();
var router = express.Router();

// Create an array that houses pre-signed Urls


// Loop through x amt of times (depending on number of documents)
// Push each url into an array called 'preSignedUrls'
// Exit loop and reset array to re-generate new urls
let preSignedUrls = [];
    function getSignedURL(req, res, next) {
    Key = ['file1.pdf', 'file2.pdf']
        for ( i = 0; i < Key.length; i++ ){
            var params = {
                Bucket: 'bucketName',
                Key: Key[i],
                Expires: 36000
            };
                console.log(Key.length);
                console.log(i);
                s3.getSignedUrl('getObject', params, function(err, signedURL) {
                    if (err) {
                        console.log(err);
                        return next(err);
                    } else {
                        console.log(signedURL);
                        console.log(params);
                        preSignedUrls.push(signedURL);
                    }
                });
        }
        res.json(preSignedUrls);

    }
router.route('/get_signed_url')
    .get(getSignedURL);

app.use(morgan('combined'));
app.use(bodyParser.json());
app.use('/v1', router);

var PORT = process.env.PORT || 3001;
var HOST = process.env.HOST || '10.123.456.789';

console.log("Listening on", HOST, PORT);
app.listen(PORT, HOST);

This is the output from node.js terminal (AKA the log commands I put into my code to test output)

2
0
2
1
https://bucketname.s3.amazonaws.com/file1.pdf?AWSAccessKeyId=ACCESSKEY&Expires=1572669520&Signature=SIGNATURE
{ Bucket: 'bucketname',
  Key: 'file2.pdf',
  Expires: 36000 }
https://bucketname.s3.amazonaws.com/file2.pdf?AWSAccessKeyId=ACCESSKEY&Expires=1572669520&Signature=SIGNATURE
{ Bucket: 'bucketname',
  Key: 'file2.pdf',
  Expires: 36000 }

What I find weird about this is that it logs the object params as file2.pdf but the pre-signed URL it outputs shows file1.pdf and file2.pdf in the order that it should. Also, the fact that Length and variable i is displayed before the objects are which confuses me since I thought for loops execute commands in order.

Upvotes: 2

Views: 4379

Answers (1)

Swapnil Mhaske
Swapnil Mhaske

Reputation: 365

Because signedURL is a response from s3.getSignedUrl(). It is run asynchronously and have its own value and context. But variable params on the other hand will return mostly last value of the for loop. This is because the function s3.getSignedUrl is not yet executed and the for loop gets executed earlier. The for loop doesn't wait for the s3.getSignedUrl() to get completed as its async. You might like to use arrow function if you want to retain the i value.

Heres a way on other thread to do this. async / await looping through array of to get s3 signed URL's

Upvotes: 3

Related Questions