CODEWITHSUNDEEP
node.jsgoogle-cloud-platform
luk
luk

Reputation: 355

Upload image from local directory to Google Cloud Storage using Node JS

I'm using Node JS / Express and would like to download a file from url to local system, and in the next step upload it Google Cloud Storage.

This is my router with middlewares:

  router.post("", fileFromUrl, uploadFromUrl, scrapeController.scrapeCreateOne);

this is a fileFromUrl middleware that is just saving a file from url to local disk

module.exports = (req, res, next) => {
try {
    console.log('Image: ', req.body.image);

    const url = req.body.image ? req.body.image : '';
    console.log(typeof url);

    if(url === '') {
        //no image url provided
        console.log('image parsing skipped');
        next()
    }
    else {
         // image url ok then 
        const pathToImage = path.resolve(__dirname, '..', 'images', Date.now() + '_file.jpg');
        const localPath = fs.createWriteStream(pathToImage);
        
        const saveFile = https.get(url, (response) => {
            console.log(response.headers['content-type']);
            response.pipe(localPath);
        })

        req.body.customImageUrl = pathToImage;
        req.body.customImageName = path.basename(pathToImage);
        
        next();
    }

}
catch (error) {
    console.log(error)
}

}

this is uploadFromUrl middleware that should upload the file from local path to the Google Cloud Storage

module.exports = (req, res, next) => {
try {
    console.log(req.body.customImageUrl);
    console.log(req.body.customImageName);
    //storage.getBuckets().then(x => console.log(x));
    //storage.createBucket(`testbucket_${Date.now()}`); / // it does work
    storage.bucket(bucketName).upload(req.body.customImageUrl, {
        gzip: true,
        metadata: {
        cacheControl: 'public, max-age=31536000',
        },
    }).then(
        req.body.customData = `https://storage.googleapis.com/${bucketName}/${req.body.customImageName}`
    ); 
    next();
}
catch (error) {
    res.send(error).json({
        message: 'Error upload middleware' + error,
    });
}

}

What is does right now, is just uploading almost empty file with 20kB to the Google Cloud Platform, not the full image. I feel that I'm not providing a proper file object to the uploadFromUrl middleware. On the other hand GCP API to upload a file is just asking for the path to the file which is being provided. Any ideas ?

Upvotes: 0

Views: 1489

Answers (1)

luk
luk

Reputation: 355

The issue was that I was trying to upload image to GCP, even though the image was not yet fully saved on the server. The solution was to wait for 'finish' event in the request I made to save it locally

 const pathToImage = path.resolve(__dirname, '..', 'images', Date.now() + '_file.jpg');
        const localPath = fs.createWriteStream(pathToImage);
        const fileRelativePath = "images/" + path.basename(pathToImage);
        const request = https.get(url, (response) => {
            //console.log(response.headers['content-type']);
            response.pipe(localPath).on('finish', () => {
                console.log('image saved on the server');
                req.body.customImagePath = fileRelativePath;
                req.body.customImageName = path.basename(pathToImage);
                // now you can go and upload the image to GCP in the next moddleware
                next();
            });
        });

Upvotes: 1

Related Questions