CODEWITHSUNDEEP
node.jsdownload
raphael75
raphael75

Reputation: 3238

nodejs script not downloading files

I have a nodejs script (v. 22.13.1) on a computer with Ubuntu 24.04. It uses puppeteer to connect to a website, get a list of images, and is supposed to download and save the images locally. The first two parts that connect to the site and get the list of images work perfectly. It's the downloading and saving that isn't working.

I am using a commonly-found method for downloading and saving the images found both here on stackoverflow and many blogs. It looks like this:

for(var x=0;x<card_arr.length;x++)
{
    var key = card_arr[x]
    const file = fs.createWriteStream(gp_args['local_path_start'] + '/' + key + '.jpg');
    console.log('getting ' + gp_args['local_path_start'] + '/' + key + '.jpg')
    console.log('from ' + gp_args['url'] + 'img/' + key + '.jpg')
    http.get(gp_args['url'] + 'img/' + key + '.jpg', res => {
        // Write data into local file
        res.pipe(file)

        // Close the file
        file.on('finish', () => {
          file.close()
          console.log(`File downloaded!`)
        })
    })
        .on('error', err => {
        console.log('Error: ', err.message)
    })

    var milliseconds = 300
    var waitTill = new Date(new Date().getTime() + milliseconds);
    while(waitTill > new Date()){}
}

My output looks similar to this:

getting images/remotesite.com/MWSTX1CKI02.jpg
from http://www.remotesite.com/img/MWSTX1CKI02.jpg
getting images/remotesite.com/MWSTX1CKE01.jpg
from http://www.remotesite.com/img/MWSTX1CKE01.jpg
getting images/remotesite.com/MWSTX1CKC01.jpg
from http://www.remotesite.com/img/MWSTX1CKC01.jpg
getting images/remotesite.com/MWSTX1CKQ03.jpg
from http://www.remotesite.com/img/MWSTX1CKQ03.jpg
getting images/remotesite.com/MWSTX1CKC02.jpg
from http://www.remotesite.com/img/MWSTX1CKC02.jpg
getting images/remotesite.com/MWSTX1CKQ05.jpg
from http://www.remotesite.com/img/MWSTX1CKQ05.jpg

However, when I check the folder images/remotesite.com/ on the local computer the folder is empty. I get no error messages, no "File downloaded!" messages, nothing other than that output. I verified that the links are correct by going to a few of them in the browser, and they load just fine. Is there something else it needs?

Upvotes: 2

Views: 38

Answers (0)

Related Questions