CODEWITHSUNDEEP
node.jssftp
Viet
Viet

Reputation: 6953

nodejs download files from sftp

I need to download some .txt.pgp files from sftp. I've tried npm ssh2, ssh2-sftp-client and node-ssh without any success.

The closest I got so far is the list of the files in the remote folder using sftp.readdir (ssh2) or sftp.list (ssh2-sftp-client).

I've tried pipe and fs.createWriteStream and sftp.fastGet but there's no file saved on my local machine.

const conn = new Client();
conn.on('ready', () => {
    console.log('Client :: ready');

    conn.sftp((err, sftp) => {
        if (err) throw err;

        sftp.readdir('out', (err, list) => {
            if (err) throw err;

            list.forEach(item => {
                console.log(item.filename);

                const fileName = item.filename;

                sftp.fastGet(fileName, fileName, {}, downloadError => {
                    if(downloadError) throw downloadError;

                    console.log("Succesfully uploaded");
                });
            })
            conn.end();
        });
    });
}).connect(config);

OR

const Client = require('ssh2-sftp-client');
const sftp = new Client();

sftp.connect(config).then(() => {
    return sftp.list('out');
})
.then(files => {
    // console.log(files);
    if (files.length > 0) {
        console.log('got list of files!');
    }

    files.map(file => {
        const fileName = file.name;

        sftp.get(fileName)
        .then(() => {
            fs.writeFile(fileName);
        }); 
    })
})
.then(() => {
    sftp.end();
}).catch((err) => {
    console.log(err);
});

Upvotes: 4

Views: 10744

Answers (1)

ecs-hk
ecs-hk

Reputation: 181

Regarding your first attempt (with the ssh2 module), there are three issues that I can see:

  1. You are calling conn.end() outside of a series of preceding async functions, which is almost definitely causing the SSH session to close before you've finished downloading the files.
  2. You are not providing the sftp.fastGet() function with the correct path to the remote file. Earlier in the code, you call sftp.readdir() with the remote directory argument 'out', which returns a list of files relative to the remote directory. (Point is: you need to prepend the remote directory to the file name to create a correctly qualified path.)
  3. You're not handling the stream event error, so I suspect you're not getting useful error messages to help troubleshoot.

Try something like:

const Client = require('ssh2').Client;
const conn = new Client();    
const sshOpt = someFunctionThatPopulatesSshOptions();    
const remoteDir = 'out';

conn.on('ready', () => {
  conn.sftp((err, sftp) => {
    if (err) throw err;
    sftp.readdir(remoteDir, (err, list) => {
      if (err) throw err;
      let count = list.length;
      list.forEach(item => {
        let remoteFile = remoteDir + '/' + item.filename;
        let localFile = '/tmp/' + item.filename;
        console.log('Downloading ' + remoteFile);
        sftp.fastGet(remoteFile, localFile, (err) => {
          if (err) throw err;
          console.log('Downloaded to ' + localFile);
          count--;
          if (count <= 0) {
            conn.end();
          }
        });
      });
    });
  });
});
conn.on('error', (err) => {
  console.error('SSH connection stream problem');
  throw err;
});
conn.connect(sshOpt);

This should address all the issues I mentioned. Specifically:

  1. We are using a count variable to ensure the SSH session is closed only after all files are downloaded. (I know it is not pretty.)
  2. We are prepending remoteDir to all of our remote file downloads.
  3. We're listening for the error event in our conn stream.

Upvotes: 8

Related Questions