CODEWITHSUNDEEP
javascriptnode.jspromisesftp
oosniss
oosniss

Reputation: 460

How to detect when multiple asynchronous calls for multiple arrays are complete in Node.js

I am using [ssh2-sftp-client][1] package to recursively read all the directories inside a given remote path.

Here is the code.

const argv = require('yargs').argv;
const client = require('ssh-sftp-client');
const server = new Client();
const auth = {
    host: '192.168.1.11',
    username: argv.u,
    password: argv.p
};

const serverRoot = '/sites/';
const siteName = 'webmaster.com';

// list of directories on the server will be pushed to this array
const serverPaths = [];

server.connect(auth).then(() => {
    console.log(`connected to ${auth.host} as ${auth.username}`);
}).catch((err) => {
    if (err) throw err;
});

server.list('/sites/').then((dirs) => {
    redursiveDirectorySearch(dirs, `${serverRoot}${siteName}/`);
})
.catch((err) => {
    if (err) throw err;
});

function recursiveDirectorySearch(dirs, prevPath) {
    let paths = dirs.filter((dir) => {
    // returns directories only
        return dir.type === 'd';
    });

    if (paths.length > 0) {
        paths.forEach((path) => {
            server
                .list(`${prevPath}${path.name}`)
                .then((dirs) => {
                    console.log(`${prevPath}${path.name}`);
                    recursiveDirectorySearch(dirs, `${prevPath}${path.name}`);
                    serverPaths.push(`${prevPath}${path.name}`);
                })
        }
    }
}

At first, a connection will be made to the server and then list whatever is under '/sites/' directory, which will then be passed to 'recursiveDirectorySearch' function. This function will receive an array of whatever is found under '/sites/' directory on the server as the first parameter, which will be filtered out so it only has directories. If one or more directory was found, a call to the server for each directory in the array will be made in order to retrieve everything under '/sites/'+'name of the directory in the array'. This same function will be called again with whatever is returned by the call to the server until no other directory is found.

Whenever a directory is found, its name in string will be pushed to 'serverPaths' array. As far as I can tell, this search is working and successfully pushing all the directory names to the array.

However, I can't think of a way to detect when this recursive search for all the directories is complete so I can do something with the 'serverPaths' array.

I tried to take advantage of Promise.all() but don't know how to use it when how many function calls are made is unknown.

Upvotes: 2

Views: 133

Answers (2)

Jaromanda X
Jaromanda X

Reputation: 1

You're simply lacking a couple of returns, add a Promise.all, and an Array#map and you're done

Note: not using Promise.all on serverPaths, but rather, using the fact that returning a Promise in .then will result in the Promise that is returned by .then taking on the Promise that is returned (hmmm, that isn't very well explained, is it, but it's Promises 101 stuff really!

server.list('/sites/').then((dirs) => {
    // added a return here
    return recursiveDirectorySearch(dirs, `${serverRoot}${siteName}/`);
})
.then(() => {
    // everything is done at this point,
    // serverPaths should be complete
})
.catch((err) => {
    if (err) throw err;
});

function recursiveDirectorySearch(dirs, prevPath) {
    let paths = dirs.filter((dir) => {
    // returns directories only
        return dir.type === 'd';
    });
    // added a return, Promise.all and changed forEach to map
    return Promise.all(paths.map((path) => {
        //added a return here
        return server
            .list(`${prevPath}${path.name}`)
            .then((dirs) => {
                console.log(`${prevPath}${path.name}`);
                // swapped the next two lines
                serverPaths.push(`${prevPath}${path.name}`);
                // added a return here, push the path before
                return recursiveDirectorySearch(dirs, `${prevPath}${path.name}`);
            })
    }));
}

Upvotes: 2

Matt Fernandez
Matt Fernandez

Reputation: 188

One of the main things that is jumping out at me is your initial if statement. (if paths.length > 0) { run recursion } This appears to work really well for the first call because you know that the data coming back will be populated with an array full of directories.

Your function however, does not appear to have logic built out for an array with a length of 0. In this scenario it would be possible for you to get all of the directory names you are looking for. Presented in the manner that you are looking for. It would also mean that your calls on the higher parts of the tree are never able to resolve.

Try to add logic to handle cases for an array with a length of zero | if (paths.length === 0) return; | This would be a hard break out of the recursive calls on the higher parts of the stack.

Upvotes: 0

Related Questions