Ideas for listing files, directories, sub files & sub directories without recursion in ftp server

Question

I'm trying to produce file listing of a given directory and it's sub directories in a ftp server.

The server works fine, and I have been successfully able to produce the file listing of the current directory. When I try to list the subdirectories and their files is where it gets complicated.

I was asked not to use a recursion algorithm, so I did some research of my own. I have tried using threads (for every directory found, start a new thread), but I wasn't able to keep my connection stable and open. Any ideas on how to do so correctly with threads, or other alternatives?

EDIT: below is my code, when using the recursive statement (last line of code), it works

   class TEST {
        public static synchronized void main(String[] args) {
        String server = args[0]; //server,path will be given as an arguments
        String pass = "SOMEPASS";
        String user = "SOMEUSER";
        int port = 21;
        FTPClient ftpClient = new FTPClient();
        try {
            ftpClient.connect(server, port);
            showServerReply(ftpClient);
            int replyCode = ftpClient.getReplyCode();
            if (!FTPReply.isPositiveCompletion(replyCode)) {
                System.out.println("Connect failed");
                return;
            }
            boolean success = ftpClient.login(user, pass);
            showServerReply(ftpClient);
            if (!success) {
                System.out.println("Could not login to the server");
                return;
            }
           /*START THE FILE LISTING HERE*/

        } catch (IOException ex) {
            System.out.println("Oops! Something wrong happened");
            ex.printStackTrace();
        } finally {
            // logs out and disconnects from server
            try {
                if (ftpClient.isConnected()) {
                    ftpClient.logout();
                    ftpClient.disconnect();
                }
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }
    }
    private static void showServerReply(FTPClient ftpClient) {
        String[] replies = ftpClient.getReplyStrings();
        if (replies != null && replies.length > 0) {
            for (String aReply : replies) {
                System.out.println("SERVER: " + aReply);
            }
        }
    }
    private static void scanDir(FTPClient client, String path) throws IOException {
        FTPFile[] files = client.listFiles(path); // Search all the files in the current directory
        
        for (int j = 0; j < files.length; j++) {
            System.out.println(files[j].getName()); // Print the name of each files
        }
        FTPFile[] directories = client.listDirectories(path); // Search all the directories in the current directory
        for (int i = 0; i < directories.length; i++) {
            String dirPath = directories[i].getName();
            System.out.println(dirPath); // Print the path of a sub-directory
            scanDir(client,dirPath); // Call recursively the method to display the files in the sub-directory DONT WANT TO DO THAT...
        }
    }
}

Answer 1

Okay, here is an example of how to handle it non-recursively, but with lists.

Mind, that this example is based on /accessing the local filesystem, but can easily be rewritten/extended for any kind of hierarchial/recursive structure.

package stackoverflow.nonrecursivefilesearch;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.stream.Stream;

public class NonRecursiveFileSearch {

    public static void main(final String[] args) throws IOException {
        final File searchDir = new File("D:\\test\\maven-test"); // set one

        System.out.println("\nOld Java");
        printDirs(listFiles_old(searchDir, true, true), "OLD: Depth first, include dirs");
        printDirs(listFiles_old(searchDir, true, false), "OLD: Breadth first, include dirs");
        printDirs(listFiles_old(searchDir, false, true), "OLD: Depth first, exclude dirs");
        printDirs(listFiles_old(searchDir, false, false), "OLD: Breadth first, exclude dirs");

        System.out.println("\nNew java.io with streams");
        printDirs(listFiles_newIO(searchDir, true), "Java NIO, include dirs");
        printDirs(listFiles_newIO(searchDir, false), "Java NIO, exclude dirs");
    }

    /**
     * this is the way to 'manually' find files in hierarchial/recursive structures
     *
     * reminder: "Depth First" is not a real depth-first implementation
     * real depth-first would iterate subdirs immediately.
     * this implementation iterates breadth first, but descends into supdirs before it handles same-level directories
     * advantage of this implementation is its speed, no need for additional lists etc.
     *
     * in case you want to exclude recursion traps made possible by symbolic or hard links, you could introduce a hashset/treeset with
     * visited files (use filename strings retrieved with canonicalpath).
     * in the loop, check if the current canonical filename string is contained in the hash/treeset
     */
    static public ArrayList<File> listFiles_old(final File pDir, final boolean pIncludeDirectories, final boolean pDepthFirst) {
        final ArrayList<File> found = new ArrayList<>();
        final ArrayList<File> todo = new ArrayList<>();

        todo.add(pDir);

        while (todo.size() > 0) {
            final int removeIndex = pDepthFirst ? todo.size() - 1 : 0;
            final File currentDir = todo.remove(removeIndex);
            if (currentDir == null || !currentDir.isDirectory()) continue;

            final File[] files = currentDir.listFiles();
            for (final File file : files) {
                if (file.isDirectory()) {
                    if (pIncludeDirectories) found.add(file);
                    // additional directory filters go here
                    todo.add(file);
                } else {
                    // additional file filters go here
                    found.add(file);
                }
            }
        }

        return found;
    }

    static private void printDirs(final ArrayList<File> pFiles, final String pTitle) {
        System.out.println("====================== " + pTitle + " ======================");
        for (int i = 0; i < pFiles.size(); i++) {
            final File file = pFiles.get(i);
            System.out.println(i + "\t" + file.getAbsolutePath());
        }
        System.out.println("============================================================");
    }

    /**
     * this is the java.nio approach. this is NOT be a good solution for cases where you have to retrieve/handle files in your own code.
     * this is only useful, if the any NIO class provides support. in this case, NIO class java.nio.file.Files helps handling local files.
     * if NIO or your target system does not offer such helper methods, this way is harder to implement, as you have to set up the helper method yourself.
     */
    static public Stream<Path> listFiles_newIO(final File pDir, final boolean pIncludeDirectories) throws IOException {
        final Stream<Path> stream = Files.find(pDir.toPath(), 100,
                (path, basicFileAttributes) -> {
                    final File file = path.toFile(); // conversion to File for easier access (f.e. isDirectory()), could also use NIO methods
                    return (pIncludeDirectories || !file.isDirectory() /* additional filters go here */ );
                });
        return stream;

    }
    static private void printDirs(final Stream<Path> pStream, final String pTitle) {
        System.out.println("====================== " + pTitle + " ======================");
        pStream.forEach(System.out::println);
        System.out.println("============================================================");
    }
}

AND, one must add, java.nio.file.Files.find() might be implemented recursively. But as it's just one call, this maybe could count as 'non-recursive' too.

ALSO, as the OP stated in comments, one might use Stack or other FIFO/LIFO collections. LIFO for a mixed depth-first, FIFO for breadth-first approach.