CODEWITHSUNDEEP
javalinuxjava-iostack-overflow
Anton Dozortsev
Anton Dozortsev

Reputation: 4902

Java Data structure files StackOverflowError

My program collect all path to files on the computer(OS Ubuntu) to one Map. The key in the Map is a file size and value is list of canonical path to files the size is equal to key.

Map<Long, ArrayList<String>> map = new HashMap<>(100000);

Total number of files on computer is: 281091

A method that collects the files, it is recursively.

private void scanner(String path) throws Exception {

        File[] dirs = new File(path).listFiles(new FileFilter() {
            @Override
            public boolean accept(File file) {

                if (file.isFile() && file.canRead()) {

                    long size = file.length();

                    String canonPath = file.getCanonicalPath();

                    if (map.containsKey(size))
                        map.get(size).add(canonPath);

                    else map.put(size, new ArrayList<>(Arrays.asList(canonPath)));

                    return false;
                }
                return file.isDirectory() && file.canRead();
            }
        });

        for (File dir : dirs) {
            scanner(dir.getCanonicalPath());
        }
    }

When I begin start scanning from the root folder "/" have exception:

Exception in thread "main" java.lang.StackOverflowError
    at java.io.UnixFileSystem.canonicalize0(Native Method)
    at java.io.UnixFileSystem.canonicalize(UnixFileSystem.java:172)
    at java.io.File.getCanonicalPath(File.java:589)
    at taskB.FileScanner.setCanonPath(FileScanner.java:49)
    at taskB.FileScanner.access$000(FileScanner.java:12)
    at taskB.FileScanner$1.accept(FileScanner.java:93)
    at java.io.File.listFiles(File.java:1217)
    at taskB.FileScanner.scanner(FileScanner.java:85)
    at taskB.FileScanner.scanner(FileScanner.java:109)
    at taskB.FileScanner.scanner(FileScanner.java:109)
    ...

But for test I filled directory "~/Documents" more than 400~ thousand files and began to scanning from it. Everything works fine.

Why when the program starts from the root directory "/" where less 300 thousand files I have exception? What should I do to prevent this was?

Upvotes: 1

Views: 2231

Answers (3)

Anton Dozortsev
Anton Dozortsev

Reputation: 4902

@Jim Garrison was right, this was due to symbolic links. Solve their problems I found here.

I use the isSymbolicLink(Path) method.

return file.isDirectory() && file.canRead() && !Files.isSymbolicLink(file.toPath());

Upvotes: 1

Jim Garrison
Jim Garrison

Reputation: 86774

The most likely explanation is that you have a symbolic link somewhere in the filesystem that creates a cycle (an infinite loop). For example, the following would be a cycle

  /home/userid/test/data -> /home/userid

While scanning files you need to ignore symbolic links to directories.

Upvotes: 1

Attila
Attila

Reputation: 28772

StackOverflow means that you called so many nested functions that your program ran out of space in memory for the function call information (retained for after returning from the call). In your case I suspect that it is due to parsing the "." (current directory) and ".." (parent directory) entries when returned in the directory list, thus you recurse into the same directory more than once.

Upvotes: 1

Related Questions