CODEWITHSUNDEEP
javamultithreadingcollections
Redandwhite
Redandwhite

Reputation: 2549

Can I check the size of a collection as it is being populated

I'm listing the files on a file system in Java using the following Apache Commons class:

Collection<File> allFiles = FileUtils.listFiles(rootDirectory, null, recursive);

This is a very long process, and can take up to 5 minutes.

Is there any way I can check the size of the collection while it is being populated?

I tried accessing it from a separate thread but simply got zero until the process was ready.

Upvotes: 12

Views: 232

Answers (4)

Amit Deshpande
Amit Deshpande

Reputation: 19185

I guess you can you try using executing platform specific commands like redirecting list command output to temp file and reading it using Java File API.

ls > temp.txt

I created a simple shell script which will list files in given index.

#!/bin/bash

RANGE_UP=$2
RANGE_BOT=$1

CURRENT_CNTR=0

FILENAME=._m_ls

ls -w 1 > $FILENAME
while read line
do
    if [ $CURRENT_CNTR -le $RANGE_UP -a $CURRENT_CNTR -gt $RANGE_BOT ]; then
    printf $line"\n"
    fi
    CURRENT_CNTR=`expr $CURRENT_CNTR + 1`
done < $FILENAME

You can now do something like ./ls_range.sh 10000 30000 >temp.txt and then read from this file using BufferedReader in a different thread.

Upvotes: 1

Roger Lindsj&#246;
Roger Lindsj&#246;

Reputation: 11543

You could use the FileUtils IOFileFilter.

Collection<File> listFiles = FileUtils.listFiles(new File("."), counter, FileFilterUtils.trueFileFilter());

where counter would be along the lines

IOFileFilter counter = new IOFileFilter() {

    @Override
    public boolean accept(File arg0, String arg1) {
        countCall();
        return true;
    }

    @Override
    public boolean accept(File arg0) {
        countCall();
        return true;
    }
};

and countCall is the method which updates progress.

Upvotes: 2

epoch
epoch

Reputation: 16615

Your only way to do this (as opposed to java7 or splitting the work like previously suggested) would be to write your own code to search through the file system while updating a callback.

I used code from commons-io, and modified it to suit your needs, obviously it needs testing, but it counts the files as they are found.

EXAMPLE

interface Counter {
    void foundFile(File file);
}

final class FileSearcher {
    public static Collection<File> listFiles(File root, FileFilter filter, Counter counter) {
        Collection<File> files = new ArrayList<File>();
        File[] found = root.listFiles(filter);

        if (found != null) {
            for (int i = 0; i < found.length; i++) {
                if (found[i].isDirectory()) {
                    files.addAll(listFiles(found[i], filter, counter));
                } else {
                    files.add(found[i]);
                    counter.foundFile(found[i]);
                }
            }
        }

        return files;
    }
}

And then use it like this:

final AtomicInteger inte = new AtomicInteger(0);
FileSearcher.listFiles(new File("C:/"), null, new Counter() {
    @Override
    public void foundFile(File file) {
        System.out.println("found file number " + inte.addAndGet(1));
    }
});

Upvotes: 1

assylias
assylias

Reputation: 328659

Until you get a return value from the method, you won't be able to access the collection as it gets populated, because you don't have access to the variable that this method uses internally.

You could split the search in smaller pieces by not using the recursive flag and deal with the recursion yourself to receive a collection per directory.

A better alternative, if you are using Java 7, is to use the FileVisitor interface to explore the file system. It has a few callback methods that enable you to track the progress.

Upvotes: 11

Related Questions