Java FileVisitor visit ordered in a directory

Question

I am trying to use the Java's FileVisitor interface to walk through a list of files and import the contents onto a database. The scenario is to import the csv contents in order. There may be a file with name Object.csv, Object_updated.csv, Object_deleted.csv and I am required to merge the contents together to a database. A folder hierarchy will contain:

basepath

        folderid

                Objects

                       objectname

                                 objectname.csv

                                 objectname_updated.csv

                                 objectname_deleted.csv

        folderid_2 ....

This is what I have come up with for a basic parsing:

 private void testMethod() throws Exception
    {
        Storage.dumpPath = "C:\\Users\\TestUser\\Documents\\RestoreDummy";

        Files.list(Paths.get(Storage.dumpPath)).sorted().forEach(path ->
        {
            logger.info("Main dir: " + path);
            try
            {
                Files.walkFileTree(path, new SimpleFileVisitor<Path>() {
                    boolean objectDir = false;

                    @Override
                    public FileVisitResult preVisitDirectory(Path paramT, BasicFileAttributes paramBasicFileAttributes)
                            throws IOException
                    {
                        logger.info("Previsit dir: " + paramT);
                        if (paramT.getParent().endsWith(Constants.objectView)
                                && paramT.getParent().getParent().equals(path))
                            objectDir = true;
                        else
                            objectDir = false;
                        return FileVisitResult.CONTINUE;
                    }

                    @Override
                    public FileVisitResult visitFile(Path mypath, BasicFileAttributes paramBasicFileAttributes)
                            throws IOException
                    {
                        if (objectDir && mypath.toString().endsWith("csv"))
                        {
                            logger.info("Visiting: " + mypath);
                        }
                        return FileVisitResult.CONTINUE;
                    }
                });
            }
            catch (Exception e)
            {
                e.printStackTrace();
            }
        });
    }

This returned a list of files is:

Main dir: C:\Users\TestUser\Documents\RestoreDummy\1

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\1\Objects

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\1\Objects\ObjectName

Visiting: C:\Users\TestUser\Documents\RestoreDummy\1\Objects\ObjectName\ObjectName.csv

Main dir: C:\Users\TestUser\Documents\RestoreDummy\2

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\2

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\2\Describes

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\2\Objects

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\2\Objects\ObjectName

Visiting: C:\Users\TestUser\Documents\RestoreDummy\2\Objects\ObjectName\ObjectName_updated.csv

Main dir: C:\Users\TestUser\Documents\RestoreDummy\3

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\3

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\3\Describes

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\3\Objects

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\3\Objects\ObjectName

Visiting: C:\Users\TestUser\Documents\RestoreDummy\3\Objects\ObjectName\ObjectName_deleted.csv

Visiting: C:\Users\TestUser\Documents\RestoreDummy\3\Objects\ObjectName\ObjectName_updated.csv

Main dir: C:\Users\TestUser\Documents\RestoreDummy\4

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\4

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\4\Describes

Main dir: C:\Users\TestUser\Documents\RestoreDummy\5

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\5

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\5\Describes

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\5\Objects

Main dir: C:\Users\TestUser\Documents\RestoreDummy\6

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\6

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\6\Describes

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\6\Objects

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\6\Objects\ObjectName

Visiting: C:\Users\TestUser\Documents\RestoreDummy\6\Objects\ObjectName\ObjectName_updated.csv

Main dir: C:\Users\TestUser\Documents\RestoreDummy\7

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\7

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\7\Describes

Previsit dir: C:\Users\TestUser\Documents\RestoreDummy\7\Objects

If there are two files objectname_updated.csv and objectname_deleted.csv, objectname_updated.csv should be walked first instead of objectname_deleted.csv. This particular instance was working but the documentation says that the file list is not guaranteed to be in order. Is there a way to obtain ordering based on a file name or do I have to store the file list presorted?

Answer 1

In the preVisitDirectory() you may get all the files and sort them and process them.

Then skip visiting the subtree.

Something like this:

private static final List<Path> PATH_ORDER = Arrays.asList(Paths.get("objectname.csv"),
   Paths.get("objectname_updated.csv"),
   Paths.get("objectname_deleted.csv"));

@Override
public FileVisitResult preVisitDirectory(Path paramT, BasicFileAttributes paramBasicFileAttributes)
        throws IOException
{
    logger.info("Previsit dir: " + paramT);
    if (paramT.getParent().endsWith(Constants.objectView)
            && paramT.getParent().getParent().equals(path))
    {
      try(Stream<Path> children = Files.list(paramT))
      {
        Path[] sortedPaths = children
            .filter(path -> path.endsWith(".csv"))
            .sorted((x, y) -> 
                   PATH_ORDER.indexOf(x.getFileName())
                   -
                   PATH_ORDER.indexOf(y.getFileName()))
           .toArray(Path[]::new);

        // Process them
      }

      return FileVisitResult.SKIP_SUBTREE;
    }

    return FileVisitResult.CONTINUE;
}                    

Note: You may also try using Guava Ordering for sorting