How do I monitor the progress of a file transfer in Apache Commons VFS?

Question

Using Apache Commons VFS, how do I monitor the progress of a file transfer. I need to be able to do it with uploads and downloads. I also need to monitor progress over HTTP, FTP, SFTP and FTPS. I can't find anything in the documentation about it.

Answer 1

The proposed solution is kinda off, and a bit "weak". Let's try with a neater one!
Note: the following code requires JDK 16.

Given an example copy operation initiated with the standard:

final var manager = VFS.getManager();
final var origin = manager.resolveFile(originUri.toString(), fileSystemOptions);
final var destination = manager.resolveFile(destinationUri.toString(), fileSystemOptions);
destination.copyFrom(origin, Selectors.SELECT_ALL);

Simply wrap the destination FileObject with a custom delegating implementation that we may call ProgressFileObject, which will accept also a ProgressListener:

final var listener = new ProgressListener() {
  @Override
  public void started() {
    System.out.println("Started");
  }

  @Override
  public void completed() {
    System.out.println("Completed");
  }

  @Override
  public void failed(@NotNull final Exception e) {
    System.out.println("Failed: " + e.getMessage());
  }

  @Override
  public void progress(@NotNull final ProgressEvent event) {
    final var out = "%d\t\t\t%d\t\t\tFile: %s".formatted(
        event.totalBytes(),
        event.totalTransferredBytes(),
        event.lastFileDestinationPath()
    );

    System.out.println(out);
  }
};

final var progressFileObject = new ProgressFileObject(destination, listener);
progressFileObject.copyFrom(origin, Selectors.SELECT_ALL);

Which will result in, with a folder:

Started
4648320         119391          File: /test_dir/AN/ANI0.evt
4648320         119511          File: /test_dir/AN/ANI0.hpj
4648320         119584          File: /test_dir/AN/ANI0.ipf
4648320         119585          File: /test_dir/AN/ANI0.ipm
4648320         1907060         File: /test_dir/AN/ANI0.LST
4648320         1907253         File: /test_dir/AN/ANI0.MRG
4648320         2472700         File: /test_dir/AN/ANI0.ODF
4648320         2472707         File: /test_dir/AN/ANI0.ph
4648320         2473421         File: /test_dir/AN/ANI0.rbj
4648320         2473547         File: /test_dir/AN/ANI0.rc
4648320         2473708         File: /test_dir/AN/ANI0.rst
4648320         2474813         File: /test_dir/AN/ANI0.rtf
4648320         2474814         File: /test_dir/AN/ANI0.txc
4648320         2474819         File: /test_dir/AN/ANI0.txm
4648320         2474820         File: /test_dir/AN/ANI0.vpf
4648320         2829348         File: /test_dir/AN/ANI0.VPG
4648320         2829466         File: /test_dir/AN/P0000001.rc
4648320         2829592         File: /test_dir/AN/P0000002.rc
4648320         4648275         File: /test_dir/AN/VPGSAV55.C2T
4648320         4648320         File: /test_dir/AN/VRPGWIN.RC
Completed

And here are the ProgressFileObject, ProgressListener, and ProgressEvent sources:

/**
 * @author Edoardo Luppi
 */
public class ProgressFileObject extends DecoratedFileObject {
  private final FileObject fileObject;
  private final ProgressListener listener;

  public ProgressFileObject(
      @NotNull final FileObject fileObject,
      @NotNull final ProgressListener listener) {
    super(fileObject);
    this.fileObject = fileObject;
    this.listener = listener;
  }

  @Override
  public void copyFrom(final FileObject file, final FileSelector selector) throws FileSystemException {
    if (!FileObjectUtils.exists(file)) {
      final var exception = new FileSystemException("vfs.provider/copy-missing-file.error", file);
      listener.failed(exception);
      throw exception;
    }

    listener.started();

    // Locate the files to copy across
    final List<FileObject> files = new ArrayList<>();
    file.findFiles(selector, false, files);

    // Calculate the total bytes that will be copied
    var totalBytes = 0L;

    for (final var srcFile : files) {
      if (srcFile.getType().hasContent()) {
        totalBytes += srcFile.getContent().getSize();
      }
    }

    var totalTransferredBytes = 0L;

    // Copy everything across
    for (final var srcFile : files) {
      // Determine the destination file
      final var relPath = file.getName().getRelativeName(srcFile.getName());
      final var destFile = resolveFile(relPath, NameScope.DESCENDENT_OR_SELF);

      // Clean up the destination file, if necessary
      if (FileObjectUtils.exists(destFile) && destFile.getType() != srcFile.getType()) {
        // The destination file exists, and is not of the same type, so delete it
        destFile.deleteAll();
      }

      // Copy across
      try {
        if (srcFile.getType().hasContent()) {
          try (final var content = srcFile.getContent()) {
            final var fileTransferredBytes = content.write(destFile);
            totalTransferredBytes += fileTransferredBytes;
            final var event = new ProgressEvent(
                totalBytes,
                totalTransferredBytes,
                fileTransferredBytes,
                srcFile.getName().getPath(),
                destFile.getName().getPath()
            );
        
            listener.progress(event);
          }
        } else if (srcFile.getType().hasChildren()) {
          destFile.createFolder();
        }
      } catch (final IOException e) {
        final var exception = new FileSystemException("vfs.provider/copy-file.error", e, srcFile, destFile);
        listener.failed(exception);
        throw exception;
      }
    }

    listener.completed();
  }

  @Override
  public URI getURI() {
    return fileObject.getURI();
  }

  @Override
  public Path getPath() {
    return fileObject.getPath();
  }

  @Override
  public boolean isSymbolicLink() throws FileSystemException {
    return fileObject.isSymbolicLink();
  }

  @Override
  public void forEach(final Consumer<? super FileObject> action) {
    fileObject.forEach(action);
  }

  @Override
  public Spliterator<FileObject> spliterator() {
    return fileObject.spliterator();
  }

  @Override
  public int hashCode() {
    return fileObject.hashCode();
  }

  @Override
  public boolean equals(final Object obj) {
    return fileObject.equals(obj);
  }

  @Override
  public String toString() {
    return fileObject.toString();
  }

  public interface ProgressListener {
    void started();
    void completed();
    void failed(@NotNull final Exception e);
    void progress(@NotNull final ProgressEvent event);
  }

  public record ProgressEvent(
      long totalBytes,
      long totalTransferredBytes,
      long lastFileTransferredBytes,
      String lastFileOriginPath,
      String lastFileDestinationPath
  ) {}
}

Answer 2

It can be done by getting input and output streams from VFS. The following example uses a utility class from commons-net (a dependency of VFS) to manage the copying and progress-monitoring. You could equally do it manually.

import org.apache.commons.net.io.Util;
import org.apache.commons.net.io.CopyStreamListener;

private void copy(FileObject sourceFile, FileObject destinationFile, CopyStreamListener progressMonitor) throws IOException {
    InputStream sourceFileIn = sourceFile.getContent().getInputStream();
    try {
        OutputStream destinationFileOut = destinationFile.getContent().getOutputStream();
        try {
            Util.copyStream(sourceFileIn, destinationFileOut, Util.DEFAULT_COPY_BUFFER_SIZE, sourceFile.getContent().getSize(), progressMonitor);
        } finally {
            destinationFileOut.close();
        }
    } finally {
        sourceFileIn.close();
    }
}