Java : copy files efficiently with channel

Question

I read one article regarding trasfer copy at https://www.ibm.com/developerworks/library/j-zerocopy/. which suggest to user channel for IO operation.

There is a banchmark of copy file operation available at https://baptiste-wicht.com/posts/2010/08/file-copy-in-java-benchmark.html

As per benchmark I can use either nio buffer or nio trasfer

I also read FileChannel does buffereing in the OS level here How to implement a buffered / batched FileChannel in Java?

Which is more efficient way to copy file with buffer or with out buffer.

code for nio buffer

public static void nioBufferCopy(File sourceFile, File targetFile, int BUFFER) {
        FileChannel inputChannel = null;
        FileChannel outputChannel = null;
        try {
            inputChannel = new FileInputStream(sourceFile).getChannel();
            outputChannel = new FileOutputStream(targetFile).getChannel();
            ByteBuffer buffer = ByteBuffer.allocateDirect(BUFFER);
            while (inputChannel.read(buffer) != -1) {
                buffer.flip();          
                while(buffer.hasRemaining()){
                    outputChannel.write(buffer);
                }           
                buffer.clear();
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
           //close resource
        }
}

code for nio trasfer

public void copyFileWithChannels(File aSourceFile, File aTargetFile) {              
        FileChannel inChannel = null;
        FileChannel outChannel = null;
        FileInputStream inStream = null;
        FileOutputStream outStream = null;      
        try {
            inStream = new FileInputStream(aSourceFile);
            inChannel = inStream.getChannel();
            outStream = new  FileOutputStream(aTargetFile);        
            outChannel = outStream.getChannel();
            long bytesTransferred = 0;
            while(bytesTransferred < inChannel.size()){
                bytesTransferred += inChannel.transferTo(bytesTransferred, inChannel.size(), outChannel);
            }
        }       
        catch (Exception e) {
            e.printStackTrace();
        }
        finally {
            //close resource
        }              
}

Answer 1

We had a use case where we needed to copy GBs of the file, and our experience NIO outperforms traditional IO but uses concepts of channels with decent buffer size. Channels provide the ability to perform a single I/O operation across multiple buffers. We used the idea of FileChannel, which is thread-safe compared to buffers and also supports the concept of a current file position that helps determine the location where the next data item will be read from or written to. Our implementation using ReadableByteChannel and WritableByteChannel is

import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.nio.channels.WritableByteChannel;

import org.apache.commons.io.IOUtils;

/**
 * A file channel that leverages NIO classes internally to perform read/write operations on the provided streams to achieve high-performance I/O.
 *
 */
public final class FileChannel implements Closeable {

    // default buffer size used for reading and writing
    private static final int BUFFER_SIZE = 65536;

    private final ReadableByteChannel source;
    private final WritableByteChannel destination;
    private final int bufferSize;
    private final boolean shouldClose;

    public FileChannel(final InputStream source, final OutputStream destination) {
        this(source, destination, BUFFER_SIZE, true);
    }

    public FileChannel(final InputStream source,
            final OutputStream destination,
            final boolean shouldClose) {
        this(source, destination, BUFFER_SIZE, shouldClose);
    }

    public FileChannel(final InputStream source,
            final OutputStream destination,
            final int bufferSize,
            final boolean shouldClose) {
        this.source = Channels.newChannel(source);
        this.destination = Channels.newChannel(destination);
        this.bufferSize = bufferSize;
        this.shouldClose = shouldClose;
    }

    /**
     * Transfer bytes from source to destination stream.
     */
    public void transfer() {
        // Used Direct byte buffers because this is the most efficient means of performing I/O on the JVM
        final ByteBuffer buffer = ByteBuffer.allocateDirect(this.bufferSize);

        while (read(buffer) != -1) {
            // prepare the buffer to be drained
            buffer.flip();

            // write to the channel
            write(buffer);

            /*
             * If partial transfer, shift remainder down. If buffer is empty, same as doing clear()
             */
            buffer.compact();
        }

        // EOF will leave buffer in fill state
        buffer.flip();

        // make sure the buffer is fully drained.
        while (buffer.hasRemaining()) {
            write(buffer);
        }
    }

    int read(final ByteBuffer byteBuffer) {
        try {
            return this.source.read(byteBuffer);
        } catch (final IOException exception) {
            throw new IllegalStateException("Error reading from source channel", exception);
        }
    }

    private void write(final ByteBuffer byteBuffer) {
        try {
            while (byteBuffer.hasRemaining()) {
                this.destination.write(byteBuffer);
            }
        } catch (final IOException exception) {
            throw new IllegalStateException("Error writing to destination channel", exception);
        }
    }

    @Override
    public void close() {
        if (!this.shouldClose) {
            return;
        }

        IOUtils.closeQuietly(this.source);
    }
}

Answer 2

This question was asked before:

Java NIO FileChannel versus FileOutputstream performance / usefulness

TL.DR.: It matters what your JVM is running on, but mostly the java.nio is slightly faster.