CODEWITHSUNDEEP
javascriptfileapinative-file-system-api-js
Triet Doan
Triet Doan

Reputation: 12085

Performance problem with the FileSystemWritableFileStream.write method

Context

One feature of my web application is copying file from A to B on user's machine, not in the OPFS. The destination (B) is a mounted NFS, so the write speed is the Internet upload speed.

File size varies between 1 GB and 200 GB.

Here is my code to copy a file:

// Source file
const file = await fileHandle.getFile();

// Destination
const fileDestination = await location.getFileHandle(file.name, { create: true });

const writable = await fileDestination.createWritable();
try {
  await writable.write(file);
  await writable.close();
}
catch (e) {
  console.error(e);
}

Problem

I notice that, the copy speed of the code above is much slower than manual copy with Ctrl + C and Ctrl + V. It fluctuates between 2 to 10 times slower.

The FileSystemWritableFileStream.write() documentation says:

No changes are written to the actual file on disk until the stream has been closed. Changes are typically written to a temporary file instead.

Does it mean that when I use the write() method, it first writes to a temporary file. And when I close the stream, the temporary file is moved to the actual file destination?

If so,

  1. Where does this temporary file locate?
  2. How can I avoid this? Can I write directly to the destination file instead?

I notice that I can also do

await file.stream().pipeTo(writable);

instead of using the FileSystemWritableFileStream.write() method. But I'm not sure if it helps.

Upvotes: 0

Views: 19

Answers (1)

Triet Doan
Triet Doan

Reputation: 12085

I've found the cause of the performance difference. As the documentation says, the changes are not done in-place, but on a temporary file. This file is created at the chosen destination with the extension .crswap when FileSystemFileHandle.createWritable() is called.

When the stream is close(), the browser scans this .crswap file. If everything is right, this temporary file becomes the "real" file.

This scanning process really slows down the performance of the whole copy process. Especially if the destination is a mounted drive, like NFS, the close() function takes even much longer to run.

This is a known issue but unfortunately, there is no solution so far.

Upvotes: 0

Related Questions