CODEWITHSUNDEEP
javascriptnode.jselectron
SRR
SRR

Reputation: 1748

Reduce time to copy folder using NodeJs

I currently have a folder that has two subfolders and about 500 files, totalling ~3GB. In my Electron application I perform a backup of that folder by essentially copy and pasting it to a different drive through NodeJS. However, this process takes about 3 mins when I copy and paste via Windows but around 15 mins when I use NodeJS.

I am using fs-extra to copy the whole folder over:

      const path = require('path');
      const fs = require('fs-extra');
      
      var source = 'some/source/directory'
      var destination = path.join('my/backup/directory','name_of_folder_after_backup')
    
      // copy source folder to destination
      await fs.copy(source, destination);

I did consider making this an asynchronous operation with just fs.writeFile() but considering this is running on an HDD at worst, I figured firing 500 write operations and waiting for it to complete might not be the best idea.

My questions are:

  1. Is the the proper way to use NodeJS to do a folder copy operation like this?
  2. Why is the Windows copy operation so much faster than the NodeJS copy operation?
  3. How to reduce the copy time of the folder? Is there some native Electron API I might be missing?

I'm not an expert in this area so I'd appreciate if your answer had some sources I could look at or a bit of a detailed explanation.

Upvotes: 1

Views: 578

Answers (1)

SRR
SRR

Reputation: 1748

Thanks to the comments above I got a solution. I used robocopy on windows to do the copy. Node spawns a child process, launches Robocopy, wraps it in a promise and then awaits it.

    const path = require("path");
    const util = require("util");
    const exec = util.promisify(require("child_process").exec);
    const source = 'your/source/dir/here';
    const destination = path.join('temp', "PDES-Backup");
    // copy source folder to destination
    //await fs.copySync(source, destination);
    const copyCommand = `Robocopy "${source}" "${destination}" /MIR /LOG:backuplog-${new Date()
      .toISOString()
      .substr(0, 10)}-${new Date().getTime()}.txt`;
    const promise = exec(copyCommand, { stdio: "inherit" });
    const child = promise.child;
    child.stdout.on("data", (data) => {
      console.log("stdout: " + data);
    });
    child.stderr.on("data", (data) => {
      console.log("stderr: " + data);
    });
    child.on("error", (error) => {
      console.error(error);
    });
    child.on("close", (code) => {
      console.warn("closing code: " + code);
    });
    // i.e. can then await for promisified exec call to complete
    const { stdout, stderr } = await promise;

For some reason the first copy gives the wrong closing code but the copy gets completed just fine

Upvotes: 2

Related Questions