CODEWITHSUNDEEP
javascriptnode.jsbuild
Andre Pena
Andre Pena

Reputation: 59336

How can I run multiple npm scripts in parallel?

In my package.json I have these two scripts:

  "scripts": {
    "start-watch": "nodemon run-babel index.js",
    "wp-server": "webpack-dev-server",
  }

I have to run these 2 scripts in parallel everytime I start developing in Node.js. The first thing I thought of was adding a third script like this:

"dev": "npm run start-watch && npm run wp-server"

... but that will wait for start-watch to finish before running wp-server.

How can I run these in parallel? Please keep in mind that I need to see the output of these commands. Also, if your solution involves a build tool, I'd rather use gulp instead of grunt because I already use it in another project.

Upvotes: 959

Views: 814794

Answers (30)

bhavesh
bhavesh

Reputation: 755

There are many ways but the most common 2 ways are here

  1. by command
    "scripts": {
        "start": "react-scripts start",
        "dev": "(cd server && npm run start) & npm run start"
      }
  1. by npm package
  • concurrently npm
  • npm-run-all npm (abandoned)
  • npm-run-all2 npm (maintenance fork of the above)

Upvotes: 9

fender0ne
fender0ne

Reputation: 311

Native solution in VScode using parallel tasks:

Ejecute it directly from the task manager F1 -> Task: Run Task -> Development and it will run both tasks in two separated shells in parallel using your scripts.

package.json

"scripts": {
    "start-watch": "nodemon run-babel index.js",
    "wp-server": "webpack-dev-server",
  }

.vscode\tasks.json

See the line with the following: "dependsOrder": "parallel"
Note: works in vscode in windows and linux

{
    "version": "2.0.0",
    "tasks": [
        {
          "label": "start-watch in package.json",
          "type": "npm",
          "script": "start-watch",
          "presentation": {
            "clear": true,
            "reveal": "silent",
            "showReuseMessage": true
          }
        },
        {
          "label": "wp-server in package.json",
          "type": "npm",
          "script": "wp-server",
          "presentation": {
            "clear": true,
            "reveal": "silent",
            "showReuseMessage": false
          }
        },
        {
          "label": "Development",
          "dependsOrder": "parallel",
          "dependsOn": ["start-watch in package.json", "wp-server in package.json"]
        },
        {
          "label": "Terminate All Tasks",
          "type": "shell",
          "command": "echo ${input:terminate}",
          "problemMatcher": []
        },
    ]
}

The only problem here is that unlike concurrently which runs the commands in the same shell where you run the concurrently command and you can kill both processes (ctrl+C) at the same time, with task they run in two different shells inside the VSCode terminal and can only be terminated separately or we could define another task that will terminate all the open tasks (see the task called Terminate All Task)

Upvotes: 2

Lou Zell
Lou Zell

Reputation: 5597

I had a similar need, and found using GNU parallel the simplest. My package.json looks like this:

"scripts": {
  "foo": "<invoke_foo>",
  "bar": "<invoke_bar>",
  "both": "parallel --ungroup 'npm run' ::: foo bar"
},

Then run both foo and bar with npm run both

Upvotes: 2

Edward Brey
Edward Brey

Reputation: 41638

Use concurrently to run the commands in parallel with a shared output stream. To make it easy to tell which output is from which process, use the shortened command form, such as npm:wp-server. This causes concurrently to prefix each output line with its command name.

In package.json, your scripts section will look like this:

 "scripts": {
    "start": "concurrently \"npm:start-watch\" \"npm:wp-server\"",
    "start-watch": "nodemon run-babel index.js",
    "wp-server": "webpack-dev-server"
  }

Upvotes: 16

Yilmaz
Yilmaz

Reputation: 49182

with installing npm install concurrently

"scripts": {
    "start:build": "tsc -w",
    "start:run": "nodemon build/index.js",
    "start": "concurrently  npm:start:*"
  },

Upvotes: 14

Jamal
Jamal

Reputation: 848

I think the best way is to use npm-run-all as below:

1- npm install -g npm-run-all <--- will be installed globally
2- npm-run-all --parallel server client

Upvotes: 3

dcsan
dcsan

Reputation: 12275

How about a good old fashioned Makefile?

This allows you a lot of control including how you manage subshells, dependencies between scripts etc.


# run both scripts
start: server client

# start server and use & to background it
server:
    npm run serve &

# start the client
client:
    npm start

call this Makefile and then you can just type

make start to start everything up. Because the server command is actually running in a child process of the start command when you ctrl-C the server command will also stop - unlike if you just backgrounded it yourself at the shell. Make also gives you command line completion, at least on the shell i'm using. Bonus - the first command will always run so you can actually just type make on it's own here.

I always throw a makefile into my projects, just so I can quickly scan later all the common commands and parameters for each project as I flip between them.

Upvotes: 4

ZiiMakc
ZiiMakc

Reputation: 36786

As you may need to add more and more to this scripts it will become messy and harder to use. What if you need some conditions to check, variables to use? So I suggest you to look at google/zx that allows to use js to create scripts.

Simple usage:

  1. install zx: npm i -g zx
  2. add package.json commands (optional, you can move everything to scripts):
  "scripts": {
    "dev": "zx ./scripts/dev.mjs", // run script
    "build:dev": "tsc -w", // compile in watch mode
    "build": "tsc", // compile
    "start": "node dist/index.js", // run
    "start:dev": "nodemon dist/index.js", // run in watch mode
  },
  1. create dev.mjs script file:
#!/usr/bin/env zx

await $`yarn build`; // prebuild if dist is empty
await Promise.all([$`yarn start:dev`, $`yarn build:dev`]); // run in parallel

Now every time you want to start a dev server you just run yarn dev or npm run dev.

It will first compile ts->js and then run typescrpt compiler and server in watch mode in parallel. When you change your ts file->it's will be recompiled by tsc->nodemon will restart the server.

Advanced programmatic usage

Load env variables, compile ts in watch mode and rerun server from dist on changes (dev.mjs):

#!/usr/bin/env zx
import nodemon from "nodemon";
import dotenv from "dotenv";
import path from "path";
import { fileURLToPath } from "url";

// load env variables
loadEnvVariables("../env/.env");

await Promise.all([
  // compile in watch mode (will recompile on changes in .ts files)
  $`tsc -w`,
  // wait for tsc to compile for first time and rerun server on any changes (tsc emited .js files)
  sleep(4000).then(() =>
    nodemon({
      script: "dist/index.js",
    })
  ),
]);

function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
}

function getDirname() {
  return path.dirname(fileURLToPath(import.meta.url));
}

function loadEnvVariables(relativePath) {
  const { error, parsed } = dotenv.config({
    path: path.join(getDirname(), relativePath),
  });

  if (error) {
    throw error;
  }

  return parsed;
}

Upvotes: 0

Neil Kistner
Neil Kistner

Reputation: 12333

Use a package called concurrently.

npm i concurrently --save-dev

Then setup your npm run dev task as so:

"dev": "concurrently --kill-others \"npm run start-watch\" \"npm run wp-server\""

Upvotes: 1052

AlexRMU
AlexRMU

Reputation: 67

A simple and native way for Windows CMD

"start /b npm run bg-task1 && start /b npm run bg-task2 && npm run main-task"

(start /b means start in the background)

Upvotes: 2

Peter M. Elias
Peter M. Elias

Reputation: 1194

Using just shell scripting, on Linux.

"scripts": {
  "cmd": "{ trap 'trap \" \" TERM; kill 0; wait' INT TERM; } && blocking1 & blocking2 & wait"
}

npm run cmd and then ^C will kill children and wait for clean exit.

Upvotes: 0

woutvdd
woutvdd

Reputation: 821

You can also use pre and post as prefixes on your specific script.

  "scripts": {
    "predev": "nodemon run-babel index.js &",
    "dev": "webpack-dev-server"
  }

And then run: npm run dev

Upvotes: 3

Muhammad Muzamil
Muhammad Muzamil

Reputation: 1242

step by step guide to run multiple parallel scripts with npm. install npm-run-all package globally

npm i -g npm-run-all

Now install and save this package within project where your package.json exists

npm i npm-run-all --save-dev

Now modify scripts in package.json file this way

"scripts": {
    "server": "live-server index.html",
    "watch": "node-sass scss/style.scss --watch",
    "all": "npm-run-all --parallel server watch"
},

now run this command

npm run all

more detail about this package in given link npm-run-all

Upvotes: 7

nick
nick

Reputation: 678

This worked for me

{
"start-express": "tsc && nodemon dist/server/server.js",
"start-react": "react-scripts start",
"start-both": "npm -p -r run start-react && -p -r npm run start-express"
}

Both client and server are written in typescript.

The React app is created with create-react-app with the typescript template and is in the default src directory.

Express is in the server directory and the entry file is server.js

typescript code and transpiled into js and is put in the dist directory .

checkout my project for more info: https://github.com/nickjohngray/staticbackeditor

UPDATE: calling npm run dev, to start things off

{"server": "tsc-watch --onSuccess \"node ./dist/server/index.js\"",
"start-server-dev": "npm run build-server-dev && node src/server/index.js",
"client": "webpack-dev-server --mode development --devtool inline-source-map --hot",
"dev": "concurrently \"npm run build-server-dev\"  \"npm run server\" \"npm run client\""}

Upvotes: 3

Mosia Thabo
Mosia Thabo

Reputation: 4267

... but that will wait for start-watch to finish before running wp-server.

For that to work, you will have to use start on your command. Others have already illustrated but this is how it will work, your code below:

"dev": "npm run start-watch && npm run wp-server"

Should be :

"dev": " start npm run start-watch && start npm run wp-server"

What this will do is, it will open a separate instance for each command and process them concurrently, which shouldn't be an issue as far as your initial issue is concerned. Why do I say so? It's because these instances both open automatically while you run only 1 statement, which is your initial goal.

Upvotes: 4

Boaz
Boaz

Reputation: 20220

How about forking

Another option to run multiple Node scripts is with a single Node script, which can fork many others. Forking is supported natively in Node, so it adds no dependencies and is cross-platform.

Minimal example

This would just run the scripts as-is and assume they're located in the parent script's directory.

// fork-minimal.js - run with: node fork-minimal.js

const childProcess = require('child_process');

let scripts = ['some-script.js', 'some-other-script.js'];
scripts.forEach(script => childProcess.fork(script));

Verbose example

This would run the scripts with arguments and configured by the many available options.

// fork-verbose.js - run with: node fork-verbose.js

const childProcess = require('child_process');

let scripts = [
    {
        path: 'some-script.js',
        args: ['-some_arg', '/some_other_arg'],
        options: {cwd: './', env: {NODE_ENV: 'development'}}
    },    
    {
        path: 'some-other-script.js',
        args: ['-another_arg', '/yet_other_arg'],
        options: {cwd: '/some/where/else', env: {NODE_ENV: 'development'}}
    }
];

let runningScripts= [];

scripts.forEach(script => {
    let runningScript = childProcess.fork(script.path, script.args, script.options);

   // Optionally attach event listeners to the script
   runningScript.on('close', () => console.log('Time to die...'))

    runningScripts.push(runningScript); // Keep a reference to the script for later use
});

Communicating with forked scripts

Forking also has the added benefit that the parent script can receive events from the forked child processes as well as send back. A common example is for the parent script to kill its forked children.

 runningScripts.forEach(runningScript => runningScript.kill());

For more available events and methods see the ChildProcess documentation

Upvotes: 14

Tiny Brain
Tiny Brain

Reputation: 640

Just add this npm script to the package.json file in the root folder.

{
  ...
  "scripts": {
    ...
    "start": "react-scripts start", // or whatever else depends on your project
    "dev": "(cd server && npm run start) & (cd ../client && npm run start)"
  }
}

Upvotes: 4

SB3NDER
SB3NDER

Reputation: 63

In a package.json in the parent folder:

"dev": "(cd api && start npm run start) & (cd ../client && start npm run start)"

this work in windows

Upvotes: 4

Artem Belik
Artem Belik

Reputation: 492

npm install npm-run-all --save-dev

package.json:

"scripts": {
  "start-watch": "...",
  "wp-server": "...",
  "dev": "npm-run-all --parallel start-watch wp-server"
}

More info: https://github.com/mysticatea/npm-run-all/blob/master/docs/npm-run-all.md

Upvotes: 4

Darkowic
Darkowic

Reputation: 690

I've checked almost all solutions from above and only with npm-run-all I was able to solve all problems. Main advantage over all other solution is an ability to run script with arguments.

{
  "test:static-server": "cross-env NODE_ENV=test node server/testsServer.js",
  "test:jest": "cross-env NODE_ENV=test jest",
  "test": "run-p test:static-server \"test:jest -- {*}\" --",
  "test:coverage": "npm run test -- --coverage",
  "test:watch": "npm run test -- --watchAll",
}

Note run-p is shortcut for npm-run-all --parallel

This allows me to run command with arguments like npm run test:watch -- Something.

EDIT:

There is one more useful option for npm-run-all:

 -r, --race   - - - - - - - Set the flag to kill all tasks when a task
                            finished with zero. This option is valid only
                            with 'parallel' option.

Add -r to your npm-run-all script to kill all processes when one finished with code 0. This is especially useful when you run a HTTP server and another script that use the server.

  "test": "run-p -r test:static-server \"test:jest -- {*}\" --",

Upvotes: 35

user8094098
user8094098

Reputation:

My solution is similar to Piittis', though I had some problems using Windows. So I had to validate for win32.

const { spawn } = require("child_process");

function logData(data) {
    console.info(`stdout: ${data}`);
}

function runProcess(target) {
    let command = "npm";
    if (process.platform === "win32") {
        command = "npm.cmd"; // I shit you not
    }
    const myProcess = spawn(command, ["run", target]); // npm run server

    myProcess.stdout.on("data", logData);
    myProcess.stderr.on("data", logData);
}

(() => {
    runProcess("server"); // package json script
    runProcess("client");
})();

Upvotes: 4

Alexandre
Alexandre

Reputation: 3170

I have been using npm-run-all for some time, but I never got along with it, because the output of the command in watch mode doesn't work well together. For example, if I start create-react-app and jest in watch mode, I will only be able to see the output from the last command I ran. So most of the time, I was running all my commands manually...

This is why, I implement my own lib, run-screen. It still very young project (from yesterday :p ) but it might be worth to look at it, in your case it would be:

run-screen "npm run start-watch" "npm run wp-server"

Then you press the numeric key 1 to see the output of wp-server and press 0 to see the output of start-watch.

Upvotes: 3

Piittis
Piittis

Reputation: 318

Simple node script to get you going without too much hassle. Using readline to combine outputs so the lines don't get mangled.

const { spawn } = require('child_process');
const readline = require('readline');

[
  spawn('npm', ['run', 'start-watch']),
  spawn('npm', ['run', 'wp-server'])
].forEach(child => {
    readline.createInterface({
        input: child.stdout
    }).on('line', console.log);

    readline.createInterface({
        input: child.stderr,
    }).on('line', console.log);
});

Upvotes: 2

nir
nir

Reputation: 3140

You should use npm-run-all (or concurrently, parallelshell), because it has more control over starting and killing commands. The operators &, | are bad ideas because you'll need to manually stop it after all tests are finished.

This is an example for protractor testing through npm:

scripts: {
  "webdriver-start": "./node_modules/protractor/bin/webdriver-manager update && ./node_modules/protractor/bin/webdriver-manager start",
  "protractor": "./node_modules/protractor/bin/protractor ./tests/protractor.conf.js",
  "http-server": "./node_modules/http-server/bin/http-server -a localhost -p 8000",
  "test": "npm-run-all -p -r webdriver-start http-server protractor"
}

-p = Run commands in parallel.

-r = Kill all commands when one of them finishes with an exit code of zero.

Running npm run test will start Selenium driver, start http server (to serve you files) and run protractor tests. Once all tests are finished, it will close the http server and the selenium driver.

Upvotes: 106

Vikash Mishra
Vikash Mishra

Reputation: 635

In my case I have two projects, one was UI and the other was API, and both have their own script in their respective package.json files.

So, here is what I did.

npm run --prefix react start&  npm run --prefix express start&

Upvotes: 2

Entity Black
Entity Black

Reputation: 3491

I have a crossplatform solution without any additional modules. I was looking for something like a try catch block I could use both in the cmd.exe and in the bash.

The solution is command1 || command2 which seems to work in both enviroments same. So the solution for the OP is:

"scripts": {
  "start-watch": "nodemon run-babel index.js",
  "wp-server": "webpack-dev-server",
  // first command is for the cmd.exe, second one is for the bash
  "dev": "(start npm run start-watch && start npm run wp-server) || (npm run start-watch & npm run wp-server)",
  "start": "npm run dev"
}

Then simple npm start (and npm run dev) will work on all platforms!

Upvotes: 23

koders
koders

Reputation: 5794

npm-run-all --parallel task1 task2

edit:

You need to have npm-run-all installed beforehand. Also check this page for other usage scenarios.

Upvotes: 9

Neil Girardi
Neil Girardi

Reputation: 4913

If you replace the double ampersand with a single ampersand, the scripts will run concurrently.

Upvotes: 15

Oleg
Oleg

Reputation: 24988

From windows cmd you can use start:

"dev": "start npm run start-watch && start npm run wp-server"

Every command launched this way starts in its own window.

Upvotes: 130

ian
ian

Reputation: 168

I ran into problems with & and |, which exit statuses and error throwing, respectively.

Other solutions want to run any task with a given name, like npm-run-all, which wasn't my use case.

So I created npm-run-parallel that runs npm scripts asynchronously and reports back when they're done.

So, for your scripts, it'd be:

npm-run-parallel wp-server start-watch

Upvotes: 3

Related Questions