CODEWITHSUNDEEP
node.jsmultithreadingexpress
Stan Luo
Stan Luo

Reputation: 3889

node.js multithreading - why all my workers respond to one incoming request?

I'm using node + express as my web server, and has a cluster of 4 workers.

The problem is, an incoming request is actually responded by all the workers. Say each worker serves the same web app which has an api to log the current worker's process id, when I use a browser to access this api, all the workers' process id got logged.

Here's the code of cluster part, in index.js of express server:

if (cluster.isMaster) {
  console.log(`Master cluster setting up ${numWorkers} workers...`);

  for (let i = 0; i < numWorkers; i++) {
    cluster.fork();
  }
} else {
  // webpack configs...

  app.get('/test_connection', (req, res) => res.send('ok'));
  app.use('/js', express.static(`${__dirname}/assets/js`));
  app.get('*', (req, res) => {
    res.sendFile(path.join(__dirname, '..', 'app', 'index.html'));
  });
  app.listen(3000);
}

And, in the app's routes, there's a controller, let's say when the user clicks a button, the web redirects to some page that triggers a controller to log out current process id with:

// inside the controller
console.log(`Connected via ${process.pid}`);

The weird thing is, when user clicks on that button, turns out all 4 worker's process ids got logged out. Shouldn't that be only one worker's id which respond to the incoming request?

EDIT:

Here's my entire server root:

const express = require('express');
const path = require('path');
const mongoose = require('mongoose');
mongoose.Promise = require('bluebird');
const webpack = require('webpack');
const webpackDevMiddleware = require('webpack-dev-middleware');
const webpackHotMiddleware = require('webpack-hot-middleware');
const webpackDevConfig = require('../webpack/webpack.config.dev');

const mongodb = require('./db/mongo/constants');
const transactionDB = require('./db/postgres');
const config = require('./config/appConfig');
const app = require('./config/express');
const ENV = require('./config/appConfig').ENV;
// const compression = require('compression');
const cluster = require('cluster');
const numWorkers = require('os').cpus().length;

if (cluster.isMaster) {
  console.log(`Master cluster setting up ${numWorkers} workers...`);

  for (let i = 0; i < numWorkers; i++) {
    cluster.fork();
  }

  cluster.on('online', worker => {
    console.log(`Worker ${worker.process.pid} is online`);
  });

  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died with code ${code}, and signal: ${signal}`);
    console.log('Starting a new worker');
    cluster.fork();
  });
} else {
    const compiler = webpack(webpackDevConfig);
    app.use(webpackDevMiddleware(compiler, {
      noInfo: true,
      publicPath: webpackDevConfig.output.publicPath,
    }));
    app.use(webpackHotMiddleware(compiler));

    app.get('/test_connection', (req, res) => res.send('ok'));
    app.use('/fonts', express.static(`${__dirname}/assets/fonts`));
    app.use('/styles', express.static(`${__dirname}/assets/styles`));
    app.use('/images', express.static(`${__dirname}/assets/images`));
    app.use('/js', express.static(`${__dirname}/assets/js`));
    app.get('*', (req, res) => {
      res.sendFile(path.join(__dirname, '..', 'app', 'index.html'));
    });

  // connect to MongoDB
  mongoose.connect(mongodb[ENV].db, (err) => {
    // connect to PostgreSQL
    transactionDB.sequelize
      .sync()
      .then(() => {
        const server = app.listen(app.get('port'));
      });
  });
}

By the line const app = require('./config/express') the app will have routes settings, including the controller thing.

Btw I commented out socket.io part but the client still tries to validate token every 3 secs, don't know if that matters.

Upvotes: 0

Views: 203

Answers (1)

Mouna Apperson
Mouna Apperson

Reputation: 1307

You are forking after you have already created the server. You need to fork before you create the server. The reason for this is that on fork, each process inherits all the open files and all then listen to the same server. Place all your code and I can tell you what you need to move around.

EDIT:

This line, I believe, is what starts your server. What you'll want to do is place everything that you have at the top inside the else (not master) branch of your code.

const app = require('./config/express');

Essentially, your master component should only handle spinning up the children and managing them. Alternatively, you can use an existing drop-in to handle this. For example, express-cluster. See here: https://www.npmjs.com/package/express-cluster

And as a last note, you will want to make sure that each worker is on a different port. If they aren't, they will get port reuse errors. This is why I'd suggest using an existing library, though, where I work we did write this ourselves. Unless you have a good reason to do it yourself, use what already exists.

Upvotes: 2

Related Questions