How to communicate between node.js process and a python process

Question

I have a node.js process and a python process running in parallel. Node.js process is a http server. Whenever I get a request I want to call a callable object (function/ or any callable) in the python process and pass some environment variable and a callable funtion from the node.js code. In psudo code it is something like

// node.js process
http.createServer(function (req, res) {
  function start_response(status_code, headers){
    // closure function that sets headers
    res.writeHead(status_code);
  }
  env = {
 'SERVER_PROTOCOL': 'HTTP/1.1',
 'HTTP_HOST': '127.0.0.1:8000',
 'HTTP_CONNECTION': 'keep-alive',
  }
  // want to call the app function in the python file likr
  // response = app(env, start_response)
  // and finally
  res.write(response)
  res.end()
})

# python process
def app(env, start_response):
    # do something
    start_response(200)
    return "hello"

What is the way to communicate between these two processes. I can spawn a python process from node.js using childprocess but how to pass the javascript function to python? How can this problem be solved? Thanks in advance.

Answer 1

You can use Named Pipes. The goal is to use Named Pipes for communication.

1. A Node.js process will write some data to a pipe (A)
2. A Python process reads from pipe (A) and manipulates the data
3. Above python process then writes the data into a pipe (B)
4. Node.js reads from pipe (B)

Python: read, process and write

First up, let’s create a simple Python process that reads from a named pipe A, processes the data and then writes it to named pipe B (for simplicity, the process_msg() function here returns the read data). The script first creates the named pipe A using os.mkfifo() command.

import os
import select

IPC_FIFO_NAME_A = "pipe_a"
IPC_FIFO_NAME_B = "pipe_b"

def get_message(fifo):
    '''Read n bytes from pipe. Note: n=24 is an example'''
    return os.read(fifo, 24)

def process_msg(msg):
    '''Process message read from pipe'''
    return msg

if __name__ == "__main__":
    os.mkfifo(IPC_FIFO_NAME_A)  # Create Pipe A

    try:
        fifo_a = os.open(IPC_FIFO_NAME_A, os.O_RDONLY | os.O_NONBLOCK)  # pipe is opened as read only and in a non-blocking mode
        print('Pipe A ready')

        while True:
            try:
                fifo_b = os.open(IPC_FIFO_NAME_B, os.O_WRONLY)
                print("Pipe B ready")
                break
            except:
                # Wait until Pipe B has been initialized
                pass

        try:
            poll = select.poll()
            poll.register(fifo_a, select.POLLIN)

            try:
                while True:
                    if (fifo_a, select.POLLIN) in poll.poll(1000):  # Poll every 1 sec
                        msg = get_message(fifo_a)                   # Read from Pipe A
                        msg = process_msg(msg)                      # Process Message
                        os.write(fifo_b, msg)                       # Write to Pipe B

                        print('----- Received from JS -----')
                        print("    " + msg.decode("utf-8"))
            finally:
                poll.unregister(fifo_a)
        finally:
            os.close(fifo_a)
    finally:
        os.remove(IPC_FIFO_NAME_A)
        os.remove(IPC_FIFO_NAME_B)

Node.js: write and read:

Next, let’s write a simple Node.js program that writes into the pipe. There are a few packages which do this: named-pipe, fifo-js. However, node offers functions which makes it convenient to set up IPC with pipes. The script below writes data every 1 second into Pipe A (send data), and reads from Pipe B (receive processed data from Python). For simplicity, the data here is the current time as string.

const fs              = require('fs');
const { spawn, fork } = require('child_process');

const path_a = 'pipe_a';
const path_b = 'pipe_b';
let fifo_b   = spawn('mkfifo', [path_b]);  // Create Pipe B

fifo_b.on('exit', function(status) {
    console.log('Created Pipe B');

    const fd   = fs.openSync(path_b, 'r+');
    let fifoRs = fs.createReadStream(null, { fd });
    let fifoWs = fs.createWriteStream(path_a);

    console.log('Ready to write')

    setInterval(() => {
        console.log('-----   Send packet   -----');
        fifoWs.write(`${new Date().toISOString()}`);
    }, 1000);  // Write data at 1 second interval

    fifoRs.on('data', data => {

        now_time  = new Date();
        sent_time = new Date(data.toString());
        latency   = (now_time - sent_time);

        console.log('----- Received packet -----');
        console.log('    Date   : ' + data.toString());
        console.log('    Latency: ' + latency.toString() + ' ms');
    });
});

Note: This content is taken from here, you can check it out for more details.