CODEWITHSUNDEEP
pythonnode.jsopencvimage-processingelectron
Oğulcan Çelik
Oğulcan Çelik

Reputation: 402

How to use Electron for an image processing application with Python in backend?

I'm developing an application which is about image processing. I used OpenCV on Python and tried to implement a GUI with PyQT but using HTML, CSS, and Javascript would be a better option for my project.

I knew that Electron offers desktop apps with GUIs' implemented with HTML, CSS, and Javascript, however, I'm not sure how to connect Python to Node & Electron.

I will need to pass image arrays generated with NumPy and OpenCV operations to frontend to show in my GUI and this needs to happen in realtime. I'll read frames in OpenCV and do some processing and show them on the GUI 24 frames per second, so performance is important.

How can I achieve this task to send image array data from Python to Electron in realtime?

Upvotes: 5

Views: 3042

Answers (2)

Oğulcan Çelik
Oğulcan Çelik

Reputation: 402

I have created a super simple repository with working examples to show two different methods for communication between Python and Nodejs, using Redis and ZeroMQ. Check it out:

https://github.com/OgulcanCelik/nodejs-python-communication

original comment:

So I figured it out. What I did was calling Python script from NodeJS with python-shell NPM package, and communicate between Python and NodeJS via stdin. Python script generates an image after doing some processing with OpenCV, encodes it as png, converts it to bytes, encode it as base64 and finally decodes it as ascii to be read on Node part.

Python part is like this:

import cv2 as cv2
import base64

source = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
success, encoded_image = cv2.imencode('.png', source)
content = encoded_image.tobytes()
print(base64.b64encode(content).decode('ascii'))

NodeJS part:

pyshell.on("message", function(message) {
  console.log(message)
});

setInterval(() => {
  pyshell.send(data);
}, 300);

More info here: Can't read Base64 encoded image in Node.js which is sent from Python

Upvotes: 6

frankie567
frankie567

Reputation: 1760

Traditionally, we use sockets for Interprocess Communication (IPC). Electron itself uses a socket to communicate between the NodeJS process and the UI window. Sockets are incredibly fast, so you won't have to worry about the performance of the data transfer itself.

Python has a built-in library for sockets and for NodeJS, you have the node-ipc library (which is the one used by Electron for its internal socket).

You'll have then to define your data protocol. For example, you could send/receive JSON objects through the sockets, which is quite trivial to encode/decode both in Python and NodeJS. But if performance matters, you can do more advanced things.

You also have to think about how you manage the processes themselves. Indeed, you'll have to ensure that the Electron app will spawn the Python process when started. And make sure it kills it when quitted (for example, by sending a message through the socket!).

My answer is quite abstract, but I hope it will give you some insights to get started. It's not a trivial task ;)

Upvotes: 5

Related Questions