CODEWITHSUNDEEP
python-3.xflasksocket.ioflask-socketio
magnoz
magnoz

Reputation: 1995

Flask-SocketIO - How to emit an event from a sub-process

I have a Flask app which upon certain rest call is running several modules using a ProcessPoolExecutor.

UPDATED: Added redis as a message queue (using docker, redis as redis's host)

socketio = SocketIO(app, message_queue='redis://redis')

(...)

def emit_event(evt, message):
    socketio.emit(evt, message, namespace='/test')

@app.route('/info', methods=['GET'])
def info():
    emit_event('update_reports', '')

(...)
if __name__ == "__main__":
    socketio.run(host='0.0.0.0', threaded=True)

Now that I added redis, it still works when emitting from the main app. Here some from the code I'm running the sub-process:

def __init__(self):
    self.executor = futures.ProcessPoolExecutor(max_workers=4)
    self.socketio = SocketIO(async_mode='eventlet', message_queue='redis://redis')

    (...)
    future = self.executor.submit(process, params)
    future.add_done_callback(functools.partial(self.finished_callback, pid))

Then in that callback I'm calling the emit_event method:

def finished_callback(self, pid, future):
    pid.status = Status.DONE.value
    pid.finished_at = datetime.datetime.utcnow
    pid.save()

    self.socketio.emit('update_reports', 'done', namespace='/test')

Getting and sending/ emitting messages from/to the client from my controller works just fine, also if I call /info from curl or postman my client gets the message -but- when trying to emit an event same way from within this subprocess callback, now it shows this error:

This is mostly for notifications, like notifying when a long process has finished and stuff like that.

INFO:socketio:emitting event "update_reports" to all [/test] ERROR:socketio:Cannot publish to redis... retrying ERROR:socketio:Cannot publish to redis... giving up

What I'm doing wrong?

Thanks!

Upvotes: 1

Views: 4759

Answers (1)

Miguel Grinberg
Miguel Grinberg

Reputation: 67492

There are specific rules that you need to follow in setting up the Flask-SocketIO extension so that external processes can emit, which include the use of a message queue that the main and external processes use to coordinate efforts. See the Emitting from an External Process section of the documentation for instructions.

Upvotes: 1

Related Questions