CODEWITHSUNDEEP
pythonhttparchitectureelectronzerorpc
Varun Mathur
Varun Mathur

Reputation: 918

Architecture recommendation for cancellable backend calls [zerorpc if possible?]

I have an electron app frontend (zerorpc-node) communicating with a python backend (zerorpc-python) that needs to:

a) be able to send requests to the backend [standard zerorpc call]

b) be able to run multiple backend processes simultaneously [followed the architecture in https://github.com/0rpc/zerorpc-node/issues/96)

c) be able to cancel a backend process at will [not sure how to do this with current architecture]

Any guidance on how to architecture a solution to (c) would be great. If necessary I am willing to switch away from zerorpc if it is limiting, but if the solution involves using zerorpc that's fantastic.

Upvotes: 1

Views: 81

Answers (1)

Varun Mathur
Varun Mathur

Reputation: 918

I ended up using gipc to spin up processes. The cancellation mechanism relies on the fact that when a gipc process is terminated, the pipe closes. The entire API is complicated, this is what I ended up with:

class ZerorpcService():
    def __init__(self):
        self.participant_id = None
        self.extraction_methods = []
        # maps pid to (process, pipe writer)
        self.processes = {}
        self. = lock.Semaphore()

    def _launch_process(self, function, kwargs):
        """
        Launches a new process
        """
        try:
            # add required arguments
            pid = kwargs["pid"]

            # start independent gipc process, communicated via pipe
            started = False
            with gipc.pipe() as (r, w):
                with self.mutex:
                    if pid in self.processes:
                        return_value = {'status': 1, 'error': 'pid already exists', "report": True}
                        return
                    proc = gipc.start_process(self._process_wrapper, args=(function, kwargs, w))
                    self.processes[pid] = proc
                    started = True
                # wait for process to send something over pipe
                return_value = r.get()
        except EOFError as eof:
            # happens when we terminate a process because the pipe closes
            return_value = {'status': 1, 'error': "pid {} terminated".format(pid), "report": False}
        except Exception as error:
            logging.exception(error)
            return_value = {'status': 1, 'error': str(error), 'traceback': traceback.format_exc(), "report": True}
        finally:
            # deletes the pid from the map
            with self.mutex:
                if started:
                    del self.processes[pid]
            return return_value

    @staticmethod
    def _process_wrapper(function, kwargs, pipe):
        """
        Executes f with kwargs and formats the result into a dict.
        Wraps it in error handling.
        Routes the return value through the pipe provided.
        """
        return_val = {'status': 0}
        try:
            raw_val = function(**kwargs)
            if raw_val is not None:
                return_val = raw_val
        except Exception as error:
            logging.exception(error)
            return_val = {'status': 1, 'error': str(error), 'traceback': traceback.format_exc(), "report": True}
        finally:
            pipe.put(return_val)

    def cancel_process(self, pid):
        if pid in self.processes:
            with self.mutex:
                process = self.processes[pid]
                if process.is_alive():
                    process.terminate()
                return {'status': 0}
        else:
            return {'status': 1, 'error': 'pid {} not found'.format(pid), "traceback": traceback.format_exc(),
                    "report": True}

Upvotes: 0

Related Questions