CODEWITHSUNDEEP
pythonmultiprocessingzeromqgetattrzerorpc
RayChang
RayChang

Reputation: 13

python zerorpc client with multiprocess to send log to server

As the title, I want a log rpc server to help me deal with the log in a new process.

I reference zerorpc and this link: https://stackoverflow.com/a/57140017/14021161

But stuck in a weird bug

The code:

import zerorpc
from loguru import logger
import time
import multiprocessing as mp
import collections

Job = collections.namedtuple('Job', ['event', 'args'])

class LogClient(mp.Process):
    """A process backed by an internal queue for simple one-way message passing.
    """
    def __init__(self, ip='127.0.0.1', port='4242'):
        super().__init__()
        self.queue = mp.Queue()
        self.c = zerorpc.Client()
        address = f"tcp://{ip}:{port}"
        self.c.connect(address)
        self.start()

    def put(self, event, *args):
        """Puts the event and args as a `Job` on the queue
        """
        job = Job(event, args)
        self.queue.put(job)

    def _addLog(self, job):
        event, args = job
        self.c.addLog(*args)

    def run(self):
        while True:
            job = self.queue.get()
            self._addLog(job)

    def addLog(self, level, *context):
        self.put('addLog', level, *context)
    


lc = LogClient()
lc.c.addLog("WARNING", 'hello', 'Ray')  # this work, but its witout multiprocess
lc.addLog("INFO", 'hello', 'Ray')  # doesn't work 
lc.terminate()

The _addLog indeed receive the job but doesn't really send it to server

Any advice would be really appreciated! Thanks in advance.

|

|

p.s. To make the question clear, I only show the addLog, but actually the server should handle many IO jobs. So the _addLog func will turn into a dispatch func with gettattr to parse all the functions in server

Upvotes: 0

Views: 395

Answers (1)

RayChang
RayChang

Reputation: 13

I find out that problem come out with zerorpc.

After trying grpc which is fast but so many details against my light project...

I end up using pure zeromq with a PUSH-PULL pattern.

Upvotes: 0

Related Questions