Golang ZeroMQ: REQ/REP senseless non-blocking

Question

On Python, the ZeroMQ .recv()/.send() operations are blocking, which is just perfect for REQ/REP.
In Golang, I must pass a zmq.DONTWAIT to the .recv() and .send() operation in order to make it work.

But the thing is, the flow needs to be lock step, so:

1. server.recv()
2. client.send()
3. client.recv()
4. server.send()

And between 3 and 4 the weirdness starts, because they are async.

When the client has sent a message and the server has not received it yet but client tries to receive a response, the lock step is no lock step any more.

Is there some kind of zmq.DOBLOCK in contrast to zmq.DONTWAIT?

Or did I get something wrong here?

---

EDIT:

I am using this go binding in C for zeromq: https://godoc.org/github.com/pebbe/zmq4#Type

As you can see here the .recv() needs a input flag, which is one of the both on the second ref:

Recv: https://godoc.org/github.com/pebbe/zmq4#Socket.Recv

Flags to be passed: https://github.com/pebbe/zmq4/blob/master/zmq4.go#L403

This is the current code I got to make a workaround which feels somewhat ugly:

package connection

import (
  "zmq4"
  "fmt"
  "time"
)


const ERRTMPUNAV="resource temporarily unavailable"


func checkError(e error){
  if e != nil {
    panic(e)
  }
}


func CreateRepNode(address string,onMessage chan<- string,send <-chan string,closeConn <-chan bool){
  stop:=false
  socket,err:=zmq4.NewSocket(zmq4.REP)
  checkError(err)
  err=socket.Bind(address)
  checkError(err)
  go func(socket *zmq4.Socket){
    for {
      msg,err:=socket.Recv(zmq4.DONTWAIT)
      fmt.Println("server message"+msg)
      if stop==true {
        return
      }
      if err != nil {
        rateLimit := time.Tick(100 * time.Millisecond)
    <-rateLimit
    continue
      }
      checkError(err)
      onMessage<-msg
      rep:=<-send
      _,err=socket.Send(rep,zmq4.DONTWAIT)
    }
  }(socket)
  <-closeConn
  stop=true
}


func CreateReqNode(address string,onMessage chan<- string,send <-chan string,closeConn <-chan bool){
  stop:=false
  socket,err:=zmq4.NewSocket(zmq4.REQ)
  checkError(err)
  err=socket.Connect(address)
  checkError(err)
  go func(){
    for {
      msg:=<-send
      if stop==true {
        return
      }
      _,err:=socket.Send(msg,zmq4.DONTWAIT)
      for {
        msg,err=socket.Recv(zmq4.DONTWAIT)
        fmt.Println("client got message "+msg)
        if err!=nil {
          if err.Error()==ERRTMPUNAV {
            w:=time.Tick(100*time.Millisecond)
            <-w
            continue
          }
        }
        break
      }
      onMessage<-msg
    }
  }()

  <-closeConn
  stop=true
}

Answer 1

ZeroMQ trivial elementary archetypes are more a set of building blocks than a production-grade solution to any need.

Go-lang is a very powerful, modern language with coroutines and other smart tools for a controlled concurrency, so forgive me to state the following list of recommendations:

• avoid blocking designs wherever one can ( a non-blocking design leaves one in full control of all things as they come ... not "hanging" in any infinite/uncontrollable waiting loop, the worse in an already developed deadlock )

• avoid relying on a SLOC examples with a single, elementary type of a Formal Communication Pattern, one shall rather develop a robust survivability-handler strategy for all the cases where something may go wrong ( Loss-of-Signal in transport network, Loss-of-Message, DDoS-level of resources overloads, ... )

---

Redesign hint - do not use REQ/REP at all. Yes, never...

ZeroMQ Scaleable Formal Communication Pattern REQ/REP is fine for learning ZeroMQ, but is lethal in real production grade deployment. For details, read here.

Next think about internally unconditional patterns, alike PAIR ( though marked experimental, for some use-cases it works great ), XREQ/XREP, PUSH/PULL or some composite signalling/transport multi-socket custom-designed own pattern.

The best next step?

What I can do for your further questions right now is to direct you to see a bigger picture on this subject with more arguments, a simple signalling-plane / messaging-plane illustration and a direct link to a must-read book from Pieter HINTJENS.

The book is worth one's time and efforts. If one is serious into distributed systems design, you will love it altogether with Pieter's passion for Zero-sharing, Zero-blocking, (almost) Zero-copy et al.