CODEWITHSUNDEEP
pythonprotocol-buffersgrpcproto
Bagzhan Zhumash
Bagzhan Zhumash

Reputation: 11

Redirectional grpc server with two different proto files

I'm currently working on a grpc server that will receive streaming grpc calls from the first server and redirect these calls to the second server, and redirect responses from the second server as streams to the first one.

I have 2 proto files

First file:

    syntax = "proto3";

package first.proto.pack;

service FirstProtoService {
  rpc StreamingCall(stream RequestToFirstServer) returns (stream ResponseForFirstServer){}
}

message RequestToFirstServer {
    oneof firstStreamingRequest {
        int32 x = 1;
        int32 y = 2;
    }
}

message ResponseForFirstServer {
  string someprocessedinformation = 1;
}

Second file:

syntax = "proto3";

package second.proto.pack;

service SecondProtoService {
  rpc StreamingCall(stream RequestToSecondServer) returns (stream ResponseFromSecondServer){}
}

message RequestToSecondServer {
  oneof secondStreamingRequest {
    int32 processedX = 1;
    int32 processedY = 2;
  }
}

message ResponseFromSecondServer {
  string computedInformation = 1;
}

First server knows about first proto file but doesn't know about second.

Second server knows about second proto file but doesn't know about first.

Middle server knows about first and second proto.

Need to write a server that will transmit requests from one server from one server to another

I started writing it on Python. But faced the problem of sending to much requests to second server

That how my service middle implementation looks on Python:

import grpc
import socket
import logging
from concurrent import futures

import first_pb2 as fasr
import first_pb2_grpc as fasr_srv

import second_pb2 as sasr
import second_pb2_grpc as sasr_srv


s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(("8.8.8.8", 80))
ip_address = s.getsockname()[0]
s.close()


_THREAD_CONCURRENCY = 24
channel = grpc.insecure_channel('localhost:50051')
client = sasr_srv.SecondProtoServiceStub(channel)


class FirstProtoService(fasr_srv.FirstProtoServiceServicer):
    def StreamingCall(self, request_iterator: fasr.RequestToFirstServer, context):
        try:
            for request in request_iterator:
                if request.x:
                    request = sasr.RequestToSecondServer(processedX=request.processedX)
                if request.y:
                    request = sasr.RequestToSecondServer(processedY=request.processedY)
                responses = client.StreamingCall(request)
                yield responses
        except StopIteration as error:
            logging.error(f'Exception occurred {error}.', exc_info=True)
            raise RuntimeError("Failed to receive request")
        except Exception as e:
            logging.error(f'-------Exception occurred: {e}.', exc_info=True)


def serve(host: str, port: str) -> None:
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=_THREAD_CONCURRENCY))
    fasr_srv.add_FirstProtoServiceServicer_to_server(FirstProtoService(), server)
    server.add_insecure_port(f'{host}:{port}')
    server.start()
    logging.info(f'Starting server on : {host}:{port}')
    server.wait_for_termination()


if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    serve(ip_address, '7777')

After a request from the first client on the side of the middle server, I get the following errors:

line 29, in StreamingCall
    request = sasr.RequestToSecondServer(processedX=request.processedX)
TypeError: Parameter to MergeFrom() must be instance of same class: expected second.proto.pack.processedX got first.proto.pack.x

Upvotes: 1

Views: 598

Answers (1)

Lidi Zheng
Lidi Zheng

Reputation: 2091

TypeError: Parameter to MergeFrom() must be instance of same class: expected second.proto.pack.processedX got first.proto.pack.x

The error message from ProtoBuf suggests the fix. You probably want to use a different variable name than request. The generated ProtoBuf code might override the object assign behavior.

Upvotes: 2

Related Questions