CODEWITHSUNDEEP
pythonapiarchitecturegrpcproto
geoai777
geoai777

Reputation: 98

gRPC API Architecture advice

Right now I'm developing microservice application in Python.

Basically it's like GUI -(gRPC)-> Server -(gRPC?)-> Workers.

There are various calls that GUI makes towards Server, for example all configs are stored on server. I decided to use #gRPC as API transport for performance and simplicity.

I've already implemented client and server classes and now I have a dilemma:

  1. Performance best practices tell us to Always re-use stubs and channels when possible. For that reason I can make universal rpc channel like:
UniMsg {
  int32 id = 1;
  string name = 2;
  bytes data = 3;
}

Pack data on client side
(in some cases, it's impossible to make full data schema in proto. For example config is written as JSON to transfer it to GUI I either have to implement all config fields in proto, which seems a little like headache to me, or I can encode JSON to bytes and send it that way. It will not be transparent, but it will allow modify config structure without need to modify/compile proto. And sending encoded bytes seems significantly better performance-wize, I don't need to serialize/deserialize it.)
, mark it with name, unpack and process on server side according to name field. This reuses stub and keeps amount of stubs required to exactly one.
Only downside is that it's kinda not RPC, it's more like data bridge.

  1. I can create several services, but they do require separate stubs and whole data descriptions. Which is good from API description standpoint (i.e. for further expansion there is no need to look in server/client "black box", you just look at proto file and send message according rules described), but less effective performance-wize, because we use new stubs.
syntax = "proto3";

message FilesAndFilters {
    repeated string file = 1;
    repeated string ext = 2;
}

message ResultList {
    repeated string file = 1;
}

//   <ServiceName>
service FList {
    // This is common method, it is described on server and called via stub on client.
    // <MethodName>(<ParamRequest>)
    rpc StreamFiles(FilesAndFilters) returns (ResultList);
}

message Settings {
    int32 profile = 1;
    bytes config = 2;
}

service CCConfig {
    rpc StreamConfig(Settings) returns (Settings);
}

Question to gRPC gurus: What would be optimal middle ground to keep everything transparent and self-explanatory and retain performance.

Please consider that at some point there could emerge argument, that my application has pretty low load and performance loss could be negligible. My goal here is to share experience and understand all the quirks before I bring technology to production, so same approach could be used on high load system.

Thank you.

Upvotes: 0

Views: 362

Answers (1)

Richard Belleville
Richard Belleville

Reputation: 1620

I think you may be misinterpreting the phrase "Always re-use stubs and channels when possible." This guidance is about avoiding a repeated teardown and recreation of a client channel, not about combining all of your RPC services into one.

In short, go with option #2. Having multiple services is idiomatic and encouraged.

Upvotes: 1

Related Questions