CODEWITHSUNDEEP
pythonjsonprotocol-buffersgrpc
Parachute
Parachute

Reputation: 1268

Converting gRPC protobuf message to json runs for long

We are trying to convert a gRPC protobuf message to finally be a json format object for processing in python. The data sent across from server in serialized format is around 35MB and there is around 15K records. But when we convert protobuf message into string (using MessageToString) it is around 135 MB and when we convert protobuf message into a JSON string (using MessageToJson) it is around 140MB. But the time taken for conversion is around 5 minutes for each. It does not add any value wherein we take so much time to convert data on the client side.

Any thoughts or suggestion or caveats that we are missing would be helpful. Thanks.

Upvotes: 0

Views: 1730

Answers (1)

Parachute
Parachute

Reputation: 1268

Fixed the issue by only picking the fields that is needed when deserializing the data, rather than deserialize all the data returned from the server.

Upvotes: 0

Related Questions