CODEWITHSUNDEEP
pythonprotocol-buffersgrpcgrpc-python
Qagan
Qagan

Reputation: 51

GRPC_TOOLS.PROTOC generates broken pb2 python file

I am trying to build an application using gRPC in Python. I have the following directory layout:

service/

├─ auth/

│  ├─ auth_pb2_grpc.py

│  ├─ auth_pb2.py

│  ├─ auth.py

│  ├─ client.py

├─ flask/

├─ todo/

├─ app.py

├─ auth.proto/

├─ generate.sh/

├─ requirements.txt

I set up the code for proto file in the following way:

syntax = "proto3";
package auth;


message RegisterRequest {
    string username = 1;
    string password = 2;
}

message RegisterResponse {
    bool success = 1;
    string message = 2;
}

message LoginRequest {
    string username = 1;
    string password = 2;
}

message LoginResponse {
    bool success = 1;
    string message = 2;
    string token = 3;
}

message ValidateTokenRequest {
    string token = 1;
}

message ValidateTokenResponse {
    bool success = 1;
    string message = 2;
}

service AuthService {
    rpc Register(RegisterRequest) returns (RegisterResponse) {}
    rpc Login(LoginRequest) returns (LoginResponse) {}
    rpc ValidateToken(ValidateTokenRequest) returns (ValidateTokenResponse) {}
}

when I run

python -m grpc_tools.protoc -I. --python_out=./auth --grpc_python_out=./auth ./auth.proto

protobuf generates the file into auth subdirectory.

Seems good, ain't it? However, when I run app.py to test newly created files, it returns -

ModuleNotFoundError: No module named 'auth_pb2'

Here are the insides of 'auth_pb2' file:

    # -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: auth.proto
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\nauth.proto\x12\x04\x61uth\"5\n\x0fRegisterRequest\x12\x10\n\x08username\x18\x01 \x01(\t\x12\x10\n\x08password\x18\x02 \x01(\t\"4\n\x10RegisterResponse\x12\x0f\n\x07success\x18\x01 \x01(\x08\x12\x0f\n\x07message\x18\x02 \x01(\t\"2\n\x0cLoginRequest\x12\x10\n\x08username\x18\x01 \x01(\t\x12\x10\n\x08password\x18\x02 \x01(\t\"@\n\rLoginResponse\x12\x0f\n\x07success\x18\x01 \x01(\x08\x12\x0f\n\x07message\x18\x02 \x01(\t\x12\r\n\x05token\x18\x03 \x01(\t\"%\n\x14ValidateTokenRequest\x12\r\n\x05token\x18\x01 \x01(\t\"9\n\x15ValidateTokenResponse\x12\x0f\n\x07success\x18\x01 \x01(\x08\x12\x0f\n\x07message\x18\x02 \x01(\t2\xca\x01\n\x0b\x41uthService\x12;\n\x08Register\x12\x15.auth.RegisterRequest\x1a\x16.auth.RegisterResponse\"\x00\x12\x32\n\x05Login\x12\x12.auth.LoginRequest\x1a\x13.auth.LoginResponse\"\x00\x12J\n\rValidateToken\x12\x1a.auth.ValidateTokenRequest\x1a\x1b.auth.ValidateTokenResponse\"\x00\x62\x06proto3')

_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'auth_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:

  DESCRIPTOR._options = None
  _REGISTERREQUEST._serialized_start=20
  _REGISTERREQUEST._serialized_end=73
  _REGISTERRESPONSE._serialized_start=75
  _REGISTERRESPONSE._serialized_end=127
  _LOGINREQUEST._serialized_start=129
  _LOGINREQUEST._serialized_end=179
  _LOGINRESPONSE._serialized_start=181
  _LOGINRESPONSE._serialized_end=245
  _VALIDATETOKENREQUEST._serialized_start=247
  _VALIDATETOKENREQUEST._serialized_end=284
  _VALIDATETOKENRESPONSE._serialized_start=286
  _VALIDATETOKENRESPONSE._serialized_end=343
  _AUTHSERVICE._serialized_start=346
  _AUTHSERVICE._serialized_end=548
# @@protoc_insertion_point(module_scope)

As I see the problem is in protobuf generation because p2b file is somehow not complete. No clue why it occurs...

Also, there was a similar question grpc_tools.protoc generates python files with broken imports, but there was no explicit answer.

Could you please hint on where I might be wrong in my code?

Upvotes: 5

Views: 2238

Answers (3)

Elena Kutanov
Elena Kutanov

Reputation: 11

Found one more stable version, which works with python:3.9-alpine

grpcio-tools==1.46.5
Protobuf==3.19.5

Upvotes: 1

Mikelangello
Mikelangello

Reputation: 29

I have had the same issue when generating the p2b file. I have been able to solve it by degrading the next packages:

  • Protobuf==3.14.0
  • grpcio-tools==1.33.2

I think that there must be a bug on the new releases of grpcio-tools and protobuf. I started using grpc in python on december 2020 and the versions I used on that time works fine.

Upvotes: 2

DazWilkin
DazWilkin

Reputation: 40296

I think you're encountering this issue with Python code-generation.

You don't include the proto file but I suspect (?) you're not generating the services/messages into a package {foo};.

In this case, I recommend that you put the generated sources into the root directory (not ./auth). import'ing the file should then work.

python3 -m grpc_tools.protoc \
--proto_path=${PWD} \
--python_out=${PWD} \
--grpc_python_out=${PWD} \
${PWD}/auth.proto

If you don't want this structure, you will need to:

  1. edit package names
  2. import's; and
  3. create package __init__.py's

Please don't include screenshots on Stack overflow. Screenshots may not outlive the Stack overflow content and they inhibit copy-and-paste. You can replace screenshots by copying-and-pasting text in the first place.

Upvotes: 0

Related Questions