CODEWITHSUNDEEP
goprotocol-buffersgrpc
littlechad
littlechad

Reputation: 1228

Golang proto file management and importing

I have 2 grpc services (service1 and service2) that interact with each other and on some cases the rpc response of service1 will consists of a struct defined in service2, after going to several situations where duplication is inevitable, i figure that as the services grow these will be hard to manage, so i restructure the proto files into something like this for now

.
├── app
...
├── proto
│   ├── service1
│   │   ├── service1.access.proto 
│   │   ├── service1.proto
│   ├── service2
│   │   ├── service2.access.proto
│   │   └── service2.proto
│   └── model
│       ├── model.service1.proto
│       └── model.service2.proto
└── proto-gen // the protoc generated files 
    ├── service1
    │   ├── service1.access.pb.go
    │   └── service1.pb.go
    ├── service2
    │   ├── service2.access.pb.go
    │   └── service2.pb.go
    └── model
        ├── model.service1.pb.go
        └── model.service2.pb.go

service1 needs to import the model definition on model/model.service2.proto, so i am importing it like this

import "model/model.service2.proto";

option go_package = "proto-gen/service1";

and i generate the .pb.go files, using this protoc command

ls proto | awk '{print "protoc --proto_path=proto proto/"$1"/*.proto --go_out=plugins=grpc:."}' | sh

the command generates the .pb.go files just fine, but the code on service1.access.pb.go doesn't seem to import the model correctly, and i don't know if it related or not but when i run the app, it throws this error

cannot load model: malformed module path "model": missing dot in first path element

i spent a few hours now googling on how can i properly import another proto file, i can't seem to find any solution

Upvotes: 3

Views: 11514

Answers (3)

Jacob
Jacob

Reputation: 2414

I was also facing a similar issue while importing. Had changed the .protoc file option package with the following.

option go_package = "./;proto-gen/service1";
  1. The first param means relative path where the code you want to generate.
  2. The path relative to the --go_out , you set in your command.

Upvotes: 0

AlphaKilo
AlphaKilo

Reputation: 13

Building on Burak Serdar, I want to provide my implementation.

Set the package on the proto you want to import similar to this where the location is your full path. My path is generally github.com/AllenKaplan/[project]/[package]/proto/

option go_package = [path];

In the file you wish to import to add an import. My path is generally [package]/proto/[package].proto

import = [path from protoc proto path]

The last part is the protoc command where you must define the protopath in a way that connects the import path and option go_package path

if executing from the github.com/AllenKaplan/[project] directory, I would call

protoc -I. --go_out=./[package]/proto [package]/proto/[package].proto

-I. === --proto_path.

the -I. sets the protopath to the entire project

One note, when calling protoc on your .proto files that you are importing, you will want to add source_relative: to the output will ensure the output is from the root with a set package.

My implementation of the imported protoc when called from github.com/AllenKaplan/[project]/[package]

protoc -I./proto --go_out=paths=source_relative:./proto [package].proto

Upvotes: 1

Burak Serdar
Burak Serdar

Reputation: 51577

The reason you got that error about model is because the generated files use the go_package of the imported file, and model is not a valid import path. You have to convince the generated file to use the full import path of the package.

This is how I did it for my source tree: I have a similar tree of proto files importing each other. If your module is named, say github.com/myapp, then run protoc with --proto-path=<directory containing github.com>, import other proto files using full path, that is github.com/myapp/proto/service1/service1.proto, and in service1.proto, define go_package = service1. This setup writes the import paths correctly in my case.

Before settling into this solution, I was using go_package=<full path to proto>, so you might give that a try as well.

Upvotes: 1

Related Questions