CODEWITHSUNDEEP
goprotocol-buffersgrpc
Viktor Doroshenko
Viktor Doroshenko

Reputation: 111

Golang project Protobuf corrupted imports

I'm not a golang developer but currently I have to fix code on it, so sorry in advance if I accidentally not understand some basic Go's concept ;) I have a third party protobuf contract which I have to use and on which I don't have influence. I'm not able to provide actual example of the contract, so I've made similar example project on Github which has the same problem. In short: there is a deeply nested structure of proto documents where some of them import others:

syntax = "proto3";

package company.nested1.nested2;

import "company/common.proto";

option go_package = "nested2";

message CompanyMessage {
    CompanyEnum compEnum = 1;
}

references

syntax = "proto3";

package company;

option go_package = "company";

enum CompanyEnum {
    VAR1 = 0;
    VAR2 = 1;
}

There is a go.mod file with following code:

module go-program

go 1.16

require (
    github.com/golang/protobuf v1.5.2
    golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
    google.golang.org/grpc v1.40.0 // indirect
    google.golang.org/protobuf v1.26.0
)

And when compiling to the Golang code there is a problem in this import. Generated *.pb.go files have imports relative to company package while they are used inside of a go project with root in go-project so these imports can't be resolved: enter image description here

As I understand, previously this problem has been solved by manual editing of imports in generated files. Like in the screen below. But in my opinion there should be a better way for this ;)

enter image description here

I'm using this command to generate go code from proto files:

protoc --proto_path=proto (find proto -name '*.proto') --go_out=plugins=grpc:.

So the question is: what is a proper way to solve such import problem in go. Seems like imports in generated files should begin with my program root namespace. But how to achieve this - maybe some options in protoc command which I haven't found. I tried also to put generated files inside of $GOPATH/src or in vendors and tried to make and publish separate module - no result. Is there some proper solution for this?

Upvotes: 1

Views: 544

Answers (1)

DazWilkin
DazWilkin

Reputation: 40061

This is more some pointers rather than a definitive answer. If I find some time tomorrow, I'll repro this to give you a definitive answer.

It is challenging. Protobufs has to find a solution for each language's package management and Go's (while IMO improved) is somewhat compounded with recent addition of Modules.

My approach my with code is to have a separate repo for the protos and generated code. If the protos change, the code is regenerated. I can either reference the generated module (!) or regenerate myself. This model keeps the protos as the definitive "source" with the timesaver of "cached" generated sources. But I'm my own 3rd-party library maintainer in this config.

So:

  1. Modules are manifest locally as a directory tree whose root contains go.mod and go.sum and subsidirectories representing packages.
  2. When you generate the code for the 3rd-party protos, I think it would be best to create this under their own module, either as a standalone directory or as a vendored subdirectory.
  3. This module should be named to reference the 3rd party and can be created with go mod init and should contain the output of protoc
  4. Modules are usually pulled from a proxy or repo but you can circumvent this (since the 3rd party isn't hosting the generated code for you) by manually adding a replace to the go mod redirecting from the module path to a local path.
  5. From your error, this should solve the missing module issue and with a subdirectory e.g. company containing that generated code, should resolve the package too.

Upvotes: 3

Related Questions