CODEWITHSUNDEEP
python-3.xprotocol-buffers
N M
N M

Reputation: 616

How to convert a compiled protocol buffer back to .proto file?

I have a compiled google protocol buffer for python 2 and I'm attempting to port this to python 3. Unfortunately, I cannot find the proto file I used to generate the compiled protocol buffer anywhere. How do I recover the proto file so that I can compile a new one for python 3. I'm unaware of what proto versions were used and all I have is the .py file meant to run on python 2.6.

Upvotes: 6

Views: 4428

Answers (2)

quickinsights
quickinsights

Reputation: 1137

As mentioned in the other post(s), you'll need to walk through the tree of your descriptor message and build your proto file contents.

You can find a full C++ example in the protocol buffers github repository. Here are some C++ code snippets from the link in order to give you an idea on how to implement this in Python:

  // Special case map fields.
  if (is_map()) {
    strings::SubstituteAndAppend(
        &field_type, "map<$0, $1>",
        message_type()->field(0)->FieldTypeNameDebugString(),
        message_type()->field(1)->FieldTypeNameDebugString());
  } else {
    field_type = FieldTypeNameDebugString();
  }

  std::string label = StrCat(kLabelToName[this->label()], " ");

  // Label is omitted for maps, oneof, and plain proto3 fields.
  if (is_map() || containing_oneof() ||
      (is_optional() && !has_optional_keyword())) {
    label.clear();
  }

  SourceLocationCommentPrinter comment_printer(this, prefix,
                                               debug_string_options);
  comment_printer.AddPreComment(contents);

  strings::SubstituteAndAppend(
      contents, "$0$1$2 $3 = $4", prefix, label, field_type,
      type() == TYPE_GROUP ? message_type()->name() : name(), number());

Where the FieldTypeNameDebugString function is shown below:

// The field type string used in FieldDescriptor::DebugString()
std::string FieldDescriptor::FieldTypeNameDebugString() const {
  switch (type()) {
    case TYPE_MESSAGE:
      return "." + message_type()->full_name();
    case TYPE_ENUM:
      return "." + enum_type()->full_name();
    default:
      return kTypeToName[type()];
  }
}

Upvotes: 0

Omni
Omni

Reputation: 1022

You will have to write code (in Python for instance) to walk through the tree of your message descriptors. They should - in principle - carry the full information of your original proto file except the code comments. And the generated Python module you still have in your posession should allow you to serialize the file descriptor for your proto file as a file descriptor proto message which could then be fed to code expressing it as proto code.

As a guide you should look into the various code generators for protoc which actually do the same: they read in a file descriptor as a protobuf message, analyze it and generate code.

Here's a basic introduction how to write a Protobuf plugin in Python

https://www.expobrain.net/2015/09/13/create-a-plugin-for-google-protocol-buffer/

Here's the official list of protoc plugins

https://github.com/google/protobuf/blob/master/docs/third_party.md

And here's a protoc plugin to generate LUA code, written in Python.

https://github.com/sean-lin/protoc-gen-lua/blob/master/plugin/protoc-gen-lua

Let's have a look at the main code block

def main():
    plugin_require_bin = sys.stdin.read()
    code_gen_req = plugin_pb2.CodeGeneratorRequest()
    code_gen_req.ParseFromString(plugin_require_bin)

    env = Env()
    for proto_file in code_gen_req.proto_file:
        code_gen_file(proto_file, env,
                      proto_file.name in code_gen_req.file_to_generate)

    code_generated = plugin_pb2.CodeGeneratorResponse()
    for k in  _files:
        file_desc = code_generated.file.add()
        file_desc.name = k
        file_desc.content = _files[k]

    sys.stdout.write(code_generated.SerializeToString())

The loop for proto_file in code_gen_req.proto_file: actually loops over the file descriptor objects for which the code generator plugin was asked by protoc to generate LUA code. So now you could do something like this:

# This should get you the file descriptor for your proto file
file_descr = your_package_pb2.sometype.GetDescriptor().file
# serialized version of file descriptor
filedescr_msg = file_descr.serialized_pb
# required by lua codegen
env = Env()
# create LUA code -> modify it to create proto code
code_gen_file(filedescr, env, "your_package.proto")

Upvotes: 4

Related Questions