CODEWITHSUNDEEP
jsongoprotocol-buffers
Cajus Pollmeier
Cajus Pollmeier

Reputation: 312

How to unmarshal JSON into protobuf containing oneof definitions using jsonpb?

I'm currently failing to unmarshal a JSON snippet which is generated by jsonpb. Maybe it's just some kind of misunderstanding on my side, but when looking at the tests I'd expect it to work somehow.

Here's the relevant snippet of the pb.proto:

syntax = "proto3";
package pb;

message Parameter {
    string name = 1;
    oneof value {
        string str_value = 2;
        int32 int_value = 3;
        bool bool_value = 4;
        float float_value = 5;
    }
}

message ParameterSet {
    bytes raw = 1;
    repeated Parameter parameters = 2;
}

message ParameterSets {
    map<string,ParameterSet> sets = 1;
}

Testing marshaling/unmarshaling using this simple snippet fails:

package main

import (
    "fmt"
    "github.com/gogo/protobuf/jsonpb"
    "strings"
)

func main() {
    m := jsonpb.Marshaler{}
    str, err := m.MarshalToString(&pb.ParameterSets{Sets: map[string]*pb.ParameterSet{
        "parameter": {
            Raw: []byte{0, 1, 2, 3, 4, 5, 6, 1, 2},
            Parameters: []*pb.Parameter{
                {Name: "itest", Value: &pb.Parameter_IntValue{42}},
                {Name: "stest", Value: &pb.Parameter_StrValue{"Foobar"}},
                {Name: "btest", Value: &pb.Parameter_BoolValue{true}},
                {Name: "ftest", Value: &pb.Parameter_FloatValue{41.99}},
           },
        },
    }},)
    fmt.Println(str)
    fmt.Println(err)

    sets := pb.ParameterSets{}
    err = jsonpb.Unmarshal(strings.NewReader(str), &sets)
    fmt.Println(sets)
    fmt.Println(err)
}

It results in:

{"sets":{"parameter":{"raw":"AAECAwQFBgEC","parameters":[{"name":"itest","intValue":42},{"name":"stest","strValue":"Foobar"},{"name":"btest","boolValue":true},{"name":"ftest","floatValue":41.99}]}}}
<nil>
{map[]}
unknown field "intValue" in pb.Parameter

How can I get the oneof values back in the proto object?

Upvotes: 5

Views: 16446

Answers (3)

Olshansky
Olshansky

Reputation: 6404

2022 Update: I haven't been able to find a good answer to this online so figured I'd share my own with the caveat that I'm far from sure if this is the best approach.

If you have a proto like this:

message WrapperProto {
   string answer_to_the_universe = 1;
   ProtoWithOneof oneof_example = 2;
}

message ProtoWithOneof {
  oneof option {
    string string_option = 1;
    uint32 int_option = 2;
  }
}

You'll need to add the following code snippet in the same package/directory where the .pb.go file is compiled to:

import "google.golang.org/protobuf/encoding/protojson"

func (p *ProtoWithOneof) UnmarshalJSON(data []byte) error {
    return protojson.Unmarshal(data, p)
}

And then, when unmarshalling WrapperProto, you can configure your json string like so:

{
  "answer_to_the_universe": "42",
  "oneof_example": {
     "string_option": "I am a string"
  }
}

or like so:

{
  "answer_to_the_universe": "42",
  "oneof_example": {
     "int_option": 69
  }
}

Upvotes: 1

Luke
Luke

Reputation: 2454

I don't think it is safe to manually build the message as you are doing since you cannot grant that the resulting JSON matches exactly what would be produced by jsonpb.Marshal. Better to just instantiate Parameter. This github issue discusses a scenario that is in a way similar to yours and one of the solutions may help you:

package main

import (
    "fmt"
    "strings"

    "github.com/golang/protobuf/jsonpb"
    "github.com/golang/protobuf/proto"
)

func main() {
    msg := &Person{
        Name: "Allen",
        Age: 30,
        Married: true,
        Number: &Person_F{float32(14500.6)},
    }

    m := jsonpb.Marshaler{}
    js, err := m.MarshalToString(msg)
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Printf("json output ==> %v\n", js)

    msg2 := &Person{}
    if err := jsonpb.Unmarshal(strings.NewReader(js), msg2); err != nil {
        fmt.Println(err)
        return
    }

    fmt.Printf("%+v\n", proto.MarshalTextString(msg2))
}

Upvotes: 1

RickyA
RickyA

Reputation: 16029

You have a Type mix up due to the two vendor folders. A type declared in two vendor folders is not the same, even if the code for them is exactly the same.

Thus for example: prototest-master/vendor/github.com/gogo/proto/messageSet (A) is not of the same type as prototest-master/proto/vendor/github.com/gogo/proto/messageSet (B) even as their imports are the same (github.com/gogo/proto)

In your project's main.go the construction of the marshaler jsonpb.Marshaler{} uses the types from (A) where the actual messages like &pb.ParameterSets{... use the types from (B) to construct itself.

Since jsonpb seems to do a lot of reflection stuff it breaks on the mixing of the two types.

A better solution is to use just one vendor folder to be clear on the types everyone uses. I would just ditch the (B) vendor folder because it adds nothing.

Upvotes: 1

Related Questions