CODEWITHSUNDEEP
c#deserializationprotocol-buffers
Nitzanu
Nitzanu

Reputation: 84

Deserializing ByteString into an object using C# Google.Protobuf

So i have a gRPC message coming in from a different service (written in another programming language). here is a mini version of this object (obviously the real one is generated by protobuf)

public class Message
{
    public string Topic { get; set; }
    public string Identifier { get; set; }
    public Google.Protobuf.ByteString Msg { get; set; }
}

My issue is that the ByteString should be deserialized into another object (also defined in the protobuf file), but when i am trying to deserialize the Msg field i keep getting this error:

Protocol message contained a tag with an invalid wire type.

and the stack trace

at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(CodedInputStream input) at Google.Protobuf.UnknownFieldSet.MergeGroupFrom(CodedInputStream input) at Google.Protobuf.CodedInputStream.ReadGroup(Int32 fieldNumber, UnknownFieldSet set) at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(CodedInputStream input) at Google.Protobuf.UnknownFieldSet.MergeFieldFrom(UnknownFieldSet unknownFields, CodedInputStream input) at Messages.RecordingStatusChangeMessage.MergeFrom(CodedInputStream input) in C:\Users\iliaar\go\src\RecorderApp\testers\NewRecorderTester\Infra.AppDataManager\Model\Protos\Pubsub\messages.pb.cs:line 3259 at ClusterRecordersModule.RecordersViewModel.OnNewMessage(Message message) in C:\Users\iliaar\go\src\RecorderApp\testers\NewRecorderTester\ClusterRecordersModule\ViewModels\RecordersViewModel.cs:line 154

I have tried calling several methods of the new object and all have failed for example i have tried using an existing instance and merging like:

innerMessageObject.MergeFrom(message.Msg.CreateCodedInput());

or using the static parser like:

InnerMessageObject.Parser.ParseFrom(message.Msg.ToByteArray());

and all have failed with the same error and the stack trace ultimately converges to the same place in Google.Protobuf library. my library version is 8.1.0 and I also tried downgrading all the way down to version 6

i would really appreciate some help. Thank you

Edit: the innerMessage object is structured as follows:

public class InnerMessageObject
{
    public string ConfigID { get; set; }
    public bool Storage { get; set; }
    public bool Signal { get; set; }
    public string StorageTransition { get; set; }
    public string SignalTransition { get; set; }
}

also, convering to Hex result in the following (valid) output

63-6F-6E-66-69-67-5F-69-64-3A-20-37-32-30-62-66-65-34-39-2D-64-62-32-39-2D-34-35-38-33-2D-39-66-65-31-2D-65-30-32-30-37-33-32-37-39-37-39-34-0A-73-74-6F-72-61-67-65-3A-20-66-61-6C-73-65-0A-73-69-67-6E-61-6C-3A-20-66-61-6C-73-65-0A-73-74-6F-72-61-67-65-5F-74-72-61-6E-73-69-74-69-6F-6E-3A-20-22-32-30-32-30-2D-30-33-2D-30-33-54-31-32-3A-32-36-3A-34-31-2E-33-32-37-34-30-34-33-5A-22-0A-73-69-67-6E-61-6C-5F-74-72-61-6E-73-69-74-69-6F-6E-3A-20-22-32-30-32-30-2D-30-33-2D-30-33-54-31-32-3A-32-36-3A-34-31-2E-33-32-37-34-30-33-32-33-31-5A-22-0A

furthermore, coverting the bytearray into string results in a string representation of the object with the correct data

var ba = pubsubMessage.Msg.ToByteArray();
return Encoding.UTF8.GetString(ba);

config_id: 720bfe49-db29-4583-9fe1-e02073279794 storage: false signal: false storage_transition: "2020-03-03T12:29:59.531473957Z" signal_transition: "2020-03-03T12:29:59.531473589Z"

Upvotes: 2

Views: 2361

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1063013

Your payload is not protobuf. If we try it through this validator, we see:

63 = field 12, type StartGroup

error: Invalid wire-type; this usually means you have over-written a file without truncating or setting the length; see Using Protobuf-net, I suddenly got an exception about an unknown wire-type

so let's look at the first two bytes as if they are protobuf - the decoder tells us what 0x63 means; the next byte 0x6F should be a field header ("tag"); in binary this is 01101111, which would be "field 13, wire-type 7"; there is no wire type 7 in protobuf. So; the decoder is correct: the payload is not valid.

But not all is lost!

If we take a guess from all those 0x6* values that this might be ASCII or UTF8, and decode it as such, we get:

config_id: 720bfe49-db29-4583-9fe1-e02073279794
storage: false
signal: false
storage_transition: "2020-03-03T12:26:41.3274043Z"
signal_transition: "2020-03-03T12:26:41.327403231Z"

which appears to be your data in some crude line-based tokenized format. But: not protobuf.

Upvotes: 2

Related Questions