How to de-serialize the file in Java by using Protocol-buffer?

Question

Want to create a serialized file in C# and then want to de-serialize the same in Java. For this , i am using Protocol buffer library. Here want i have done:

In C#, I added protobuf-net.dll and the class which is to be serialized is represented as:

Person.cs

 [ProtoBuf.ProtoContract]
 public class Person
 {
    [ProtoBuf.ProtoMember(1)]
    public int Id {get;set;}
    [ProtoBuf.ProtoMember(2)]
    public string Name { get; set; }
 }

and in the main method, serialized it like this:

 var person = new Person
    {
        Id = 12345,
        Name = "Fred",
    };
  using (var file = File.Create("somepath\\person.bin"))
    {
        Serializer.Serialize(file, person);
    }

This bin file i copied and pulled in the ecilpse filesystem- android /sdcard and tried to de-serialize it

In eclipse -JAVA,

Added protobuf.jar, external library and created person.proto file which contains:

message Person{
required int32 Id=1;
required string Name=2;
}

Can anyone please suggest how to de-serialize the object which was created in the C#?

Answer 1

The usual way how Protocol Buffers work is:

• You define your protocol in a small .proto text file.
• You run the protobuf compiler on that file
• proto compiler generates source code in all the languages you need to serialize & parse objects.

The library you use in C# is special. It does not need that .proto file to start with. Instead you can create the protocol within your code by annotating classes (or whatever the terminology in C# is).

In order to use your in-code-generated protocol from Java you need the .proto file since Java does not support that type of definition in code.

You can either write the .proto file by hand or let the C# library generate it (see Marc Gravell's answer) - I would suggest you generate the file so you can't make mistakes in the definition.

Once you have that file you run the protobuf compiler (protoc) (dowload) on the .proto file and it will generate a .java file for you. That file has the Person class and everything required to serialize and parse.

Now you include both protobuf[version].jar and the generated .java file in your project. The .proto file itself is not required in the project.

Once you have done that simply use the generated code to parse the file:

Person person = Person.parseFrom(new FileInputStream(file));

The only line of code that you write is that one above.

More details about the Java integration is found in the official tutorial

Answer 2

Create an InputStream to the file created in C#, and call

Person.parseFrom(InputStream)

There are other overloads to this method if you'd rather deal with bytes from that file.

If you are implementing a protocol you'll need to include a header to identify what type of data the bytes represent. From there you'd just select the correct proto to parse the data with.

EDIT

Here's a class I created for mapping id's to class's and vice versa to assist in developing a protocol with protobufs. If you aren't developing a protocol for network transmission this might not help, but I'm pretty sure you are.

I know you didn't ask for this, but perhaps you'll find it useful.

Register ids to all your protobuff generators, then retrieve the correct generator for unserializing bytes on receive. Get the correct ID for each protobuf object before you send. ID would be included in every packet so you know what type of data is in each packet. (Packet being abstract here, this would work with a stream protocol too.)

public class MessageTypeMap {
private final Object lock;
final HashMap<Integer, GeneratedMessageLite> messageParserMap;
final HashMap<Class<?>, Integer> messageClassParserMap;

public MessageTypeMap() {
    this.messageParserMap = new HashMap<Integer, GeneratedMessageLite>();
    this.messageClassParserMap = new HashMap<Class<?>, Integer>();
    this.lock = new Object();
}

public void addMessageType(int typeID, GeneratedMessageLite message) {
    synchronized (this.lock) {
        this.messageParserMap.put(typeID, message);
        this.messageClassParserMap.put(message.getDefaultInstanceForType()
                .getClass(), typeID);
    }
}

public GeneratedMessageLite getBuilderFor(int id) throws ProtocolException {
    synchronized (this.lock) {
        if (this.messageParserMap.containsKey(id)) {
            GeneratedMessageLite lite = this.messageParserMap.get(id);
            return lite;
        } else {
            throw new ProtocolException("No message builder for ID " + id);
        }
    }
}

public int getIDFor(Object obj) throws ProtocolException {
    synchronized (this.lock) {
        if (obj == null) {
            throw new NullPointerException(
                    "Object null while retrieving type id.");
        }
        Class<?> c = obj.getClass();
        if (this.messageClassParserMap.containsKey(c)) {
            int typeID = this.messageClassParserMap.get(c);
            return typeID;
        } else {
            throw new ProtocolException("No type id for class "
                    + c.getSimpleName());
        }
    }
}

}

Usage:

MessageTypeMap map = new MessageTypeMap();
//register the person type.
map.addMessageType(100, Person.getDefaultInstance());
//use this to unserialize whatever object some bytes are.
GeneratedMessageLite builder = mpa.getBuilderFor(100);
//be sure to include the type id with each transmission of an object.
int id = map.getIDFor(Person.getDefaultInstance());

Answer 3

What does the Java documentation say about deserializing an object serialized by the Java version? Basically: "do that". There is no difference in the serialized data.

If the problem is that the Java examples start with a .proto file, then use:

string proto = Serializer.GetProto<Person>();

Although the .proto you show looks fine.