CODEWITHSUNDEEP
serializationprotocol-buffers
Just Another Code Monkey
Just Another Code Monkey

Reputation: 11

Reference Semantics in Google Protocol Buffers

I have slightly peculiar program which deals with cases very similar to this (in C#-like pseudo code):

class CDataSet
{
   int m_nID;
   string m_sTag;
   float m_fValue;
   void PrintData()
   {
      //Blah Blah
   }
};

class CDataItem
{
  int m_nID;
  string m_sTag;
  CDataSet m_refData;
  CDataSet m_refParent;
  void Print()
  {
      if(null == m_refData)
       {
         m_refParent.PrintData();
       }
     else
       {
         m_refData.PrintData();
       }
  }
};

Members m_refData and m_refParent are initialized to null and used as follows: m_refData -> Used when a new data set is added m_refParent -> Used to point to an existing data set. A new data set is added only if the field m_nID doesn't match an existing one.

Currently this code is managing around 500 objects with around 21 fields per object and the format of choice as of now is XML, which at 100k+ lines and 5MB+ is very unwieldy.

I am planning to modify the whole shebang to use ProtoBuf, but currently I'm not sure as to how I can handle the reference semantics. Any thoughts would be much appreciated

Upvotes: 1

Views: 535

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1062580

Out of the box, protocol buffers does not have any reference semantics. You would need to cross-reference them manually, typically using an artificial key. Essentially on the DTO layer you would a key to CDataSet (that you simply invent, perhaps just an increasing integer), storing the key instead of the item in m_refData/m_refParent, and running fixup manually during serialization/deserialization. You can also just store the index into the set of CDataSet, but that may make insertion etc more difficult. Up to you; since this is serialization you could argue that you won't insert (etc) outside of initial population and hence the raw index is fine and reliable.

This is, however, a very common scenario - so as an implementation-specific feature I've added optional (opt-in) reference tracking to my implementation (protobuf-net), which essentially automates the above under the covers (so you don't need to change your objects or expose the key outside of the binary stream).

Upvotes: 1

Related Questions