CODEWITHSUNDEEP
c#protocol-buffersprotobuf-net
tianwei
tianwei

Reputation: 39

OverFlowException and EndOfStreamException in Protobuf-net

I have been reading about protobuf-net and it is amazing!

In general, it works perfect. But I meet some problems.

I am trying to write communication code between Python and C# with protobuf.

The .proto belows:

message GetAllCalculate{
    required string agentID=1;
}

message CalculateInfo{
    required string CalStarttime=1;
    optional string CalEndtime=2;
    required string Smiles=3;
    optional string CAS=4;
    optional string ChName=5;
    optional string EnName=6;
    required string Param=7;
    required string Result=8;
    required bool IsFinished=9;
}

message GetAllCalulateResponse{
    required bool  isSuccessful = 1;
    required int32 Count=2;
    repeated CalculateInfo History=3;

}

In Python client, the code like:

msg_resp = GetAllCalulateResponse()
  calculateInfo = [None] * 2
    cnt = 0
    for result in resultSets:   #resultSets can read from other place,like database
        calculateInfo[cnt] = msg_resp.History.add()
        calculateInfo[cnt].CalStarttime = str(result.calculateStartTime)
        calculateInfo[cnt].CalEndtime = result.calculateEndTime.strftime('%Y-%m-%d %X')
        calculateInfo[cnt].IsFinished = result.isFinished
        calculateInfo[cnt].Param = result.paramInfo
        **calculateInfo[cnt].Result = str('ff'*50) #result.result**

        calculateInfo[cnt].Smiles = result.smilesInfo.smilesInfo
        calculateInfo[cnt].CAS = result.smilesInfo.casInfo


        nameSets = CompoundName.objects.filter(simlesInfo=result.smilesInfo.pk,isDefault=True)
        for nameSet in nameSets:
            if nameSet.languageID.languageStr == Chinese_Name_Label:
                calculateInfo[cnt].ChName = nameSet.nameStr 
            elif nameSet.languageID.languageStr == English_Name_Label:
                calculateInfo[cnt].EnName = nameSet.nameStr

        cnt = cnt +1

C# Code(using Protobuf-net):

string retString = HTTPPost2UTF8(bytes, GetAllCalculateHandlerAPI); //Get from Python Clint
bytesOut = System.Text.Encoding.UTF8.GetBytes(retString);
MemoryStream streamOut = new MemoryStream(bytesOut);
GetAllCalulateResponse response = Serializer.Deserialize <GetAllCalulateResponse>(streamOut);

But When I make **calculateInfo[cnt].Result = str('ff'*50) #result.result** big, like str('ff') * 5000, C# client will throw OverFlowException. When I set it str('ff') * 100, it will throw EndOfStreamException.

How can solve this problem? Thanks in advance!

Upvotes: 1

Views: 636

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1063338

This has me worried:

string retString = HTTPPost2UTF8(bytes, GetAllCalculateHandlerAPI); //Get from Python
bytesOut = System.Text.Encoding.UTF8.GetBytes(retString);
MemoryStream streamOut = new MemoryStream(bytesOut);

protobuf data is not text, and it is certainty not UTF8. There is no valid way of passing protobuf binary around as UTF8 without corrupting it. Options:

  1. exchange it entirely as raw binary; no text
  2. Use something like base-64, which can safely encode arbitrary binary into ASCII

Upvotes: 2

Related Questions