Protocol Buffer Corrupting Data

Question

This is my first question on here and first project on my own. I have a client (C++ on Windows 10) using the SimConnect SDK to pull data from the Prepar3D Flight Simulator and a server (C on Ubuntu) that is receiving this data. I am using sockets and Protocol Buffers. The client receives the data from the sim, maps it to a protocol buffer and sends it. The server receives it from the socket, decodes the protocol buffer and then processes it. The issue is that the last field (dWindDirection) is always corrupted on the receiving end. I'm not entirely sure how/why it's being corrupted but it's consistent as in the same input to the socket always leads to the same corrupted output from the socket. I followed Protocol Buffer over socket in C++ for the use of protocol buffers over a socket. I have included my code below as well as the output.

Source code can also be found here. https://github.com/nbreen/Simconnect-Data-Client

Client function that sends the protobuff

DWORD WINAPI createProto(LPVOID lParam) {
    ObjectData* toConvert = (ObjectData*)lParam;
    fopen_s(&dataOut,"dataOut.txt", "a+");
    fprintf(dataOut, "Before serialiing \n");
    logData(toConvert);
    simConnect::simData convertedData;
    std::string toStr(toConvert->szTitle);

    convertedData.set_sztitle(toStr);
    convertedData.set_dabsolutetime(toConvert->dAbsoluteTime);
    convertedData.set_dtime(toConvert->dTime);
    convertedData.set_usimonground(toConvert->uSimOnGround);
    convertedData.set_daltitude(toConvert->dAltitude);
    convertedData.set_dheading(toConvert->dHeading);
    convertedData.set_dspeed(toConvert->dSpeed);
    convertedData.set_dverticalspeed(toConvert->dVerticalSpeed);
    convertedData.set_dgpseta(toConvert->dGpsEta);
    convertedData.set_dlatitude(toConvert->dLatitude);
    convertedData.set_dlongitude(toConvert->dLongitude);
    convertedData.set_dsimtime(toConvert->dSimTime);
    convertedData.set_dtemperature(toConvert->dTemperature);
    convertedData.set_dairpressure(toConvert->dPressure);
    convertedData.set_dwindvelocity(toConvert->dWindVelocity);
    convertedData.set_dwinddirection(toConvert->dWindDirection);

    printf("Size after serializing is %ld\n", convertedData.ByteSizeLong());
    long pktSize = convertedData.ByteSizeLong();
    fprintf(dataOut, "After serializing before socket\n%s\n\n", convertedData.DebugString().c_str());
    char* pkt = new char[pktSize];
    google::protobuf::io::ArrayOutputStream aos(pkt, pktSize);
    google::protobuf::io::CodedOutputStream* coded_pkt = new google::protobuf::io::CodedOutputStream(&aos);
    coded_pkt->WriteVarint64(convertedData.ByteSizeLong());
    convertedData.SerializeToCodedStream(coded_pkt);
    int iResult = send(clientSocket, pkt, pktSize, 0);
    printf("Sent bytes %d\n", iResult);
    fclose(dataOut);

    return 0;
}

Server function that receives and processes the protobuff

while(1) {
        result = recv(client, sizeBuff, 4, MSG_PEEK);
        printf("Receive is %d\n", result);
        if (result == -1) {
            printf("Error receiving data with byteSize\n");
            break;
        }else if (result == 0) {
            break;
        }

        if (result > 0) {
            printf("First read byte count is %d\n", result);
            readMessage(client, readHeader(sizeBuff));
        }
    }

google::protobuf::uint64 readHeader(char *buf) {
  google::protobuf::uint64 size;
  google::protobuf::io::ArrayInputStream ais(buf,4);
  google::protobuf::io::CodedInputStream coded_input(&ais);
  coded_input.ReadVarint64(&size);//Decode the HDR and get the size
  std::cout<<"size of payload is "<<size<<std::endl;
  return size;
}

void readMessage(int csock, google::protobuf::uint64 siz) {
  int bytecount;
  simConnect::simData payload;
  char buffer [siz+4];//size of the payload and hdr
  //Read the entire buffer including the hdr
  if((bytecount = recv(csock, (void *)buffer, 4+siz, 0))== -1){
                fprintf(stderr, "Error receiving data %d\n", errno);
        }
  std::cout<<"Second read byte count is "<<bytecount<<std::endl;
  //Assign ArrayInputStream with enough memory 
  google::protobuf::io::ArrayInputStream ais(buffer,siz+4);
  google::protobuf::io::CodedInputStream coded_input(&ais);
  //Read an unsigned integer with Varint encoding, truncating to 32 bits.
  coded_input.ReadVarint64(&siz);
  //After the message's length is read, PushLimit() is used to prevent the CodedInputStream 
  //from reading beyond that length.Limits are used when parsing length-delimited 
  //embedded messages
  google::protobuf::io::CodedInputStream::Limit msgLimit = coded_input.PushLimit(siz);
  //De-Serialize
  payload.ParseFromCodedStream(&coded_input);
  //Once the embedded message has been parsed, PopLimit() is called to undo the limit
  coded_input.PopLimit(msgLimit);
  //Print the message
  //std::cout<<"Message is "<<payload.DebugString();
  FILE *outFile = fopen("out.txt", "a+");
  /*std::string strPkt(buffer);
  payload.ParseFromString(strPkt);*/
  fprintf(outFile, "From socket\n%s\n\n", payload.DebugString().c_str());
  fclose(outFile);
  cudaProcess(payload);
}

Client Output

Before serialiing 
Title: F-22 Raptor - 525th Fighter Squadron
Absolute Time: 63631767652.278290 seconds
Zulu Time: 68452.281250 Seconds
Sim On Ground: 0 
Altitude: 11842.285630 Feet
Heading: 3.627703 Radians
Speed: 426.008209 Knots
Vertical Speed: 596.607849 Feet Per Second
GPS ETA: 0.000000 Seconds
Latitude: 30.454685 Degrees
Longitude: -86.525197 Degrees
Sim Time: 2996.388142 Seconds
Temperature: -8.145923 Celsius
Air Pressure: 648.718994 Millibars
Wind Velocity: 36.988354 Feet Per Second
Wind Direction: 270.000000 Degrees


After serializing before socket
szTitle: "F-22 Raptor - 525th Fighter Squadron"
dAbsoluteTime: 63631767652.27829
dTime: 68452.28125
usimOnGround: 0
dAltitude: 11842.285629708613
dHeading: 3.6277025313026936
dSpeed: 426.00820922851562
dVerticalSpeed: 596.60784912109375
dGpsEta: 0
dLatitude: 30.454685493870045
dLongitude: -86.525197127202063
dSimTime: 2996.388142343636
dTemperature: -8.1459226608276367
dAirPressure: 648.718994140625
dWindVelocity: 36.988353729248047
dWindDirection: 270

Server Output

From socket
szTitle: "F-22 Raptor - 525th Fighter Squadron"
dAbsoluteTime: 63631767652.27829
dTime: 68452.28125
usimOnGround: 0
dAltitude: 11842.285629708613
dHeading: 3.6277025313026936
dSpeed: 426.00820922851562
dVerticalSpeed: 596.60784912109375
dGpsEta: 0
dLatitude: 30.454685493870045
dLongitude: -86.525197127202063
dSimTime: 2996.388142343636
dTemperature: -8.1459226608276367
dAirPressure: 648.718994140625
dWindVelocity: 36.988353729248047
dWindDirection: 1.2168372663711258e-309 <-- This is always the corrupt value when Wind direction is 270

Answer 1

As jpa said in the comments I was not accounting for the prefix I was appending to the packet so though the size of the data was correct it was not accounting for the 4 byte header that I was appending. However, when the server read the packet it was accounting for the total size and the header that was appended so I was always short by 4 bytes when decoding the packet on the server side giving me a garbage value for the last field of the object. In the client code adding 4 (The size of the header) to pktSize fixed the problem.

Answer 2

If you look at the raw hex values for the dWindDirection, they are:

270:                     0x00 0x00 0x00 0x00 0x00 0xe0 0x70 0x40
1.2168372663711258e-309: 0x00 0x00 0x00 0x00 0x00 0xe0 0x00 0x00

So the two last bytes of your packet are being set to 0 instead of the real value.

It sounds like there might be a problem with your socket reading code. I would start by verifying the data in network connection using e.g. Wireshark. If it is correct there, set a breakpoint in your reception code and check sizes & buffer content.