Sending a flexible Amount of Data over Network by using Asio (Boost)

Question

I got a client and a server application which will send each other data by using the Asio (Standalone) library. Both applications consists of two (logical) parts:

1. A high level part: dealing with complex objects e.g. users, permissions,...
2. A low level part: sending data over network between client and server

Let's assume the complex objects are already serialized by using Protocoll Buffers and the low level part of the application receives the data as std::string from the high level part. I would like to use this function from Protocoll Buffers for this job:

bool SerializeToString(string* output) const;: serializes the message and stores the bytes in the given string. Note that the bytes are binary, not text; we only use the string class as a convenient container.

And say I transfer this data with async_write on the client side:

size_t dataLength = strlen(data);

//writes a certain number of bytes of data to a stream.
asio::async_write(mSocket,
                      asio::buffer(data, dataLength),
                      std::bind(&Client::writeCallback, this,
                                std::placeholders::_1,   
                                std::placeholders::_2)); 

How can I read this data on the server side? I don't know how much data I will have to read. Therefore this will not work (length is unknown):

 asio::async_read(mSocket,
                     asio::buffer(mResponse, length),
                     std::bind(&Server::readCallback, this,
                               std::placeholders::_1,
                               std::placeholders::_2));

What is the best way to solve this problem? I could think of two solutions:

1. Append a 'special' character at the end of data and read until I reach this 'end of data signal'. The problem is, what if this character appears in data somehow? I don't know how Protocoll Buffers serializes my data.
2. Send a binary string with size_of_data + data instead of data. But I don't know how to serialize the size in an platform independent way, add it to the binary data and extract it again.

Edit: Maybe I could use this:

    uint64_t length = strlen(data);
    uint64_t nwlength = htonl(length);
    uint8_t len[8];
    len[0] = nwlength >> 56;
    len[1] = nwlength >> 48;
    len[2] = nwlength >> 40;
    len[3] = nwlength >> 32;
    len[4] = nwlength >> 24;
    len[5] = nwlength >> 16;
    len[6] = nwlength >> 8;
    len[7] = nwlength >> 0;

    std::string test(len);

    mRequest = data;
    mRequest.insert(0, test);

and send mRequest to the server? Any traps or caveats with this code? How could I read the length on server side and the content afterwards? Maybe like this:

void Server::readHeader(){

    asio::async_read(mSocket,
                     asio::buffer(header, HEADER_LENGTH),
                     std::bind(&Server::readHeaderCallback, this,
                               std::placeholders::_1,
                               std::placeholders::_2),
                     asio::transfer_exactly(HEADER_LENGTH));
}

void Server::readHeaderCallback(const asio::error_code& error,
                                        size_t bytes_transferred){

    if(!error && decodeHeader(header, mResponseLength)){
        //reading header finished, now read the content
        readContent();
    }
    else{
        if(error) std::cout << "Read failed: " << error.message() << "\n";
        else std::cout << "decodeHeader failed \n";       
    }
}

void Server::readContent(){

    asio::async_read(mSocket,
                     asio::buffer(mResponse, mResponseLength),
                     std::bind(&Server::readContentCallback, this,
                               std::placeholders::_1,
                               std::placeholders::_2),
                     asio::transfer_exactly(mResponseLength));
}

void Server::readContentCallback(const asio::error_code& error,
                                         size_t bytes_transferred){
    if (!error){
       //handle content
    }
    else{
        //@todo remove this cout
        std::cout << "Read failed: " << error.message() << "\n";      
    }
}

Please note that I try to use transfer_exactly. Will this work?

Answer 1

client must call

socket.shutdown(asio::ip::tcp::socket::shutdown_both);
socket.close();

on the server size read until EOF detected

std::string reveive_complete_message(tcp::socket& sock)
{
  std::string json_msg;
  asio::error_code error;
  char buf[255];

  while (1)
  {
    //read some data up to buffer size 
    size_t len = sock.read_some(asio::buffer(buf), error);

    //store the received buffer and increment the total return message 
    std::string str(buf, len);
    json_msg += str;
    if (error == asio::error::eof)
    {
      //EOF received, the connection was closed by client
      break;
    }
    else if (error)
    {
      throw asio::system_error(error);
    }
  }

  return json_msg;
}

Answer 2

When sending variable length messages over a stream-based protocol, there are generally three solutions to indicate message boundaries:

• Use a delimiter to specify message boundaries. The async_read_until() operations provide a convenient way to read variable length delimited messages. When using a delimiter, one needs to consider the potential of a delimiter collision, where the delimiter appears within the contents of a message, but does not indicate a boundary. There are various techniques to handle delimiter collisions, such as escape characters or escape sequences.

• Use a fixed-length header with a variable-length body protocol. The header will provide meta-information about the message, such as the length of the body. The official Asio chat example demonstrates one way to handle fixed-length header and variable-length body protocols.

  If binary data is being sent, then one will need to consider handling byte-ordering. The hton() and ntoh() family of functions can help with byte-ordering. For example, consider a protocol that defines the field as two bytes in network-byte-order (big-endian) and a client reads the field as a uint16_t. If the value 10 is sent, and a little-endian machine reads it without converting from network-order to local-order, then the client will read the value as 2560. The Asio chat example avoids handling endianness by encoding the body length to string instead of a binary form.

• Use the connection's end-of-file to indicate the end of a message. While this makes sending and receiving messages easy, it limits the sender to only one message per connection. To send an additional message, one would need to established another connection.

---

A few observations about the code:

• The Protocol Buffers' SerializeToString() function serializes a message to a binary form. One should avoid using text based functions, such as strlen(), on the serialized string. For instance, strlen() may incorrectly determine the length, as it will treat the first byte with a value of 0 as the terminating null byte, even if that byte is part of the encoded value.

• When providing an explicitly sized buffer to an operation via asio::buffer(buffer, n), the default completion condition of transfer_all will function the same as transfer_exactly(n). As such, the duplicate use of variables can be removed:

  asio::async_read(mSocket,
                   asio::buffer(header, HEADER_LENGTH),
                   std::bind(&Server::readHeaderCallback, this,
                            std::placeholders::_1,
                            std::placeholders::_2));
  

• The htonl() overloads support uint16_t and uint_32t, not uint64_t.

• Asio supports scatter/gather operations, allowing a receive operation to scatter-read into multiple buffers, and transmit operations can gather-write from multiple buffers. As such, one does not necessarily need to have both the fixed-length header and message-body contained with a single buffer.

  std::string body_buffer;
  body.SerializeToString(&body_buffer);
  std::string header_buffer = encode_header(body_buffer.size());
  
  // Use "gather-write" to send both the header and data in a
  // single write operation.
  std::vector<boost::asio::const_buffer> buffers;
  buffers.push_back(boost::asio::buffer(header_buffer));
  buffers.push_back(boost::asio::buffer(body_buffer));
  boost::asio::write(socket_, buffers);