how to receive unkown size buffer in async mode based boost::asio

Question

I am trying to receive an unkown size buffer, part of my code is as blow:

void Connection::asyncRead() 
{
    auto self(shared_from_this());
    socket_.async_read_some(boost::asio::buffer(buffer_out),
        [this, self](boost::system::error_code ec, std::size_t length)
    {
        OnRead(ec, length);
    });
}

I don't know the size of the buffer, so I try to receive the buffer in a fixed size buffer, how to know whether the buffer ends?

Answer 1

If you want to send/receive messages which don't have fixed size you can use the approach where you define header of your message, for example with 4-bytes field to store the size of content of your message:

[header(4 bytes) to store the size of message][content of message]

then you always know that the first step is to read 4 bytes, prepare buffer for data and read further data until buffer is filled.

Another way is shutting down the socket (below is pseudocode)

The receiving side             |  the sending side
--------------------------------------------------------------------------------
error_code ec;                 |
asio::read(sock,buf,ec) [2]    |
                               | prepare some buffer with unknown size
                               | string buf;
                               | buf += ...; // add data
                               | asio::write(sock,buf)
                               | sock.shutdown(socket_base::shutdown_send); [1]

by calling sock.shutdown() in the sending side [1] you can inform the receiving side that the whole message was sent. Then in the receiving side after a message was read [2] you should check status of ec error-code variable whether it is boost::asio::eof. If you get end-of-file you know that message is complete. If ec is different than eof this means that an error occurred.

As of 1.66 boost version you can use dynamic_buffer to store data, it adapts string or vector to be a buffer. Or you can consider streambuf to read non-fixed buffers.

---

EDIT

The use of dynamic_buffer added. According to reference dynamic_buffer can be used in free functions like async_read, async_until but not in async_read_some as member function of socket. Below are the codes of client and server:

Server:

using namespace boost;

struct Data  {
  std::shared_ptr<asio::ip::tcp::socket> sock;
  std::string buf; // buf is empty [1]
};

void readHandler (
    const boost::system::error_code& ec,
    size_t length,
    std::shared_ptr<Data> d) {
  std::cout << "readHandler" << std::endl;
  if (ec == boost::asio::error::eof)
  {
    // here we got the whole message
    std::cout << d->buf << std::endl;
  }
  else 
  {
    std::cout << "Error" << std::endl;
  }
}

int main() {
  try {
    asio::ip::tcp::endpoint ep(asio::ip::address_v4::any(),9999);
    asio::io_service ios;
    asio::ip::tcp::acceptor acceptor(ios, ep); 

    std::shared_ptr<asio::ip::tcp::socket> sock{new asio::ip::tcp::socket(ios)};

    acceptor.accept(*sock);

    std::shared_ptr<Data> data(new Data);
    data->sock = move(sock);

    boost::asio::async_read (*(data->sock), asio::dynamic_buffer(data->buf), 
        std::bind(readHandler,std::placeholders::_1, std::placeholders::_2,data)); // [2]

    ios.run(); // wait until async_write is complete
  }
  catch (system::system_error &e) {
    std::cout << "error " << e.what() << std::endl;
  }
  return 0;
}

in [1] we create empty buffer as string object, in [2] we call async_read to get data using dynamic_buffer. Handler passed into async_read is called when the sending side shutdowns the socket on its side.

Client:

using namespace boost;

int main() {
  try {
    asio::ip::tcp::endpoint ep(asio::ip::address::from_string("127.0.0.1"),9999);
    asio::io_service ios;
    asio::ip::tcp::socket sock(ios, ep.protocol());
    sock.connect(ep);

    std::string buf = "some message";
    for (int i = 0; i < buf.size(); ++i) {
       // synchronous function was used to make simpler code
       asio::write(sock,asio::buffer(buf.c_str()+i,1)); // send 1 char
       std::this_thread::sleep_for(std::chrono::seconds(1)); // delay 1 second
    }
    sock.shutdown(asio::socket_base::shutdown_send);
  }
  catch (system::system_error &e) {
    std::cout << "Error " << e.what() << std::endl;
  }
  return 0;
}

as you can see we are sending char by char from string with 1-second delay. So when you start server and then client, server should receive the whole message after ~ 12 seconds. async_read is waiting in server until eof come - it is send by the call of shutdown on socket by client.