boost.asio : multi ports application-layer protocol

Question

I am designing a client-server app and I need to use three different channels between them. Below on the schema, the client opens a connection to the server (1), and then the server opens two connections to the client on other ports (2).

|-----------|                  |-----------|
|           |  (1) port: 8000  |           |
|           |<-----------------|           |
|           |                  |           |
|           |                  |           |
|  Server   |  (2) port: 8001  |  Client   |
|           |----------------->|           |
|           |                  |           |
|           |  (2) port: 8002  |           |
|           |----------------->|           |
|-----------|                  |-----------|

A simple channel server example :http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/example/cpp11/echo/async_tcp_echo_server.cpp

My questions is how to implement this, based on boost.asio framework ? Or, how to open the new connections (2) from the server ?

P.S. Maybe a better protocol could be to open the two other connections from client when the first is established ? But what could be the structure used in asio in this case ?

Thanks a lot in advance !

Answer 1

As has been pointed out, any system that's required to be robust should not assume that routing/firewalling is such that connections can be made back to the client from the server.

How would we go around this?

• client connects to server:8000
• server responds with a session UID
• client is responsible for connecting two more "channels", using the session UID as a correlation ID
• server decides that a session is completely ready if all 3 "channels" (i.e. sockets) participating in with a given session UID have been connected. If there's a timeout for this to happen, the server jots the whole session and closes any sockets that had been opened.

This is fairly easy, but requires a wire protocol to coordinate the sessions and channel "roles" for connections. I did initially elect not to implement this as a demo. Instead I thought it would be a nice exercise to learn Boost Asio, and implemented the back-channels (initiated from the server side) as in your original drawing.

The full code is on github: https://gist.github.com/sehe/9946161

Notes:

• there is a 'general' listener implementation that is used for both the server (port 8000) and "back-channels" (ports 8001,8002). See listener.hpp
• I opted for the stackless coroutine approach. This requires Boost Asio 1.54
• This approach has led to relative abuse of shared_ptr<>, at least in my view. The reason is that it's very beneficial if the coroutine (which is also the completion functor) is copyable without any issues. I could probably clean this up by making the class itself enable_shared_from_this and bind to shared_from_this instead. The benefit now is that there are (almost) no bind-expressions.

• Creating the 'back channels' is done in the server class which overrides on_accept:

  virtual bool on_accept(tcp::socket& socket) override
  {
      auto host = socket.remote_endpoint().address().to_string();
      // for now setting up the back-connections is all synchronous -
      // that might not work well in practice (scaling, latency) but...
      try
      {
          tcp::resolver resolver(socket.get_io_service());
          auto ep1 = resolver.resolve(tcp::resolver::query(host, "8001"));
          auto ep2 = resolver.resolve(tcp::resolver::query(host, "8002"));
  
          backsock1 = make_shared<tcp::socket>(socket.get_io_service());
          backsock2 = make_shared<tcp::socket>(socket.get_io_service());
  
          backsock1->connect(*ep1);
          backsock2->connect(*ep2);
  
          std::cerr << "on_accept: back channels connected for " << host << "\n";
      } catch(std::exception const& e)
      {
          std::cerr << "on_accept: '" << e.what() << "' for " << host << "\n";
          return false;
      }
  
      return base_type::on_accept(socket);
  }
  

• If on_accept fails (e.g. for our server cannot connect the back-channels) an error is returned on the "main" socket (initial connection) and the session is aborted

There are three programs:

• run_server (which listens on port 8000)

• run_client (which connects to port 8000 and listens on 8001,8002), and sends 1 message. You can observe how the the server responds by connecting on the back-channels and sending different messages on all three sockets.

• test (which combines the two):

  #include <boost/asio.hpp>
  #include <boost/thread.hpp>
  #include "server.hpp"
  #include "client.hpp"
  
  int main()
  {
      boost::asio::io_service svc;
  
      // start service on a separate thread
      boost::thread th([&svc] { 
              svc.post(demo::server(svc));
              svc.run(); 
              });
  
      boost::this_thread::sleep_for(boost::chrono::milliseconds(500)); // allow server to start accepting
  
      // post client traffic to the service
      std::cerr << "Starting a test client that sends a message...\n";
      demo::client client(svc, "localhost", "8000");
  
      // await interrupt (or new connections)
      th.join();
  }
  

The output of the last program looks like:

Starting a test client that sends a message...
on_accept: back channels connected for 127.0.0.1
listener 127.0.0.1:8000: accepting connection from 127.0.0.1:40999
listener 127.0.0.1:8000: 'hello world from demo client' received from 127.0.0.1:40999
listener 127.0.0.1:8001: accepting connection from 127.0.0.1:40132
listener 127.0.0.1:8002: accepting connection from 127.0.0.1:37970
listener 127.0.0.1:8001: 'We've received a request of length 29' received from 127.0.0.1:40132
listener 127.0.0.1:8002: 'We're handling it in void demo::server::do_back_chatter(const string&)' received from 127.0.0.1:37970
listener 127.0.0.1:40999: 'ECHO hello world from demo client' received from 127.0.0.1:8000