TCP Server architecture

Question

I'm building a TCP server in C++ which is multi-threaded and each client has their own thread and I have a question which I was hoping someone could answer for me. I understand that there is no silver bullet to this type of question and there will probably be multiple ways to deal with it, i'm mostly looking for feedback on my thoughts to give myself better clarity of the situation and how to manage it.

So the basic setup I have right now is something like this...

1. Class A listens for incoming connections
2. When a connection is made it instantiates a new class (Class B) and spawns a new thread for it
3. Class A goes back to listening for incoming connections whilst Class B manages that one particular connection.

I was wondering what the best way to send messages to and from the clients would be from a architectural point of view.

My initial thoughts were to pass in a reference of all the clients to each client thread and then each client thread has a "SendTo" function which can then send a message to that specific user.

But after awhile of thinking I started wondering "Would it be good design to pass a reference of all the clients to each thread?" and I wasn't really sure, which I guess is why i'm here!

Anyway, so I started thinking about having a "SendTo" callback function to Class A which would pass the to/from ID from the Class B thread trying to send a message. If I did this I figured i'd need a mutex at the top of this function as it would be a shared resource, correct? But after that I started thinking "If there's alot of messages going back and forth and they all have to go through this same function wouldn't that be inefficient and slow?"

So I'm just a little bit confused as to what the best design approach would be, there's probably a better way to deal with this that I don't even know of which is another reason why I wanted to write this post.

Answer 1

I assume we're talking about a chat style applications, where clients can send messages to each other. There can be private messages, group messages, or broadcast messages. The important part is that these messages happen asynchronously, i.e. a message sent by one client happens totally independently from anything that happens in another client.

It is reasonable to have a specialized object that deals with such events. In addition to the main event type, "client sends message for other(s)", it can deal with other events: "client connects/disconnects", "client joins/leaves group" etc. Client handlers may subscribe to these events.

One can envision a system with many event queues: a queue for each client's private messages, a group queue for each client group, and a one broadcast queue that serves all clients. Each client handler is subscribed to as many or as few of those as it needs.

When a handler receives a message from a client via TCP, it simply pushes it to an appropriate queue (perhaps many copies, one for each addressee; or make a separate queue-manager thread replicate messages as needed).

A queue can be made lock-free (e.g. boost), which means nobody is blocking anybody else.

Answer 2

You won't get around protecting against race conditions in either case:

First, you need some data structure to store all the connections existing so far. And then, there are the sockets themselves.

I would recommend a two-level locking:

While it is no problem if the common data structure is read concurrently, you get into deep trouble if A modifies it while any B thread is reading it. Have a look at shared mutexes, this is exactly what you need for...

Any single socket cannot be accessed for writing other than exclusively. So provide each one with an ordinary single access mutex of its own. What you get then in the end is:

A, new client:

• create a new socket
• acquire socket list mutex, exclusive lock
• insert new socket into data structure
• release lock

B, sending common message:

• acquire socket list mutex, shared lock
• iterate over socket list, for each socket:
  • aquire socket's mutex, exclusive lock (only one available...)
  • write message to socket
  • release socket's mutex
• release socket list mutex

This minimizes the times any exclusive lock actually needs to be held (socket and socket's mutex are managed by class B, of course).

It is just a metter of taste if you implement this as class B receiving a reference to the data structure and the shared mutex (preferrably then, however, a separate class containing both and providing a single, consistent interface) or if you simply provide a callback (or a reference to class A) which is called by any B. The former provides better reusability (especially, if you implement the new class as a template), latter better separation (B cares for one client, nothing more) and might require a little less code (no separate class; still you could implement a such for reusability).