IOCP thread vs threadpool to process messages

Question

When you're dealing with socket IO using BeginReceive/EndReceive, the callback is invoked by an IOCP thread.

Once you're done receiving you need to process the data.

• Should you do it on the callback's calling thread?
• Or should you run the task using the ThreadPool.QueueUserWorkItem

Normally samples do the work on the callback thread, which is a little bit confusing.

If you're dealing with several hundred active connections, running the processing on the IOCP thread ends up with a process with hundreds of threads. Would ThreadPool help limiting the number of concurrent threads?

Answer 1

IOCP threads has meant for serving IO operations. if your work executed by iocp callback can be long running or incures locks or other facilities that my block execution, you'd better hand it of to a worker thread.

Answer 2

I don't know that there's a generic 'right answer' for everyone - it will depend on your individual use case.

These are some things to consider if you do consider going down the ThreadPool path.

Is Out of Order / Concurrent message processing something you can support?

If Socket A receives messages 1, 2, and 3 in rapid succession - they may end up being processed concurrently, or out of order.

.NET has per-CPU thread pools, if one CPU runs out of work, it may 'steal' tasks from other CPUs. This may mean that your messages might be executed in some random order.

Don't forget to consider what out-of-order processing might do to the client - eg if they send three messages which require responses, sending the responses out of order may be a problem.

If you must process each message in order, then you'll probably have to implement your own thread pooling.

Do you need to be concerned about potential Denial of Service?

If one socket suddenly sends a swarm of data, accepting all of that for processing may clog up internal queues and memory. You may have to limit the amount of un-processed data you accept.