CODEWITHSUNDEEP
c#.netc#-4.0tcp
Travyguy9
Travyguy9

Reputation: 4344

Manage many TCP connections effectively and efficiently

I am curious if someone can give me any ideas or suggestions on how to manage a lot of TCP connections efficiently. I am talking around 1000 tcp connections (maybe more). The application managing all these connections needs to pull information from the client (other end of the connection) periodically. For example, maybe every 30 seconds. I would be using .NET 4.0 for this. Is there anything built in to help with this or a special way to structure everything to manage all these connections without making this application be soo bogged down that it is not useful?

Upvotes: 4

Views: 6708

Answers (6)

Len Holgate
Len Holgate

Reputation: 21644

Firstly, use the async methods as others have said. 1000 connections shouldn't be a problem with reasonable hardware and a sensible design.

I'd have a collection of these connections protected by a single writer multiple reader lock. I'd probably have a timer per connection to deal with the periodic nature of the polling. I'd fire off the request to the set top box when the timer fires and then allow the async reads on that connection to accumulate the response and then process it. Ideally I'd try and avoid accessing the connection collection and store connection related data so that it can be accessed without locking from the read completions.

I blog about supporting 'lots' of concurrent connections here: http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-or-more-concurrent-tcp-connections---part-2---perf-tests-from-day-0.html the important thing, IMHO, is to make sure you are testing for this kind of load from the very start so that you can quickly spot when you add poor designs decisions that don't scale.

Upvotes: 0

ChrisLively
ChrisLively

Reputation: 88092

Just to add a "minor" thing to the already great answers here.

One often overlooked area is the capabilities of the network cards themselves. Make sure the network adapter in your server supports TCP Offloading. For that matter, pay for a nice one and verify that the drivers are the latest/greatest.

A TCP Offloading engine built into the adapter works at a lower level than your code does and will handle all of the work of shuffling data to/from the clients. This can have a huge performance and scalability impact on your software.

Upvotes: 1

Stephen Cleary
Stephen Cleary

Reputation: 457302

Handling thousands of connections is not difficult, as long as you only use the asynchronous APIs. The "one thread per connection" concept not only doesn't scale, but it's (in the general protocol case) wrong. Note that the asynchronous APIs are different than using a ThreadPool.

Asynchronous programming takes a bit of time to catch on to, but it's not really that complex. You'll find yourself having to manually track more state than you've done in synchronous programming.

Ensure your protocol classes can handle partial receives of any size (which requires additional state), and that you have one protocol instance per connection.

Any socket errors should result in the connection being closed and the state cleaned up.

And log. Everything. TraceSource is your friend; learn how to use the .NET tracing which can be turned on (even in production) by editing app.config.

Upvotes: 3

Aliostad
Aliostad

Reputation: 81700

I cannot remember any built-in support for this particular scenario. But basically here is how I would do it and generally known as a good practice:

1) Have a thread pool (small in size - perhaps around 5) responsible for connections. This would read from a queue of jobs (open or close) and manage opening and closing connections.

2) Have a thread pool (medium in size) for managing communications and operations. I think a pool of 50 should be OK.

3) You will have one thread responsible for orchestrating the whole operation but this should not be bugged down with any operation and only queue operations to other threads.

Upvotes: 2

Andrew Barber
Andrew Barber

Reputation: 40160

I can give you one quick bit of insight:

Do not even think about spawning a thread per-connection. Use the framework's built-in async/task frameworks instead to manage the connections. That is; let the thread pool do it.

Also, be careful to watch closely what your threads are doing; Be sure you can't ever be reading/writing a connection's stream from multiple threads at once. But at the same time, be careful with how you use locking mechanisms to do so, so you don't end up with a process with 1000 active connections, and a thread pool full of threads waiting to acquire locks that they will never get.

Upvotes: 7

PMC
PMC

Reputation: 4766

Using an Asynchronous Server Socket

Upvotes: 0

Related Questions