CODEWITHSUNDEEP
c#serializationnetwork-programmingthreadpool
Marko
Marko

Reputation: 5454

Multi-thread serialization in C#

Problem: I have large number of big messages to serialize and send over network. I would like to maximize performance, so I'm thinking about creating multiple threads for message serialization and one thread for sending the data. Idea is to dynamically determine number of threads for serialization based on network performance. If data is sent fast and serialization is bottleneck, add more threads to boost serialization. If network is slow, use less threads and stop completely if send buffer is full.

Any ideas how to do that?

What would be the best algorithm to decide if more or less threads are needed?

How to correctly concatenate serialization results from multiple threads?

Please answer to any of this questions? Thanks

Upvotes: 3

Views: 1450

Answers (2)

k rey
k rey

Reputation: 631

You can chunk the data into packets and place the packets into a queue. The process that looks at network performance would look at the queue and decide how many packets to send. The downside to this implementation is that you will need to assemble the packets on the receiving end where they may not be received in the proper order.

Upvotes: 1

Henk Holterman
Henk Holterman

Reputation: 273581

It can be treated as a Producer/Consumer problem, In Fx4 you can use a BlockingCollection.

But frankly I would expect the (network) I/O to be the bottleneck, not the serialization. You will have to measure.

Upvotes: 3

Related Questions