CODEWITHSUNDEEP
c#smtp
Canacourse
Canacourse

Reputation: 1865

Good design for Transparent SMTP Proxy server in C#

If you were to design a transparent SMTP proxy in C# (.net 4) to meet the following initial requirements

Considering these factors broadly speaking how would your design look? Would you create Listener, Sender and logger concrete classes or something more abstract? And would you use callbacks, threads or processes and why?

Upvotes: 4

Views: 1237

Answers (2)

Morten Mertner
Morten Mertner

Reputation: 9474

It sounds to me like you want to design something that is future proof, but without immediately doing the full monty of data partitioning, clustering, etc.

Partitioning It is advisable to consider ways to partition your data load in advance. Initially you can apply partitioning logic that simply routes everything to the same destination, but this will allow you to easily divide the load when the need arises - and the ability to verify that it works in advance.

Queuing I would highly recommend a queuing solution, as it allows you to separate the workload of receiving messages and the actual sending of it. Queuing is also great for reliable messaging, in that you will want to guarantee once-only delivery. Look at some of the questions comparing MSMQ with Service Broker, as they serve different audiences and both of them have various caveats.

SMTP Most email servers allow you to deliver more than a single email over a single connection (and I don't just mean the same mail with multiple recipients). This can dramatically boost the number of emails you can push to the remote mail server, but depends on how it has been configured. If you didn't configure them or don't know the value allow, I'd recommend a probing strategy where you start out with trying to deliver two mails and log the result for the remote server. Try the double next time if it works and reduce to half if it fails. Sort of like how the TCP window increases when you transmit reliably.

Extensibility I wouldn't give this much thought. Achieving scalability and reliability is MUCH harder and hence much more important to get right. Extensibility is simply the ability to hook in additional steps along the way, and you can add the seams for doing that when the core system is in place, or when you start adding functionality that you feel should be optional yet built-in.

Upvotes: 0

RQDQ
RQDQ

Reputation: 15569

This is a non-trivial application. Some ideas that should help:

SMTP Scalability

In general, scaling network application means being able to scale out (as in more machines) rather than up (a bigger expensive machine). This means being able to have multiple servers be able to handle SMTP requests. Note that this will likely need to have support at the network level (routers that can distribute messages to an 'SMTP farm').

Yes, to make an SMTP scale and peform, you'll likely want to utilize multiple threads (likely from some sort of thread pool). Note that a multithreaded sockets implementation is not trivial.

In terms of processes, I think one process (likely a Windows Service) with multiple threads for each SMTP server is a good way to go.

Database Scalability

Keep in mind that the database can be a scalability bottleneck as well. To design for large loads, you would want to be able to horizontally scale your data tier as well. That means being able write to more than one db server. That leads to being able to report from a set of database servers (which is much more complicated than reporting from one).

SMTP Reliability

Is this a concern / requirement? If so, this is another reason for supporting a farm (well, if we have multiple server for reliability we might call it a cluster) of servers instead of just one. Note that the farm would have to have a way of letting the cluster know that it has failed (through some sort of heartbeat mechanism perhaps).

Database Reliability

To make the database reliable, you would have to do some clustering as well. This is neither cheap or trivial (but has been done a number of times with a number of database platforms).

Queuing

One way to handle surges in server load is to queue messages. This way, the server can keep passing messages through, but you're not waiting for the chain of extensible modules to finish their processing. Note that this adds another layer of complexity and a point of failure to the system.

Extensibility

One way to approach adding functionality such as database logging and attachment scanning is to add a chain of "MessageInsepctors" or "MessageHandlers". You would probably want to allow configuration of these in a particular order (e.g. virus scan before logging so you don't log infected items).

Another aspect to consider is which plug ins can block a message from passing through (such as a virus scanner) and a plug in that can execute after the message has passed (logging).

In terms of adding the plug in support, you could use something like MEF (Managed Extensibility Framework).

Reinventing the Wheel

Putting all of this functionality into place would take a considerable amount of development time. It might be cheaper / faster / easier to just purchase a solution off the shelf that does all of this for you (this problem has already been solved a number of times).

Upvotes: 7

Related Questions