CODEWITHSUNDEEP
c#-3.0workflow
ltech
ltech

Reputation: 1

Trying to determine best design for this workflow - c# - 3.0

Input Server - files of type jpg,tif, raw,png, mov come in via FTP

Each file needs to be watermarked, if applicable, and meta data added to the file

Then each file needs to be moved to an orders directory where an order file is generated and then packaged as zip file and moved to processing server.

The file names are of [orderid_userid_guid].[jpg|tif|mov|png...]

As I expect the volume to grow I dont want to work on one file at a time and move it through the work flow. I would prefer multi threaded/asynchronous if possible..

Upvotes: 0

Views: 162

Answers (1)

Andy White
Andy White

Reputation: 88365

I might setup a message queuing and processing system for this.

One process/thread/service will monitor the FTP server, and when new files appear, will grab them and dump them into a queue (possibly MSMQ, or just a staging folder, etc.)

Another process monitors this queue, and when a file appears, it grabs it and does watermarking/metadata/etc., then drops it in another queue/folder.

Another process monitors this queue and grabs new files for zipping. After zipping, drop in another queue.

...and so on.

You can setup "work dispatchers" at the end of each queue to grab files and dispatch them to however many worker threads you want.

You don't necessarily have to split it out into this many separate processes and queues - that's up to you to decide. The "queues" can be implemented in a number of different ways as well. You could look at MSMQ as a start, but you might also consider just moving files between folders, etc. WCF and Windows Workflow Foundation might be good technologies to look at first.

Upvotes: 1

Related Questions