CODEWITHSUNDEEP
c#-4.0loggingconsole-applicationprogress
tucaz
tucaz

Reputation: 6684

Reporting progress on a million call process

I have a console/desktop application that crawls a lot (think million calls) of data from various webservices. At any given time I have about 10 threads performing these call and aggregating the data into a MySql database. All seeds are also stored in a database.

What would be the best way to report it's progress? By progress I mean:

I thought about logging all of them somehow and tailing the log to get the data. Another idea was to offer some kind of output to a always open TCP endpoint where some form of UI could read the data and display some aggregation. Both ways look too rough and too complicated.

Any other ideas?

Upvotes: 2

Views: 109

Answers (2)

Malachi
Malachi

Reputation: 3221

since it is a Console Application, just use Writeline, just have the application spit the important stuff out to the Console.

I did something Similar in an application that I created to export PDF's from a SQL Server Database back into PDF Format

you can do it many different ways. if you are counting records and their size you can run a tally of sorts and have it show the total every so many records..

I also wrote out to a Text File, so that I could keep track of all the PDFs and what case numbers they went to and things like that. that information is in the answer that I gave to the above linked question.

you could also write things out to a Text File every so often with the statistics.

the logger that Eric J. mentions is probably going to be a little bit easier to implement, and would be a nice tool for your toolbox.

these options are just as valid depending on your specific needs.

Upvotes: 1

Eric J.
Eric J.

Reputation: 150118

The "best way" depends on your requirements. If you use a logging framework like NLog, you can plug in a variety of logging targets like files, databases, the console or TCP endpoints.

You can also use a viewer like Harvester as a logging target.

When logging multi-threaded applications I sometimes have an additional thread that writes a summary of progress to the logger once every so often (e.g. every 15 seconds).

Upvotes: 1

Related Questions