Reputation: 55
I want to design a windows form application with having my data (approx. 70k values) in an array to transfer it to a csv file in minimal time, the csv file path and name to be given by the windows form application.
The data transfer is done only when first button named start is pressed and it should stop when second button stop is pressed irrespective of number of values.
Here is the code:
public static void Write(double[] data, string outputPath)
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < data.GetLength(0); i++)
{
sb.AppendLine(string.Join(",", data[i]));
}
File.WriteAllText(outputPath, sb.ToString());
}
and this is the function I am using which has the varibale dscaledData which is an array. I want to take values from this array and store it in a csv file
Write(dScaledData, @csvadd);
The @ part shows the csv path with name The data values stored in dscaledData variable is very large approx. 70k
Upvotes: 2
Views: 155
Reputation: 54433
You want to use the stop button to somehow break your write loop. But as it is, it will keep running and indeed block the ui thread. So you need to run the write loop in a different thread..
There are various way to do it in .NET, e.g. a Task.Factory
, threads
or a BackgroundWorker
.
Here is a simple but effective implementation using a Task.Factory
:
We create a class level variable stopped
which we set and reset from the Buttons
. There are other, more involved ways to cancel a Task
, see here, but this is good enough for the purpose..
For my tests I also create dummy data..
private void cb_Stop_Click(object sender, EventArgs e)
{ stopped = true; // this will be checked in our output loop }
private void cb_Start_Click(object sender, EventArgs e)
{
stopped = false;
// create my test data..
List<double> data = new List<double>();
for (int i = 0; i < 10000000; i++) data.Add((i+ 1d) / i * 1d );
string filename = "D:\\xxxxx.txt";
// this is an optional callback to provide feedback.
// It obviuosly slows things down greatly..
Action<int> callback = (value) => st_lines.Invoke(new Action(()
=> st_lines.Text = value + " lines written.."));
// now we start the write loop in another task..
// ..passing in our data and (optinally) the callback
Task myFirstTask = Task.Factory.StartNew(()
=> Write(data.ToArray(), filename, callback));
}
static bool stopped = true; // our flag
public static void Write(double[] data, string outputPath, Action<int> aCallback)
{ // your write loop
StringBuilder sb = new StringBuilder();
for (int i = 0; i < data.GetLength(0); i++)
{
if (stopped) break; // our new test
sb.AppendLine(string.Join(",", data[i]));
aCallback( i ); // optional callback
}
File.WriteAllText(outputPath, sb.ToString());
}
A few notes about the timing situation:
Adding 70k lines to a StringBuilder
should not take a lot of time. Look at my code: I had to crank the test data up to 10M lines to even be able to press the stop button on my machine!
Maybe the creation of the data is the bottleneck or maybe it is the writing to disk..? Or maybe there is no bottleneck?? You should understand this before you decide on an implementation!
If other parts are identified as the bottlenaeck you would have to change the code accordingly to break out of the other loops..
I have added the callback mechanism so you can see how you can bring information about the progress of the task back to the UI thread. It should help your users decide on whether stopping the task makes sense..
But adding the callback slows things down a lot, so instead of calling it for every single line you could do it like this: if (i % 1000 == 0 ) aCallback(i);
and only update the progress label (or a ProgressBar
, of course) every 1000 lines.
Upvotes: 3
Reputation: 1691
You will want to use the BackgroundWorker for this. It is exactly what you Need.
Do not use File.WriteAllText, but use File.WriteLine in the worker-thread.
In my opinion 70k lines is not very much and if your amount of data doesnt grow substantially in the future, a human will not be able to start/stop unless you artificially pause the writer thread with a Thread.Sleep()
Upvotes: 1