How can I calculate the time that a specific operation (pre-defined function) needs to finish?

Question

I always wondered how all the programs that has a progress bar can know almost exactly how much time it takes for an operation to finish (and the whole processing of the program's work), and thus being able to map that with the progress bar.

In C#, I always face difficulties when dealing with a progress bar. I came to a conclusion that there's no generic solution to this, is there?

I'm not asking this:

"For example: how would you know how much time File.ReadAllBytes(path) takes to read a file? Simple, start a stopwatch before it, and stop and read the time after it. But that's just on your computer, on your CPU, on your disk! - The reading of 1MB will mostly be different from your machine and others"

I'm asking how can I know how much time a thing needs to finish before even going inside of it? I mean, I have to know how much time File.ReadAllBytes() takes, before it gets executed so that the progress bar could step accordingly DURING the method's execution.

One dumb way of doing this, is to do it twice! Start the operation, calculate the time, then run it again, but this time you step the progress bar (lol)

I don't know if complexity has anything to do with this. I use big-O when I write my own methods and functions, not when dealing with a pre-defined function.

EDIT: Just an example: I made a program that XORes a file and overwrite it with the XORed form. Now this involves 1-Reading the file's bytes to a byte array. 2-XORing them. 3-Writing them back again to the file. Now if I wanna use a progress bar, how can I go about knowing how much time do those operations take, so that I could make the progress bar increase accordingly?

My solution to this was to use a global variable, (like prgBar), assign the XORing to a separate thread, and incrementing "prgBar" in that thread each time I XOR a byte from the byte array I read from the file, and then in the main thread, I used a timer, each tick I did: prgressBar.Value = prgBar I'm facing problems with this, it's not even accurate, it might start late.

Answer 1

Actually you should read the file in batches (eg. 20KB each time ) so you can increment the progress bar after each batch

Answer 2

The quick answer is "They don't". Haven't you ever seen progress bars which jump from "20 minutes left" to "5 minutes left" in the course of 20 seconds? Or ones which say "0:00 left" and just sit there?

---

The better answer is "They estimate". There's two forms of this I can think of, but I haven't done any extensive research. They both use past performance to estimate future progress, though.

The first form works for a series of discrete tasks of similar complexity but arbitrary length, such as file copying/deletion. You start off with a low base estimate or a "Calculating..." message, and then as each file is finished you make a new estimate of how long it will take based on how long it's taken to copy/delete all the files so far. For example, if you're deleting 2000 files, and the first 5 have deleted in {400, 200, 900, 100, 400 ms}, then you know that you can delete an average of 5 files in 2000ms, which means you can delete 2000 files in 800000ms or 800 seconds, and display a progress bar to your user saying "13 minutes remaining" (798 seconds). It might not be accurate, but if you keep revising it after every few files, it will progress fairly smoothly.

The second form is when the length of tasks and their complexity are known (and greatly varies), but the speed is still unknown, such as with an installer. In this scenario you can make your estimate by running your installer many times in testing, and figuring out how different components' times relate. This could be as complex as comparing individual steps to derive a formula, or as simple as just assigning percentages of install time to each stage. Then, when the user does their install, you know that the first 2% worth of components took X seconds duration, so estimate 50*X seconds left. This is why installers often seem to hang or jump around - they're waiting on something that's unexpectedly taking much longer than usual, or they discovered that they didn't need to perform a step on your system.

---

In the end, it's all just an estimate. Percentages could be hard ("I'm on step 1/5, so show 20% complete"), but durations can never be. See Jbecwar's answer for the theory behind this.

Answer 3

You have to calculate the completeness of the operation yourself and it's not always possible to do accurately. For reading a file you could do something like get number of lines, count how many you've read, then do current/total to display progress. This actually works a lot better for more complex operations.

Say you have an array of a 1000 objects who's data needs to be processed. You're doing this in a for loop so the progress bar can simply display i/array.length. The ReadAllBytes example is no good because there is no way of taking the progress of that task. The .NET method will return all the bytes, so you can only know when the task starts and completes, not to mention the task will be so quick that a progress bar is unnecessary.

Answer 4

What your describing is the halting problem. Long story short you can't tell when an arbrtary process will end, but you can make a guess.

For example lets say you have 1MB to transfer. If you transfer one byte at a time and then update the progress bar after each byte, then it will look smooth. Could also display a time estimate based on the past. For example if it took me 1 sec to move 10% of the 1MB then in 9 seconds I should be done.

Hope that helps.

Answer 5

You are correct, there is no once-size-fits-all for progress bars unless you consider using the progress bar as an "I'm still processing" bar. Could have the progress bar update from 0 to 100(max) over and over again.

There are times where you will know how you are progressing, and can report it back to the user.

Lets take your File.ReadAllBytes(path) example:

You should be able to get the size of your file. You can also determine how many bytes you have read so far by incrementing a variable. By doing some simple math, you can take bytesRead/bytesInFile to get a % of your completion. Report this back to the user on by a timer that ticks every 500ms, and you should have progress bar with proper feedback.