BackgroundWorker freezes my UI and BeginInvoke seems to go turtle speed

Question

I have to do a very seemingly easy task: 1) list all files in a directory (and its sub-directories), 2) show them all in a multiline textbox and then 3) do some stuff inside each file. I'm stuck at 2) due to 2 problems, here's what I have:

1. Form1.cs is where I manage the UI and start a BackgroundWorker that runs the Logic.cs's main function
2. DependencyMapper.cs is... well, where I do the folders/files thing (in Fetch()) and call a Form1 method that populates each line (the current file's name) into a Form1's textbox using BeginInvoke.

Less talk and more code. This is the skinny, awfully working version of my code:

Form1.cs

public partial class Form1 : Form
{
    public DependencyMapper dep;
    BackgroundWorker bwDep;

    public Form1()
    {
        // I read here in SO to try put the BW stuff here don't know why, but hasn't helped.
        InitializeComponent();

        bwDep = new BackgroundWorker();
        bwDep.DoWork += bwDep_DoWork;
        bwDep.RunWorkerCompleted += bwDep_RunWorkerCompleted;
    }

    private void button1_Click(object sender, EventArgs e)
    {
            bwDep.RunWorkerAsync();
    }
    void bwDep_DoWork(object sender, DoWorkEventArgs e)
    {
        dep.Fetch(extensions);
    }
    public void SendBack(string msg) // To receive Fetch()s progress
    {
        textBox2.BeginInvoke(new Action(() =>
        {
            textBox2.Text += msg + "\r\n";
            textBox2.SelectionStart = textBox2.Text.Length;
            textBox2.ScrollToCaret();
        }));
    }
}

DependencyMapper.cs

public class DependencyMapper
{
    private Form1 form;
    public DependencyMapper(Form1 form1)
    {
        this.form = form1;
    }
    public void Fetch()
    {
        DirectoryInfo folder = new DirectoryInfo(form.Texto1);
        FileInfo[] files = folder.GetFiles("*.*", SearchOption.AllDirectories);
        for (int i = 0; i < files.Length; i++)
        {
            form.SendBack(files[i].FullName); // Kind of talking back to the UI through form's reference and SendBack method which uses BeginInvoke.
        }
    }
}

So, does my app work? Yes, but two huge problems I can't solve:

1. It freezes the UI (wtf lazy BackgroundWorker?). Not completely because the textbox is adding each file one by one but like it's supposed to but I can't move the window around or click any buttons.
2. It's veeery slow. Definitely I'm doing something wrong. My app currently fills the textbox at a rate of 10 files per second approx. And I'm coding it to find particular snippets of text in hundreds of files...omg

PS: Before using BackgroundWorker I was using Thread: the UI didn't freeze at all but the textbox populating ratio was as slow. That's why I decided to venture with BackgroundWorker which only brought problem #1.

Thanks.

Answer 1

You are fire-hosing the UI thread, producing results far faster than the UI can display them. It might start off reasonably well, but gets worse and worse over time. You force the TextBox to reallocate memory to find room for the appended string, copy the original text and append the new text. This will seriously start to drag once the textbox contains a megabyte of text, give or take. The same kind of problem that the System.String class has. Which has StringBuilder as the fix, TextBox doesn't have anything similar.

The usual symptoms are that the UI thread will start burning 100% core. At some critical point it goes completely catatonic, not repainting the UI anymore and getting unresponsive to input. Because every time it updated the Text property, it has yet another delegate to invoke. There's no time left to do the lower priority jobs, like painting and emptying the message queue. The invoke queue itself starts falling behind, gather more and more invoke requests that are waiting to be processed. In extreme cases that will crash your program with OutOfMemoryException but that takes a very long time. Even when the background worker finishes, the UI thread is busy for quite a bit longer after that, trying to work down the backlog.

The UI is in general non-functional, even before it hits the wall. The user is just staring at a blindingly fast scrolling textbox that does not permit actually reading anything. Using a textbox in itself is very counter-productive, it doesn't make any sense for the user to edit the list.

Clearly you need to do this differently. At least gather a whole bunch of files in a StringBuilder before you invoke. That will give the TextBox a break having to reallocate so often. It never makes sense to invoke more often than once every 50 msec, that's about as fast as a human eye can see changes without them turning into a blur. Most programs that do this just show a sample of the files getting iterated so that the user has some feedback on progress. In which case it is entirely reasonable to gather all the data in a StringBuilder and not update the TextBox until the file iteration completes.