Async StreamWriter Loop stops writing when BufferBlock is empty

Question

I recently converted my LogManager class from a static to singleton class, for easier binding in WPF (Though I think this problem also existed before this). After checking every if everything worked, I noticed something strange. The SaveLoop task I use for writing my logs to a file and sending them to the view didn't quite work as I thought it did.

In the view they are all added almost instantly, and if I close the application the SaveLoop task gets canceled and I can view the logs in the created file. However when the application crashes (or I press the stop button in Visual Studio), the file with the logs contains approx 8-10 logs less than the actual count of logs in the view.

What I noticed is the writer stops writing (in the middle of a message) when the BufferBlock contains no more logs, and only continues when there's new logs added.

Is it because the BufferBlock halts every async operation in the task because it has no more items to provide?

This is the output in the console when I run the application (The LogStatus message is another looping task that detects is anything has changed in the last 1000milliseconds):

12345678
12345678
12345678
1Setting LogStatus=Idle

I really have no clue how to handle this problem, if anybody could help me that would be greatly appreciated!

---

SaveLoop Task (Console output is for debugging purposes only):

private async Task SaveLoop(CancellationToken cancel)
{
    string logPath = "logs\\";
    string logFile = _startTime.ToString("yyyy-MM-dd_HH-mm-ss") + ".log";
    string fullPath = Path.Combine(logPath, logFile);

    if (!Directory.Exists(logPath))
        Directory.CreateDirectory(logPath);

    using (FileStream fileStream = new FileStream(fullPath, FileMode.Append, FileAccess.Write, FileShare.Read, 8192, useAsync: true))
    {
        using (StreamWriter writer = new StreamWriter(fileStream))
        {
            while (true)
            {
                LogMessage logMessage = null;

                try
                {
                    Console.Write("1");
                    logMessage = await _logQueue.ReceiveAsync(cancel).ConfigureAwait(false);

                    Console.Write("2");
                    await writer.WriteAsync(String.Format("({0}) ", logMessage.LogID)).ConfigureAwait(false);
                    Console.Write("3");
                    await writer.WriteAsync(String.Format("[{0}][{1}] ", logMessage.Time.ToString("HH:mm:ss:fff"), logMessage.Level.ToString())).ConfigureAwait(false);
                    Console.Write("4");
                    await writer.WriteAsync(logMessage.Message).ConfigureAwait(false);
                    Console.Write("5");
                    await writer.WriteLineAsync(String.Format(" ({0} - {1})", logMessage.Method, logMessage.Location)).ConfigureAwait(false);

                    Console.Write("6");

                    AddLogToCollections(logMessage);

                    Console.Write("7");
                    _timeout = 10;

                    if (cancel.IsCancellationRequested)
                    {
                        Console.WriteLine("0");
                        break;
                    }

                    Console.WriteLine("8");
                }
                catch (TaskCanceledException tce)
                {
                    Console.WriteLine("Task canceled!");
                    Console.WriteLine("{0} {1}", tce.Message, tce.Source);
                    break;
                }
                catch (Exception e)
                {
                    Console.WriteLine("{0} {1}", e.Message, e.Source);
                    Console.WriteLine("{0}", e.StackTrace);
                    if (logMessage != null)
                    {
                        Console.WriteLine("Log: {0}", logMessage.ToString());
                    }
                }
            }
        }
    }
}

Answer 1

This has nothing to do with the Dataflow block. The reason this happens is because StreamWriter and FileStream use buffering to be more efficient. So, what you should do is to add await writer.FlushAsync() at the end of your loop, which empties the buffers and writes the data to the disk.