StreamWriter can't be closed while its wrapped Pipe isn't drained?

Question

I have a simple server-client application, using a named pipe. I use a StreamWriter within the server, and a StreamReader within the client. The StreamWriter doesn't get disposed as long as the client process doesn't read from the pipe (that is, doesn't read from the StreamReader, wrapping the pipe). I'd like to understand why.

Here are the details:

This is the server:

using System;
using System.IO;
using System.IO.Pipes;

class PipeServer
{
    static void Main()
    {
        using (NamedPipeServerStream pipeServer =
            new NamedPipeServerStream("testpipe"))
        {
            Console.Write("Waiting for client connection...");
            pipeServer.WaitForConnection();
            Console.WriteLine("Client connected.");

            try
            {
                StreamWriter sw = new StreamWriter(pipeServer);
                try
                {
                    sw.WriteLine("hello client!");
                }
                finally
                {
                    sw.Dispose();
                }
                // Would print only after the client finished sleeping
                // and reading from its StreamReader
                Console.WriteLine("StreamWriter is now closed"); 
            }
            catch (IOException e)
            {
                Console.WriteLine("ERROR: {0}", e.Message);
            }
        }
    }
}

and here's the client:

using System;
using System.IO;
using System.IO.Pipes;
using System.Threading;

class PipeClient
{
    static void Main(string[] args)
    {
        using (NamedPipeClientStream pipeClient =
            new NamedPipeClientStream(".", "testpipe"))
        {
            Console.Write("Attempting to connect to pipe...");
            pipeClient.Connect();
            Console.WriteLine("Connected to pipe.");

            using (StreamReader sr = new StreamReader(pipeClient))
            {
                Thread.Sleep(100000);

                string temp = sr.ReadLine();
                if (temp != null)
                {
                    Console.WriteLine("Received from server: {0}", temp);
                }
            }
        }
    }
}

Notice the Thread.Sleep(100000); in the Client: I added it to make sure that the StreamWriter is not being disposed in the Server as long as the client process is sleeping, and the server won't execute Console.WriteLine("StreamWriter is now closed");. Why?

EDIT:

I cut off the previous information which in second-thought I guess is probably irrelevant. I'd also like to add that - thanks to Scott in the comments - I observed this behavior happening the other way around: If the server writes, then sleep, and the client (tries to) read with its StreamReader - the reading isn't happening untill the server awakes.

SECOND EDIT:

The other-way-around-behavior I talked about in the first edit is irrelevant, it is a flush issue. I tried giving it some more trials, and came to a conclusion the Scott's right - the pipe can't be disposed if it isn't drained. Why, then? This also seems to be in contradiction to the fact that StreamWriter assumes it owns the stream, unless otherwise specified (see here).

Here are the added details to the code above:

In the server program, the try-finally now looks like this:

try
{
    sw.AutoFlush = true;
    sw.WriteLine("hello client!");
    Thread.Sleep(10000);
    sw.WriteLine("hello again, client!");
}
finally
{
    sw.Dispose(); // awaits while client is sleeping
}
Console.WriteLine("StreamWriter is now closed");

In the client program, the using block now looks like this:

using (StreamReader sr = new StreamReader(pipeClient))
{
    string temp = sr.ReadLine();
    Console.WriteLine("blah"); // prints while server sleeps

    Console.WriteLine("Received from server: {0}", temp); // prints while server is sleeping
    Thread.Sleep(10000);

    temp = sr.ReadLine();
    Console.WriteLine("Received from server: {0}", temp);
}

Answer 1

So the problem is down to the way Windows Named Pipes work. When CreateNamedPipe is called you can specify the output buffer size. Now the documentation says something like this:

The input and output buffer sizes are advisory. The actual buffer size reserved for each end of the named pipe is either the system default, the system minimum or maximum, or the specified size rounded up to the next allocation boundary.

The trouble here is the NamedPipeServerStream constructor passes 0 as the default output size (we can verify this using the source, or in my case just firing up ILSpy). You might assume this would create a "default" buffer size, as per the comment, but it doesn't, it literally creates a 0 byte output buffer. This means that unless someone is reading the buffer then any write to the pipe blocks. You can also verify the size using the OutBufferSize property.

The behaviour you're seeing is because when you dispose the StreamWriter it calls Write on the pipe stream (as it has buffered the contents). The write call blocks, so Dispose never returns.

To verify this we'll use WinDBG (as VS just shows the blocking thread on the sw.Dispose call).

Run the server application under WinDBG. When it's blocking pause the debugger (hit CTRL+Break for example) and issue the following commands:

Load the SOS debugger module: .loadby sos clr

Dump all running stacks with interleaved managed and un-managed stack frames (might need to run this twice because of a dumb bug in sos.dll): !eestack

You should see the first stack in the process looks something this (heavily edited for brevity):

Thread   0
Current frame: ntdll!NtWriteFile+0x14
KERNELBASE!WriteFile+0x76, calling ntdll!NtWriteFile
System.IO.Pipes.PipeStream.WriteCore
System.IO.StreamWriter.Flush(Boolean, Boolean))
System.IO.StreamWriter.Dispose(Boolean))

So we can see that we're stuck in NtWriteFile which is because it can't write out the string to a buffer of size 0.

To fix this issue is easy, specify an explicit output buffer size using the one of the NamedPipeServerStream constructors, such as this one. For example this will do everything the default constructor will do but with a sensible output buffer size:

new NamedPipeServerStream("testpipe", PipeDirection.InOut, 1, PipeTransmissionMode.Byte, PipeOptions.None, 0, 4096)