CODEWITHSUNDEEP
c#asynchronousstreamstreamwritermemorystream
Robert Hegner
Robert Hegner

Reputation: 9396

Return stream immediately and then write to stream asynchronously

In my current code I have a method like this to read data from a device (pseudo code):

public async Task<string> ReadAllDataFromDevice()
{
    var buffer = "";
    using (var device = new Device())
    {
        while(device.HasMoreData)
        {
            buffer += await device.ReadLineAsync();
        }
    }
    return buffer;
}

I then want to send all that data via the network to some receiver. The amount of data can be really large. So clearly the above design is not very memory-efficient since it requires to read all the data before I can start sending it to the network socket.

So what I'd like to have is a function that returns a stream instead. Something like this:

public async Task<Stream> ReadAllDataFromDevice()
{
    var stream = new MemoryStream();
    using (var device = new Device())
    using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
    {
        while(device.HasMoreData)
        {
            var line = await device.ReadLineAsync();
            await streamWriter.WriteLineAsync(line);
        }
        await streamWriter.FlushAsync();
    }
    return stream;
}

This returns a stream but it clearly does not solve my problem, because the stream is returned only after all the data has been read from the device.

So I came up with this:

public Stream ReadAllDataFromDevice()
{
    var stream = new MemoryStream();
    Task.Run(async () => {
        using (var device = new Device())
        using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
        {
            while(device.HasMoreData)
            {
                var line = await device.ReadLineAsync();
                await streamWriter.WriteLineAsync(line);
            }
            await streamWriter.FlushAsync();
        }
    });
    return stream;
}

Is this a good design? I'm especially concerned about thread-safety, lifetime of the stream object used in the lambda, and exception handling.

Or is there a better pattern for this kind of problem?

Edit

Actually I just came up with another design that looks much cleaner to me. Instead of having the ReadAllDataFromDevice() function returning a stream, I let the consumer of the data provide the stream, like this:

public async Task ReadAllDataFromDevice(Stream stream)
{
    using (var device = new Device())
    using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
    {
        while(device.HasMoreData)
        {
            var line = await device.ReadLineAsync();
            await streamWriter.WriteLineAsync(line);
        }
        await streamWriter.FlushAsync();
    }
}

Upvotes: 1

Views: 1786

Answers (1)

Robert Hegner
Robert Hegner

Reputation: 9396

This is the design I'm using now:

public async Task ReadAllDataFromDevice(Func<Stream, Task> readCallback)
{
    using (var device = new Device())
    {
        await device.Initialize();
        using (var stream = new DeviceStream(device))
        {
            await readCallback(stream);
        }
    }
}

The line-by-line device access is encapsulated in the custom DeviceStream class (not shown here).

The consumer of the data would look something like this:

await ReadAllDataFromDevice(async stream => {
    using (var streamReader(stream))
    {
        var data = await streamReader.ReadToEndAsync();
        // do something with data
    }
});

Upvotes: 1

Related Questions