CODEWITHSUNDEEP
c#ftpwebrequestbuffering
Ian
Ian

Reputation: 4457

If I read from a stream from WebRequest.GetResponseStream am I reading from memory or the network

I have the following code that downloads a file from an FTP server:

var req = (FtpWebRequest)WebRequest.Create(ftp_addr + file_path);
req.Credentials = new NetworkCredential(...);
req.Method = WebRequestMethods.Ftp.DownloadFile;

using (var resp = req.GetResponse())
using (var strm = resp.GetResponseStream())
using (var mem_strm = new MemoryStream())
{
    //The Web Response stream doesn't support seek, so we have to do a buffered read into a memory stream
    byte[] buffer = new byte[2048];
    int red_byts = 0;
    do
    {
        red_byts = strm.Read(buffer, 0, 2048);
        mem_strm.Write(buffer, 0, red_byts);
    }
    while (strm.CanRead && red_byts > 0);

    //Reset the stream to position 0 for reading
    mem_strm.Position = 0;
    //Now I've got a mem stream that I can use
}

Since the raw stream returned from "GetResponseStream" cannot be read or sought (cannot perform a seek on it). It seems to me that this code is actually performing a request to the FTP server for the next chunk of the file and copying it into memory. Am I correct, or is the entire response downloaded when you can GetResponseStream?

I just want to know so I can correctly apply awaits with ReadAsync calls in asynchronous methods that make use of FTP downloading. My intuition tells me to change the line:

red_byts = strm.Read(...);

to

red_byts = await strm.ReadAsync(...);

The documentation of WebRequest.GetResponseStream doesn't seem to specify, nor does the documentation for FtpWebRequest.GetResponseStream.

Upvotes: 1

Views: 428

Answers (1)

Stephen Cleary
Stephen Cleary

Reputation: 456887

The raw stream should be a network stream; to verify this, check out strm.GetType().Name.

It seems to me that this code is actually performing a request to the FTP server for the next chunk of the file and copying it into memory. Am I correct, or is the entire response downloaded when you can GetResponseStream?

Neither.

It is not sending separate requests for each call to Read/ReadAsync; rather, there is only one request, and the stream represents the body of the one response.

It is also not downloading the entire response before returning from GetResponseStream. Rather, the stream represents the download. Some buffering is going on - as the server is sending data, the network stack and the BCL are reading it in for you - but there's no guarantee that it's download by the time you start reading the stream.

I just want to know so I can correctly apply awaits with ReadAsync calls in asynchronous methods that make use of FTP downloading. My intuition tells me to [use async]

Yes, you should use asynchronous reads. If some data is already buffered, they may complete synchronously; otherwise, they will need to wait until the server sends more data.

Upvotes: 1

Related Questions