C# Download big file from Server with less memory consumption

Question

I have a big file of memory size 42 mb. I want to download the file with less memory consumption.
Controller Code

public ActionResult Download()
{
    var filePath = "file path in server";
    FileInfo file = new FileInfo(filePath);
    Response.ContentType = "application/zip";                        
    Response.AppendHeader("Content-Disposition", "attachment; filename=folder.zip");                   
    Response.TransmitFile(file.FullName);
    Response.End(); 
}

alernative method tried with Stream

public ActionResult Download()
{           
    string failure = string.Empty;
    Stream stream = null;
    int bytesToRead = 10000;


    long LengthToRead;
    try
    {
        var path = "file path from server";
        FileWebRequest fileRequest = (FileWebRequest)FileWebRequest.Create(path);
        FileWebResponse fileResponse = (FileWebResponse)fileRequest.GetResponse();

        if (fileRequest.ContentLength > 0)
            fileResponse.ContentLength = fileRequest.ContentLength;

        //Get the Stream returned from the response
        stream = fileResponse.GetResponseStream();

        LengthToRead = stream.Length;

        //Indicate the type of data being sent
        Response.ContentType = "application/octet-stream";

        //Name the file 
        Response.AddHeader("Content-Disposition", "attachment; filename=SolutionWizardDesktopClient.zip");
        Response.AddHeader("Content-Length", fileResponse.ContentLength.ToString());

        int length;
        do
        {
            // Verify that the client is connected.
            if (Response.IsClientConnected)
            {
                byte[] buffer = new Byte[bytesToRead];

                // Read data into the buffer.
                length = stream.Read(buffer, 0, bytesToRead);

                // and write it out to the response's output stream
                Response.OutputStream.Write(buffer, 0, length);

                // Flush the data
                Response.Flush();

                //Clear the buffer
                LengthToRead = LengthToRead - length;
            }
            else
            {
                // cancel the download if client has disconnected
                LengthToRead = -1;
            }
        } while (LengthToRead > 0); //Repeat until no data is read

    }
    finally
    {
        if (stream != null)
        {
            //Close the input stream                   
            stream.Close();
        }
        Response.End();
        Response.Close();
    }
    return View("Failed");
}

due to size of the file, it is consumpting more memory which leads to performance issue.
After checking in iis log, the download process is taking 42 mb and 64 mb each respectively.
Thanks in advance

Answer 1

A better option would be to use FileResult instead of ActionResult:

Using this method means you don't have to load the file/bytes in memory before serving.

public FileResult Download()
{
     var filePath = "file path in server";
     return new FilePathResult(Server.MapPath(filePath), "application/zip");
}

Edit: For larger files FilePathResult will also fail.

Your best bet is probably Response.TransmitFile() then. I've used this on larger files (GBs) and had no issues before

public ActionResult Download()
{
   var filePath = @"file path from server";

    Response.Clear();
    Response.ContentType = "application/octet-stream";
    Response.AppendHeader("Content-Disposition", "filename=" + filePath);

    Response.TransmitFile(filePath);

    Response.End();

    return Index();
}

From MSDN:

Writes the specified file directly to an HTTP response output stream, without buffering it in memory.

Answer 2

There is the Rizwan Ansari post that worked for me:

There are situation when you need to provide download option for a big file located somewhere on server or generated at runtime. Below function could be used to download files of any size. Sometimes downloading big file throws exception OutOfMemoryException showing “Insufficient memory to continue execution of the program”. So this function also handle this situation by breaking down file in 1 MB chunks (can be customized by changing bufferSize variable).

Usage:

DownloadLargeFile("A big file.pdf", "D:\\Big Files\\Big File.pdf", "application/pdf", System.Web.HttpContext.Current.Response);

You can change "application/pdf" by the right Mime type

Download Function:

public static void DownloadLargeFile(string DownloadFileName, string FilePath, string ContentType, HttpResponse response)
    {
        Stream stream = null;

        // read buffer in 1 MB chunks
        // change this if you want a different buffer size
        int bufferSize = 1048576;

        byte[] buffer = new Byte[bufferSize];

        // buffer read length
        int length;
        // Total length of file
        long lengthToRead;

        try
        {
            // Open the file in read only mode 
            stream = new FileStream(FilePath, FileMode.Open, FileAccess.Read, FileShare.Read);

            // Total length of file
            lengthToRead = stream.Length;
            response.ContentType = ContentType;
            response.AddHeader("Content-Disposition", "attachment; filename=" + HttpUtility.UrlEncode(DownloadFileName, System.Text.Encoding.UTF8));

            while (lengthToRead > 0)
            {
                // Verify that the client is connected.
                if (response.IsClientConnected)
                {
                    // Read the data in buffer
                    length = stream.Read(buffer, 0, bufferSize);

                    // Write the data to output stream.
                    response.OutputStream.Write(buffer, 0, length);

                    // Flush the data 
                    response.Flush();

                    //buffer = new Byte[10000];
                    lengthToRead = lengthToRead - length;
                }
                else
                {
                    // if user disconnects stop the loop
                    lengthToRead = -1;
                }
            }
        }
        catch (Exception exp)
        {
            // handle exception
            response.ContentType = "text/html";
            response.Write("Error : " + exp.Message);
        }
        finally
        {
            if (stream != null)
            {
                stream.Close();
            }
            response.End();
            response.Close();
        }
    }

Answer 3

I had similar problem but I didn't have file on local disk, I had to download it from API (my MVC was like a proxy). The key thing is to set Response.Buffer=false; on your MVC Action. I think @JanusPienaar's first solution should work with this. My MVC action is:

public class HomeController : Controller
{
    public async Task<FileStreamResult> Streaming(long RecordCount)
    {
        HttpClient Client;
        System.IO.Stream Stream;

        //This is the key thing
        Response.Buffer=false;

        Client = new HttpClient() { BaseAddress=new Uri("http://MyApi", };
        Stream = await Client.GetStreamAsync("api/Streaming?RecordCount="+RecordCount);
        return new FileStreamResult(Stream, "text/csv");
    }
}

And my test WebApi (which generates the file) is:

public class StreamingController : ApiController
{
    // GET: api/Streaming/5
    public HttpResponseMessage Get(long RecordCount)
    {
        var response = Request.CreateResponse();

        response.Content=new PushStreamContent((stream, http, transport) =>
        {
            RecordsGenerator Generator = new RecordsGenerator();
            long i;

            using(var writer = new System.IO.StreamWriter(stream, System.Text.Encoding.UTF8))
            {
                for(i=0; i<RecordCount; i++)
                {
                    writer.Write(Generator.GetRecordString(i));

                    if(0==(i&0xFFFFF))
                        System.Diagnostics.Debug.WriteLine($"Record no: {i:N0}");
                    }
                }
            });

            return response;
        }

        class RecordsGenerator
        {
            const string abc = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
            char[] Chars = new char[14];//Ceiling(log26(2^63))

            public string GetRecordString(long Record)
            {
                int iLength = 0;
                long Div = Record, Mod;

                do
                {
                    iLength++;
                    Div=Math.DivRem(Div, abc.Length, out Mod);
                    //Save from backwards
                    Chars[Chars.Length-iLength]=abc[(int)Mod];
                }
                while(Div!=0);

                return $"{Record} {new string(Chars, Chars.Length-iLength, iLength)}\r\n";
            }
        }
    }
}

If RecordCount is 100000000, the file generated by TestApi is 1.56 GB. Neither WebApi nor MVC consumes so much memory.

Answer 4

Try setting the Transfer-Encoding header to chunked, and return an HttpResponseMessage with a PushStreamContent. Transfer-Encoding of chunked means that the HTTP response will not have a Content-Length header, and so the client will have to parse the chunks of the HTTP response as a stream. Note, I've never run across a client (browser, etc) that didn't handle Transfer Encoding chunked. You can read more at the link below.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding

    [HttpGet]
    public async Task<HttpResponseMessage> Download(CancellationToken token)
    {
        var response = new HttpResponseMessage(System.Net.HttpStatusCode.OK)
        {
            Content = new PushStreamContent(async (stream, context, transportContext) =>
            {
                try
                {
                    using (var fileStream = System.IO.File.OpenRead("some path to MyBigDownload.zip"))
                    {
                        await fileStream.CopyToAsync(stream);
                    }
                }
                finally
                {
                    stream.Close();
                }
            }, "application/octet-stream"),
        };
        response.Headers.TransferEncodingChunked = true;
        response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment")
        {
            FileName = "MyBigDownload.zip"
        };
        return response;
    }

Answer 5

you just have to Using IIS to Enable HTTP Downloads look at this link

and you just need to return the HTTP path of the file it will be download so fast and so easy.