CODEWITHSUNDEEP
c#asp.netasp.net-web-api
Henning Krause
Henning Krause

Reputation: 5422

Streaming large files (>2GB over IIS) using WebAPI

I'm trying to upload very large files (>2GB) to my WebAPI application (Running on .NET 4.5.2, Windows 2012R2).

Setting the httpRuntime maxRequestLength property is of no use because it's only working with files smaller than 2GB.

I'm currently using a custom MultipartFormDataStreamProvider to read the entire stream on the Server and I've already turned off buffering using a custom WebHostBufferPolicySelector.

What I discovered is that ASP.NET (or WebAPI for that matter) uses a HttpBufferlessInputStream under the Hood which has a field called _disableMaxRequestLength. If I set this value to to true (via reflection), I can stream files of any size.

However, fiddling around these with these internas is clearly not a good way.

The HttpRequest class used for the request has a method called GetBufferlessInputStream which has an overload that allows to disable the maxRequestLength.

My question is: How can I get the WebAPI to use this overload instead of the Standard one?

Is there any way to replace the Default HttpRequest or HttpContext class? Or do I really Need to use reflection for the whole stuff?

This is the code I'm currently using to disable the maxRequestLength:

    private void DisableRequestLengthOnStream(HttpContent parent)
    {
        var streamContentProperty = parent.GetType().GetProperty("StreamContent", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
        if (streamContentProperty == null) return;
        var streamContent = streamContentProperty.GetValue(parent, null);
        if (streamContent == null) return;
        var contentProperty = typeof(StreamContent).GetField("content", BindingFlags.Instance | BindingFlags.NonPublic);
        if (contentProperty == null) return;
        var content = contentProperty.GetValue(streamContent);
        if (content == null) return;
        var requestLengthField = content.GetType().GetField("_disableMaxRequestLength", BindingFlags.Instance | BindingFlags.NonPublic);
        if (requestLengthField == null) return;
        requestLengthField.SetValue(content, true);
    }

Upvotes: 26

Views: 23796

Answers (3)

Henning Krause
Henning Krause

Reputation: 5422

Ok, I found a pretty simple solution. The answer from @JustinR. would work, of course. But I wanted to continue to use a MultipartFormDataStreamProvider because that handles all the MIME stuff.

The solution is to simply create a new StreamContent instance with the correct bufferless Input stream and populate it with the headers from the original Content:

var provider = new MultipartFormDataStreamProvider(path);
var content = new StreamContent(HttpContext.Current.Request.GetBufferlessInputStream(true));
foreach (var header in Request.Content.Headers)
{
    content.Headers.TryAddWithoutValidation(header.Key, header.Value);
}

await content.ReadAsMultipartAsync(provider);

Upvotes: 7

Filip W
Filip W

Reputation: 27187

IMHO there is no easy way to do that.

The call to GetBufferlessInputStream is buried deep inside HttpControllerHandler, which is the lowest possible layer of ASP.NET Web API (it's an HTTP Handler on top of which the entire Web API stack is built.

You can see the code here.

As you see it's full of statics, long methods with nested logical conditions, internals and privates, so it's not really customisable at all.
While the entire HttpControllerHandler in Web API can theoretically be replaced with a custom implementation (this is done inside HttpControllerRouteHandler - by overriding the GetHttpHandler method), it's de facto impossible (you could try to internalize this code in your application, but you'll end up dragging lots of extra internal classes too).

The best thing (and I dread to say that) that comes to my mind is to modify the source HttpControllerHandler class to use the overload of GetBufferlessInputStream that disables the request length limit and recompile the System.Web.Http.WebHost assembly, and deploy that modded version with your app.

Upvotes: 1

Justin R.
Justin R.

Reputation: 24061

According to MSDN, the way to read an unlimited stream length is HttpRequest.GetBufferlessInputStream. You could do something like:

public void ReadStream(HttpContext context, string filePath)
{
    using (var reader = new StreamReader(context.Request.GetBufferlessInputStream(true)))
    using (var filestream = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.Read, 4096, true))
    using (var writer = new StreamWriter(filestream))
    {
        var readBuffer = reader.ReadToEnd();
        writer.WriteAsync(readBuffer);
    }
}

Upvotes: 2

Related Questions