CODEWITHSUNDEEP
c#httpasp.net-web-api2owin
RB.
RB.

Reputation: 37182

Does WebAPI support reading chunked requests by chunk?

Problem

I am trying to upload some data to a web-service.

I want to upload the data in chunks, and have the web-service read each chunk in turn. However, what I find in practice is that the web-service will only read a full buffer at a time.

Is there a way to get WebAPI (running self-hosted by Owin ideally, but I can use IIS if necessary) to respect the transfer chunks?

I have verified in Wireshark that my client is sending the data chunked hence why I believe this is a WebAPI issue.

For clarity, streaming data in the response works absolutely fine - my question is about reading chunked data from the request stream.

Code

The controller looks like this:

using System;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using System.Web.Http;

public class StreamingController : ApiController
{
    [HttpPost]
    public async Task<HttpResponseMessage> Upload()
    {
        var stream = await this.Request.Content.ReadAsStreamAsync();
        var data = new byte[20];
        int chunkCount = 1;
        while (true)
        {
            // I was hoping that every time I sent a chunk, then 
            // ReadAsync would return, but I find that it will only
            // return when I have sent 20 bytes of data. 
            var bytesRead = await stream.ReadAsync(data, 0, data.Length);

            if (bytesRead <= 0)
            {
                break;
            }

            Console.WriteLine($"{chunkCount++}: {Encoding.UTF8.GetString(data)}");
        }

        return new HttpResponseMessage(HttpStatusCode.OK);
    }
}

My test client looks like this:

void Main()
{
    var url = "http://localhost:6001/streaming/upload";
    var relayRequest = (HttpWebRequest)HttpWebRequest.Create(url);
    relayRequest.Method = "POST";
    relayRequest.AllowWriteStreamBuffering = false;
    relayRequest.AllowReadStreamBuffering = false;
    relayRequest.SendChunked = true;
    relayRequest.ContentType = "application/octet-stream";
    var stream = relayRequest.GetRequestStream();

    string nextLine;
    int totalBytes = 0;

    // Read a series of lines from the console and transmit them to the server.
    while(!string.IsNullOrEmpty((nextLine = Console.ReadLine())))
    {
        var bytes = Encoding.UTF8.GetBytes(nextLine);
        totalBytes += bytes.Length;
        Console.WriteLine(
            "CLIENT: Sending {0} bytes ({1} total)", 
            bytes.Length, 
            totalBytes);
        stream.Write(bytes, 0, bytes.Length);
        stream.Flush();
    }

    var response = relayRequest.GetResponse();
    Console.WriteLine(response);
}

Justification

My specific motivation is I am writing a HTTPS tunnel for an RTP client. However, this question would also make sense in the context of an instant-messaging chat application. You wouldn't want a partial chat message to come through, and then have to wait for message 2 to find out the end of message 1...!

Upvotes: 2

Views: 2991

Answers (1)

spender
spender

Reputation: 120430

The decoding of Transfer-Encoding: chunked happens a long way away from your controllers. Depending on your host, it may not even happen in the application at all, but be handled by the http.sys pipeline API that most servers plug into.

For your application to even have a chance of looking into this data, you'll need to move away from IIS/HttpListener and use Sockets instead.

Of interest might be the Nowin project, that provides all the OWIN features without using HttpListener, instead relying on the Socket async APIs. I don't know much about it, but there might be hooks to get at the stream before it gets decoded... Seems like a lot of effort though.

Upvotes: 2

Related Questions