Parse HTTP request body to JSON string in .net core 3.0

Question

I have implemented the following method format request body

private async Task<string> FormatRequest(HttpRequest request)
{
      request.EnableBuffering();

      //Create a new byte[] with the same length as the request stream
      var buffer = new byte[Convert.ToInt32(request.ContentLength)];

      //Copy the entire request stream into the new buffer
      await request.Body.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);

      //Convert the byte[] into a string using UTF8 encoding
      var bodyAsText = Encoding.UTF8.GetString(buffer);

      request.Body.Position = 0;

      return bodyAsText;
}

I got the following result

------WebKitFormBoundaryY8OPXY2MlrKMjBRe
 Content-Disposition: form-data; name="RoleId"
 
 2
 ------WebKitFormBoundaryY8OPXY2MlrKMjBRe
 Content-Disposition: form-data; name="AuthenticationSettingsId"
 
 3
.....

Expected result

"{\"fields\":[\"RoleId\",\"2\",\"AuthenticationSettingsId\",\"1\",\"recommendation\",\"reviewerId\"],\"size\":100,\"filter\":[{\"id\":\"ApplicationId\",\"operator\":\"and\",\"parent\":\"\",\"nested\":false,\"type\":\"integer\",\"value\":[360]}],\"aggregate\":[],\"sort\":[]}"

Note: Previously we used request.EnableRewind() it was returning the above result and later upgraded to .net core 3.0

Answer 1

Here is a high level of how I handle JSON queries. If you really want to get fancy you can implement all this into an abstract class and inherit direct to your data model.

There are plenty of different ways to get where you want to be, hopefully this helps you get there.

I've put comments in the code, but feel free to ask away if something doesn't make sense.

class SomeHttpJsonUtility
{
    
    //If you want to parse your return data
    //directly into a data model
    class DataModel{

        class ReturnData
        {
            [JsonPropertyName("fields")]
            public Field[] Fields { get; set; }
        }

        class Field
        {
            [JsonPropertyName("RoleId")]
            public int RoleId { get; set; }

            //...you get the idea

        }
    }

    //Some data if your sending a post request
    private Dictionary<string, string> postParameters = new Dictionary<string, string>();

    //Creates a HTTP Client With Specified Parameters
    //You can do this any number of ways depending on the
    //source you are querying
    private HttpClient GetClient()
    {
        HttpClient _client = new HttpClient();

        _client.DefaultRequestHeaders.Clear();

        _client.DefaultRequestHeaders.Add(
            "UserAgent",
            new string[] { "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0" });

        _client.DefaultRequestHeaders.Add(
            "AcceptLanguage",
            new string[] { "en-US" });

        _client.DefaultRequestHeaders.Add(
            "AcceptEncoding",
            new string[] { "gzip", "deflate", "br" });

        _client.DefaultRequestHeaders.Add(
            "Accept",
            new string[] { "*/*" });

        _client.DefaultRequestHeaders.Add(
            "Connection",
            new string[] { "keep-alive" });

        return _client;
    }

    private void GetJson(Uri from_uri)
    {
        //Get the HttpClient With Proper Request Headers
        HttpClient _client =
            GetClient();

        Task.Run(async () =>
        {
            //If your data comes from a get request
            HttpResponseMessage _httpResponse =
                await _client.GetAsync(
                    requestUri:from_uri);

            //Or if your response comes from a post
            _httpResponse =
                await _client.PostAsync(
                    requestUri: from_uri,
                    content: new FormUrlEncodedContent(postParameters)
                    );

            //since your initial post used a stream, we can
            //keep going in that direction
            //Initilize a memory stream to process the data
            using(MemoryStream _ms = new MemoryStream())
            {
                //Send the http response content
                ////into the memory stream
                await _httpResponse.Content.CopyToAsync(
                    stream: _ms);

                //Goto the start of the memory stream
                _ms.Seek(
                    offset: 0,
                    loc: SeekOrigin.Begin);

                //Option 1:
                //Send direct to data model
                //  This is utilizing the Microsoft Library:
                //  System.Text.Json.Serialization;
                DataModel dataModel =
                    JsonSerializer.Deserialize<DataModel>(
                    utf8Json: _ms);

                //Option 2:
                //Send to a string

                using(StreamReader _sr = new StreamReader(_ms))
                {
                    string dataAsSting = _sr.ReadToEnd();
                }
            }
        }).Wait();

    }
}

If your query is only a Get request, then it's pretty easy get get the exact headers you need.

Using Firefox hit F12 and goto the web address.

Click the Network Tab, then Headers and view the request data.

You really only need a few of these:

• Accept
• Accept-Encoding
• Accept-Language
• Connection
• User-Agent

Mozilla has some nice resources regarding the different header objects.

• https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept

Host should be taken care of by the HttpClient.

Cookies should be handled by the HttpClient (if you need them)

If you are actually getting the data back as Gzip you'll have to implement a reader, unless the HttpClient you are using will automatically decode it.

And at the end, victory :-)

Answer 2

Karthik, it appears you are sending a multipart request from a webkit browser.

If you would be able to just change it on the client side from multipart to application/json your problem would be fixed.

If this is not possible, you can just use:

private async Task<string> FormatRequest(HttpRequest request)
{
    var form = request.Form.ToDictionary(x => x.Key, x => x.Value);
    return JsonSerializer.Serialize(form);
}

This could parses your form into a dictionary and returns it as a Json. (This code is written in dotnet 6, which has System.Text.Json. If you need to stick in .net 3.1, you would need to use a JsonSerializer like Newtonsoft.)

Answer 3

You need to to tokenize / encode your string with some JSON encoder. Here you have two choices:

• the internal (Microsoft) JsonConverter
• the Newtonsoft.Json JsonConverter

Answer 4

Could you try to read this way?

var reader = new System.IO.StreamReader(request.Body);        
var body = reader.ReadToEndAsync().Result;

Then you can use Newtonsoft or a similar library on body.

Answer 5

I think you need to set the content-type on the request when you send it to application/json