Unhandled Exception in ASP.Net Core Web API on Linux

Question

I have an ASP.Net Core 3.1 WebAPI with a single node. (This is essentially a proxy endpoint to not have to deal with a nightmarish Whitelist at the ultimate end of this chain.) The clients, which I also control, all send programmatically identical requests to the proxy. Sometimes, or often, the Kestral server reports the following errors in the Linux server's syslog.

Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:       An unhandled exception has occurred while executing the request.
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]: Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Unexpected end of request content.
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException.Throw(RequestRejectionReason reason)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.ReadAsyncInternal(CancellationToken cancellationToken)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at System.Text.Json.JsonSerializer.ReadAsync[TValue](Stream utf8Json, Type returnType, JsonSerializerOptions options, CancellationToken cancellationToken)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMeta
data metadata, Object value)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]: --- End of stack trace from previous location where exception was thrown ---
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boo
lean isCompleted)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isComple
ted)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
Oct  2 19:06:52 CCBD-Status-API-East CCBD-Status-API[979]:    at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)

the request is very small

{'Url': '<URL data>', 'Status': 0}

API Controller code:

public IActionResult GetStreamStatus([FromBody] StreamInfo input)
{
    try
    {
        if (!ModelState.IsValid) return BadRequest();
        NetworkCredential netCred = new NetworkCredential(_config.Value.User, _config.Value.Pass);
        var cred = new CredentialCache();
        cred.Add(new Uri(input.Url), "Digest", netCred);
        _logger.LogInformation($"Attempting to get status from: {input.Url}");
        var httpClient = new HttpClient(new HttpClientHandler {Credentials = cred, PreAuthenticate = true});
        var resp = httpClient.GetAsync(new Uri(input.Url));
        _logger.LogInformation($"{input.Url} returned: {resp.Result.StatusCode}");
        input.Status = (int) resp.Result.StatusCode;

        return Ok(input);
    }
    catch (Exception e)
    {
        _logger.LogError(e.ToString());
        return BadRequest();
    }
}

Startup.cs Configure Method

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    app.UseExceptionHandler(a => a.Run(async context =>
    {
        var exceptionHandlerPathFeature = context.Features.Get<IExceptionHandlerPathFeature>();
        var exception = exceptionHandlerPathFeature.Error;

        var result = JsonConvert.SerializeObject(new {error = exception.Message});
        context.Response.ContentType = "application/json";
        await context.Response.WriteAsync(result);
    }));

    app.UseForwardedHeaders(new ForwardedHeadersOptions
    {
        ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
    });

    app.UseRouting();

    app.UseEndpoints(endpoints =>
    {
        endpoints.MapControllers();
    });
}

EDIT: Adding Client Code

def check_stream(tc, app, feed, server):
    try:
        logger.info("Check Stream: {}".format(server))
        stream_url = ""
        if server == 'pri':
            stream_url = <Building URL>
        elif server == 'bak':
            stream_url = <Building URL>

        json_input = {"Url": stream_url, "Status": 0}
        logger.info(json_input)
        headers = {'Content-Type': 'application/json'}

        resp = requests.post(<Proxy URL>, data=json.dumps(json_input),
                             headers=headers)
        
        logger.info(resp.headers)
        logger.info("Response Text: {}".format(resp.text))
        results = json.loads(resp.text)
        code = results.get('status')

        return code
    except Exception as e:
        logger.error("Stream Check exception: {} {}".format(type(e).__name__, str(e.args)))

Answer 1

I figured it out. I shifted the node to be async and it works without issue.

public async Task<IActionResult> GetStreamStatusAsync([FromBody] StreamInfo input)
{
    try
    {
        if (!ModelState.IsValid) return BadRequest();
        NetworkCredential netCred = new NetworkCredential(_config.Value.User, _config.Value.Pass);
        var cred = new CredentialCache();
        cred.Add(new Uri(input.Url), "Digest", netCred);
        _logger.LogInformation($"Attempting to get status from: {input.Url}");
        var httpClient = new HttpClient(new HttpClientHandler {Credentials = cred, PreAuthenticate = true});
        var resp = await httpClient.GetAsync(new Uri(input.Url));
        _logger.LogInformation($"{input.Url} returned: {resp.StatusCode}");
        input.Status = (int) resp.StatusCode;

        return Ok(input);
    }
    catch (Exception e)
    {
        _logger.LogError(e.ToString());
        return BadRequest();
    }
}