JsonReaderException - Unexpected character encountered while parsing value

Question

I've come across many questions on StackOverflow regarding this error, none of them have achieved what I'm attempting.

What I want to do is translate the following array of error messages into something more readable

{
    "parent.booleanChild": [
        "Unexpected character encountered while parsing value: T. Path 'parent.booleanChild', line 0, position 0",
        "Unexpected character encountered while parsing value: r. Path 'parent.booleanChild', line 0, position 0"
    ]
}

Desired outcome

{
    "parent.booleanChild": [
        "Value 'True' is not valid, only 'true', 'false' and 'null' are allowed."
    ]
}

Example request

{
    "parent": {
        "booleanChild": True
    }
}

I have tried implementing a custom JsonConverter but am finding that the JsonReaderException is raised before the converters are executed.

Has anybody achieved something similar, that allows them to produce more meaningful & readable error messages, without implementing a custom IInputFormatter?

Answer 1

I have achieved something similar to what I think you refer to in your question, although it is still quite hacky. And probably very flakey.

I honestly can't believe that this isn't supported (there are many reasons I need this including security and limiting complexity of the codebase) - this especially puzzling as the new System.Text.Json.Serialization doesn't yet support many of the features that Newtonsoft Json offers.

My attempt follows something similar to your Option 2 although no string pattern matching. it relies on you being able to derive a suitable message from an exception that was raised during binding (and indeed an exception existing, this wont work if the messages are not generated from exceptions.)

I did this in Asp.Net Core 3 although you could probably do something similar in 2.2. I chose not to override the JsonInputFormatter because I found you would have to also override most of the wire-up to do so.

In my case, all validation errors are generated by a specific exception base class which has enough context to derive what type of response should be given to the user. The exception will be wrapped in a JsonSerializationException and discarded by the JsonInputFormatter because it thinks this is a runtime exception. However as you indicate the Error event on JsonSerializerSettings allows us to access this exception.

I save the exception and the path provided by Json.Net to a store scoped to the request, In my POC case I pass an IServiceProvider reference obtained through the service collection used at start up. In this way, we can obtain a reference to original HttpContext which then can be picked up by the InvalidModelStateResponseFactory.

My start up code looks like this:-

            services
            .AddControllers()
            .ConfigureApiBehaviorOptions(options => options.InvalidModelStateResponseFactory = HandleTypedErrors)
            .AddNewtonsoftJson(options => options.ApplySerializationErrorRecording(services));
    }

    private static IActionResult HandleTypedErrors(ActionContext actionContext)
    {
        if (actionContext.ModelState.IsValid)
        {
            return null;
        }

        var errors = actionContext.ModelState
            .Select(modelError => new
            {
                State = modelError,
                Exception = actionContext.HttpContext.RetrieveSerializationException<ErrorException>(modelError.Key)
            })
            .Where(state => state.Exception != null)
            .Select(modelError => new
            {
                ErrorField = modelError.State.Key,
                ErrorCode = modelError.Exception.ErrorCode,
                ErrorDescription = modelError.Exception.Message
            })
            .ToList();

        return errors.Any() ? new BadRequestObjectResult(errors.ToList()) : null;
    }

The apply error recording looks something like this:-

public static MvcNewtonsoftJsonOptions ApplySerializationErrorRecording(this MvcNewtonsoftJsonOptions options, IServiceCollection services)
        {
            options.SerializerSettings.Error += (sender, args) =>
            {
                var provider = services.BuildServiceProvider();
                var context = provider.GetRequiredService<IHttpContextAccessor>().HttpContext;

                if (context == null || !(args.ErrorContext.Error is JsonException) ||
                    !(args.ErrorContext.Error.InnerException is ErrorException ex))
                    return;

                var errors = context.Items["Error"] as Dictionary<string, ErrorException> ??
                             new Dictionary<string, ErrorException>();
                errors.Add(args.ErrorContext.Path, ex);
                context.Items["Error"] = errors;
            };

            return options;
        }

Answer 2

This is caused by behavior of the JsonInputFormatter ReadRequestBodyAsync method.

There are a few options I've come across for using custom error messages, none of which are elegant.

Option 1: Override ReadRequestBody to add a fixed error message.

public class CustomJsonInputFormatter : JsonInputFormatter
{
    private readonly IArrayPool<char> charPool;
    private readonly MvcOptions options;

    public CustomJsonInputFormatter(ILogger logger, JsonSerializerSettings serializerSettings, ArrayPool<char> charPool, ObjectPoolProvider objectPoolProvider, MvcOptions options, MvcJsonOptions jsonOptions)
        : base(logger, serializerSettings, charPool, objectPoolProvider, options, jsonOptions)
    {
        this.charPool = new JsonArrayPool<char>(charPool);
        this.options = options;
    }

    public override async Task<InputFormatterResult> ReadRequestBodyAsync(
        InputFormatterContext context,
        Encoding encoding)
    {
        if (context == null)
        {
            throw new ArgumentNullException(nameof(context));
        }

        if (encoding == null)
        {
            throw new ArgumentNullException(nameof(encoding));
        }

        var request = context.HttpContext.Request;

        var suppressInputFormatterBuffering = options?.SuppressInputFormatterBuffering ?? false;

        if (!request.Body.CanSeek && !suppressInputFormatterBuffering)
        {
            // JSON.Net does synchronous reads. In order to avoid blocking on the stream, we asynchronously
            // read everything into a buffer, and then seek back to the beginning.
            request.EnableBuffering();
            Debug.Assert(request.Body.CanSeek);

            await request.Body.DrainAsync(CancellationToken.None);
            request.Body.Seek(0L, SeekOrigin.Begin);
        }

        using (var streamReader = context.ReaderFactory(request.Body, encoding))
        {
            using (var jsonReader = new JsonTextReader(streamReader))
            {
                jsonReader.ArrayPool = charPool;
                jsonReader.CloseInput = false;

                var successful = true;
                Exception exception = null;
                void ErrorHandler(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs eventArgs)
                {
                    successful = false;

                    var path = eventArgs.ErrorContext.Path;

                    var key = ModelNames.CreatePropertyModelName(context.ModelName, path);
                    context.ModelState.TryAddModelError(key, $"Invalid value specified for {path}");
                    eventArgs.ErrorContext.Handled = true;
                }

                var type = context.ModelType;
                var jsonSerializer = CreateJsonSerializer();
                jsonSerializer.Error += ErrorHandler;
                object model;
                try
                {
                    model = jsonSerializer.Deserialize(jsonReader, type);
                }
                finally
                {
                    // Clean up the error handler since CreateJsonSerializer() pools instances.
                    jsonSerializer.Error -= ErrorHandler;
                    ReleaseJsonSerializer(jsonSerializer);
                }

                if (successful)
                {
                    if (model == null && !context.TreatEmptyInputAsDefaultValue)
                    {
                        // Some nonempty inputs might deserialize as null, for example whitespace,
                        // or the JSON-encoded value "null". The upstream BodyModelBinder needs to
                        // be notified that we don't regard this as a real input so it can register
                        // a model binding error.
                        return InputFormatterResult.NoValue();
                    }
                    else
                    {
                        return InputFormatterResult.Success(model);
                    }
                }

                if (!(exception is JsonException || exception is OverflowException))
                {
                    var exceptionDispatchInfo = ExceptionDispatchInfo.Capture(exception);
                    exceptionDispatchInfo.Throw();
                }

                return InputFormatterResult.Failure();
            }
        }
    }
}

Option 2: Perform pattern matching in InvalidModelStateResponseFactory and replace the error

Unexpected character encountered while parsing value: T. Path 'parent.booleanChild', line 0, position 0

Option 3: Set AllowInputFormatterExceptionMessages to false and make the assumption in the InvalidModelStateResponseFactory that any blank messages will be due to serialization errors.

I am not marking this as the answer as I am sure somebody else will have a better idea.

I have raised a GitHub issue which proposes what I think may be a solution.

Other SO questions I found:

ASP.NET Core handling JSON deserialization problems

Overriding ModelBindingMessageProvider error messages