How to deserialize a JSON response to an object list

Question

I've written a web API to access some JSON data on a file system. The API is set up to return the data as json to the client. However, when I try to deserialize the JSON response into a list of my objects, it fails.

I've tried cleaning up the response, as it seems to have extra characters in it, but that didn't seem to work as the cleaning tended to produce wonky non-JSON. I feel like I'm making this more complicated than it needs to be.

My data model class:

public abstract class CatalogueBase
{
    string name;
    string description;
    int id;

    public string Name { get => name; set => name = value; }
    public string Description { get => description; set => description = value; }
    public int Id { get => id; set => id = value; }
}

public class Media : CatalogueBase
{
    MediaType type;
    MediaRating rating;
    string genre;

    public MediaType Type { get => type; set => type = value; }
    public MediaRating Rating { get => rating; set => rating = value; }
    public string Genre { get => genre; set => genre = value; }

    public Media()
    {

    }
}

The web API endpoint which is grabbing and sending over the data:

[HttpGet]
[Route("GetMedia/")]
public ActionResult<string> GetAll()    
{
    string[] files = Directory.GetFiles(this.config.Value.JSONFileDirectory);
    if (files.Length > 0)
    {
        try
        {
            List<string> jsonFiles = new List<string>();
            string json;

            foreach (var file in files)
            {
                using (StreamReader reader = new StreamReader(file))
                {
                    json = reader.ReadToEnd();
                    jsonFiles.Add(Regex.Unescape(json));
                }  
            }

            return JsonConvert.SerializeObject(jsonFiles, Formatting.Indented);
        }
        catch (Exception e)
        {
            throw new Exception("Could not parse JSON file.", e);
        }
    }
    else
    {
        return NotFound("No files were found in the catalog.");
    }
}

Note that each file contains a single Media object previously serialized as JSON.

The client side code which is calling the above endpoint:

public async Task<List<Media>> GetAllMedia()
{
    client = GetHttpClient(@"GetMedia/");
    HttpResponseMessage response = await client.GetAsync("");
    string content = await response.Content.ReadAsStringAsync();
    var media = JsonConvert.DeserializeObject<List<Media>>(content);
    return media;
}

And finally a sample JSONresponse I'm getting in my client:

"\"[\\r\\n  \\\"{\\\\\\\"Type\\\\\\\":0,\\\\\\\"Rating\\\\\\\":5,\\\\\\\"Genre\\\\\\\":\\\\\\\"Drama\\\\\\\",\\\\\\\"Name\\\\\\\":\\\\\\\"Memes\\\\\\\",\\\\\\\"Description\\\\\\\":\\\\\\\"A movie set during the American Civil War\\\\\\\",\\\\\\\"Id\\\\\\\":1}\\\",\\r\\n  \\\"{\\\\r\\\\n\\\\\\\"Id\\\\\\\": 2,\\\\r\\\\n\\\\\\\"Name\\\\\\\": \\\\\\\"Gods and Generals\\\\\\\",\\\\r\\\\n\\\\\\\"Description\\\\\\\": \\\\\\\"A movie set during the American Civil War\\\\\\\",\\\\r\\\\n\\\\\\\"Type\\\\\\\": \\\\\\\"Movie\\\\\\\",\\\\r\\\\n\\\\\\\"Rating\\\\\\\": \\\\\\\"Excellent\\\\\\\",\\\\r\\\\n\\\\\\\"Genre\\\\\\\" : \\\\\\\"Drama\\\\\\\"\\\\r\\\\n}\\\\r\\\\n\\\",\\r\\n  \\\"{\\\\r\\\\n\\\\\\\"Id\\\\\\\": 3,\\\\r\\\\n\\\\\\\"Name\\\\\\\": \\\\\\\"Avengers Endgame\\\\\\\"\\\\r\\\\n\\\\\\\"Description\\\\\\\": \\\\\\\"The end of the Avengers series.\\\\\\\",\\\\r\\\\n\\\\\\\"Type\\\\\\\": \\\\\\\"Movie\\\\\\\",\\\\r\\\\n\\\\\\\"Rating\\\\\\\": \\\\\\\"Excellent\\\\\\\",\\\\r\\\\n\\\\\\\"Genre\\\\\\\" : \\\\\\\"Action\\\\\\\"\\\\r\\\\n}\\\\r\\\\n\\\"\\r\\n]\""

The JSON response itself looks like an issue to me, Way too many \\ and other special characters. I've tried to clean it up with string .Replace calls, but that didn't work. Any ideas?

The actual error message:

System.AggregateException: 'One or more errors occurred. (Error converting value ...json string..... to type 'System.Collections.Generic.List`1[CatalogManager.Models.MediaManager.Media]'. Path '', line 1, position 732.)'

---

Edit: Tried some of the suggestions in the comments, and I received the following json response:

"[\"{\\\"Type\\\":0,\\\"Rating\\\":5,\\\"Genre\\\":\\\"Drama\\\",\\\"Name\\\":\\\"Memes\\\",\\\"Description\\\":\\\"A movie set during the American Civil War\\\",\\\"Id\\\":1}\",\"{\\r\\n\\\"Id\\\": 2,\\r\\n\\\"Name\\\": \\\"Gods and Generals\\\",\\r\\n\\\"Description\\\": \\\"A movie set during the American Civil War\\\",\\r\\n\\\"Type\\\": \\\"Movie\\\",\\r\\n\\\"Rating\\\": \\\"Excellent\\\",\\r\\n\\\"Genre\\\" : \\\"Drama\\\"\\r\\n}\\r\\n\",\"{\\r\\n\\\"Id\\\": 3,\\r\\n\\\"Name\\\": \\\"Avengers Endgame\\\"\\r\\n\\\"Description\\\": \\\"The end of the Avengers series.\\\",\\r\\n\\\"Type\\\": \\\"Movie\\\",\\r\\n\\\"Rating\\\": \\\"Excellent\\\",\\r\\n\\\"Genre\\\" : \\\"Action\\\"\\r\\n}\\r\\n\"]"

And when I try to parse it with JObject, I get the following error:

System.AggregateException
  HResult=0x80131500
  Message=One or more errors occurred. (Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1.)
  Source=System.Private.CoreLib
  StackTrace:
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at CatalogManager.Pages.IndexModel.OnGet() in C:\Users\tpzap_000\source\repos\CatalogManager\Pages\Manager\Index.cshtml.cs:line 73
   at Microsoft.AspNetCore.Mvc.RazorPages.Internal.ExecutorFactory.ActionResultHandlerMethod.Execute(Object receiver, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.RazorPages.Internal.PageActionInvoker.<InvokeHandlerMethodAsync>d__30.MoveNext()

Inner Exception 1:
JsonReaderException: Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1.

---

Final Edit: I tried dbc's suggestion and that worked. I think the core problem was on my API side I was parsing my json files as strings rather than into my Media model object and then using Newtonsoft to parse it into properly formed json. I changed my API side code to the following:

[HttpGet]
[Route("GetMedia/")]
public ActionResult<List<Media>> GetAll()    
{

    string[] files = Directory.GetFiles(this.config.Value.JSONFileDirectory);
    if (files.Length > 0)
    {
        try
        {
            List<Media> jsonFiles = new List<Media>();
            string json;

            foreach (var file in files)
            {
                using (StreamReader reader = new StreamReader(file))
                {
                    json = reader.ReadToEnd();
                    Media currentMedia = JsonConvert.DeserializeObject<Media>(json);

                    //jsonFiles.Add(Regex.Unescape(json));
                    jsonFiles.Add(currentMedia);
                }  
            }



            return Ok(jsonFiles);
        }
        catch (Exception e)
        {
            throw new Exception("Could not parse JSON file.", e);
        }
    }
    else
    {
        return NotFound("No files were found in the catalog.");
    }

}

And it worked successfully on the front end.

Answer 1

Your basic problem is that your server is returning a string, but the client expects application/json content containing a serialized List<Media>, thereby causing a compatibility problem during deserialization.

The easiest way to ensure that the client and server are compatible is to return exactly the same type from the server, i.e. List<Media> (or ActionResult<List<Media>> if you want the possibility of returning an error). This allows the framework to handle both the serialization and content negotiation, so you don't have to. The following does the job:

[HttpGet]
[Route("GetMedia/")]
public ActionResult<List<Media>> GetAll()
{
    string[] files = Directory.GetFiles(this.config.Value.JSONFileDirectory);
    if (files.Length > 0)
    {
        try
        {
            var serializer = JsonSerializer.CreateDefault();
            return files.Select(f =>
                {
                    using (var reader = new StreamReader(f))
                    using (var jsonReader = new JsonTextReader(reader))
                    {
                        return serializer.Deserialize<Media>(jsonReader);
                    }
                })
                .ToList();
        }
        catch (Exception e)
        {
            throw new Exception("Could not parse JSON file.", e);
        }
    }
    else
    {
        return NotFound("No files were found in the catalog.");
    }
}

Notes:

• It can be more efficient to deserialize a JSON file via direct streaming rather than loading into an intermediate string, then deserializing the string. See Performance Tips: Optimize Memory Usage for details. If this is not a concern you could just do:

  return files.Select(f => JsonConvert.DeserializeObject<Media>(File.ReadAllText(f))).ToList();