CODEWITHSUNDEEP
c#jsonserializationdeserializationjsonlines
After_8
After_8

Reputation: 329

C# Type safe JSON-Lines Deserialization

Currently I am working with the Shopify GraphQL Bulk Query.
This Query returns a JSON Lines file. Such a file may look like this:

{"id":"gid:\/\/shopify\/Product\/5860091625632","title":"Levis Jeans","description":"Cool Jeans","vendor":"Levis","status":"ACTIVE"}
{"id":"gid:\/\/shopify\/ProductImage\/20289865679008","__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118963360","title":"32","position":1,"image":null,"selectedOptions":[{"name":"Size","value":"32"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":10,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118963360"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178118996128","title":"31","position":2,"image":null,"selectedOptions":[{"name":"Size","value":"31"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"available":3,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178118996128"}
{"id":"gid:\/\/shopify\/ProductVariant\/37178119028896","title":"34","position":3,"image":null,"selectedOptions":[{"name":"Size","value":"34"}],"inventoryItem":{},"__parentId":"gid:\/\/shopify\/Product\/5860091625632"}
{"available":5,"location":{"id":"gid:\/\/shopify\/Location\/57510625440"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}
{"available":15,"location":{"id":"gid:\/\/shopify\/Location\/57951518880"},"__parentId":"gid:\/\/shopify\/ProductVariant\/37178119028896"}

Each line of this file is a valid JSON-object and the lines are connected via __parentId with each other.
My Goal is to Deserialize this into C# Classes like this:

class Product
{
    public string Id { get; set; }
    public string Title { get; set; }
    public string Description { get; set; }
    public IEnumerable<ProductImage> Images { get; set; }
    public IEnumerable<ProductVariant> Variants { get; set; }
}
class ProductImage
{
    public string Id { get; set; }
}
class ProductVariant
{
    public string Id { get; set; }
    public IEnumerable<IDictionary<string, string>> SelectedOptions { get; set; }
    public IEnumerable<InventoryLevel> Levels { get; set; }
}
class InventoryLevel
{
    public int Available { get; set; }
}

And the output of a potential function performing the deserialization:

var file = new System.IO.StreamReader(@"c:\test.jsonl");
var products = DeserializeJsonL<IEnumerable<Product>>(file);

Shopify suggests to read the file in reverse. I get the Idea. But I cannot imagine how to deserialize this file in a type safe way. How could I determine if the current line is a ProductVariant, a ProductImage or something else? I cannot influence the JSONL Output to include type information.

I am pretty sure without type information I cannot deserialize it safely. But how should I handle this data then to insert into a database for example?

EDIT the classname in {"id":"gid:\/\/shopify\/Product\/5860091625632"} cannot be used to determine the Type!

Upvotes: 1

Views: 1299

Answers (2)

Mark King 1972
Mark King 1972

Reputation: 1

In GraphQL there is an automatic field called __typename. You can use that field to get the Type information. So in your GraphQL query you can do something like this

{
   product (id:"foo"){
       id
       __typename
       variants{
            nodes{
               id
               __typename
            }
       }
   }
}

This will give you back the Type information for each line.

Upvotes: 0

After_8
After_8

Reputation: 329

I ended up adding some sort of type information to my graphql-query by defining a unique fieldname for each type which may be on a new line in the resulting JSON Lines file.

For that i used GraphQL field aliases:

someQuery {
   uniqueFieldAlias : fieldName
}

When i read the file i search on each line for the unique fieldname. Then i deserialize the line into the corresponding class.

using (var file = new StreamReader(await res.Content.ReadAsStreamAsync()))
{
    string line;

    while ((line = await file.ReadLineAsync()) != null)
    {
        if (line.Contains("\"uniqueFieldAlias\""))
        {
            var product = JsonSerializer.Deserialize<Product>(line);
            products.Add(product);
            continue;
        }
        if (line.Contains("\"otherUniqueAlias\""))
        {
            var somethingElse = JsonSerializer.Deserialize<SomeClass>(line);
            products[productIndex].Something.Add(somethingElse);
            continue;
        }
    }
}

The idea is inspired by @Caius Jard comments

Upvotes: 0

Related Questions