Elm - Decode Json with dynamic keys

Question

I'd like to decode a Json file that would look like this:

{ 'result': [
    {'id': 1, 'model': 'online', 'app_label': 'some_app_users'}, 
    {'id': 2, 'model': 'rank', 'app_label': 'some_app_users'}, 
]}

or like this:

{ 'result': [
    {'id': 1, 'name': 'Tom', 'skills': {'key': 'value', ...}, {'key': 'value', ...}},
    {'id': 1, 'name': 'Bob', 'skills': {'key': 'value', ...}, {'key': 'value', ...}},
]}

Basically, the content under result is a list of dicts with the same keys - but I don't know these keys in advance and I don't know their value types (int, string, dict, etc.).

The goal is to show databases tables content; the Json contains the result of the SQL query.

My decoder looks like this (not compiling):

tableContentDecoder : Decode.Decoder (List dict)
tableContentDecoder =
    Decode.at [ "result" ] (Decode.list Decode.dict)

I use it like this:

Http.send GotTableContent (Http.get url tableContentDecoder)

I'm getting that error:

Function list is expecting the argument to be: Decode.Decoder (Dict.Dict String a)

But it is: Decode.Decoder a -> Decode.Decoder (Dict.Dict String a)

What's the correct syntax to use the dict decoder? Will that work? I couldn't find any universal Elm decoder...

Answer 1

I couldn't figure out how to get the Decode.dict to work so I have changed my Json and splited the columns and results:

data={
    'columns': [column.name for column in cursor.description],
    'results': [[str(column) for column in record] for record in cursor.fetchall()]
}

I also had to convert all the results to String to make it simple. The Json will have 'id': "1" for example.

With the Json done that way, the Elm code is really simple:

type alias QueryResult =
    { columns : List String, results : List (List String) }

tableContentDecoder : Decode.Decoder QueryResult
tableContentDecoder =
    Decode.map2
        QueryResult
        (Decode.field "columns" (Decode.list Decode.string))
        (Decode.field "results" (Decode.list (Decode.list Decode.string)))

Answer 2

Decode.list is a function that takes a value of type Decoder a and returns a value of the type Decoder (List a). Decode.dict is also a function that takes a value of type Decoder a that returns a decoder of Decoder (Dict String a). This tells us two things:

• We need to pass a decoder value to Decode.dict before we pass it to Decoder.list
• A Dict may not fit your use case as Dicts can only map between two fixed types and do not support nest values like 'skills': {'key': 'value', ...}

Elm doesn't provide a universal decoder. The motivation for this has to do with Elm's guarantee of "no runtime errors". When dealing with the outside world, Elm needs to protect its runtime from the possibility of external failures, mistakes, ect. Elm's primary mechanism for doing this is types. Elm only lets data in that is correctly described and by doing so eliminates the possibility of errors that a universal decoder would introduce.

Since your primary goal is to display content, something like Dict String String might work, but it depends on how deeply nested your data is. You could implement this with a small modification to your code: Decode.at [ "result" ] <| Decode.list (Decode.dict Decode.string).

Another possibility is using Decode.value and Decode.andThen to test for values that indicate which table we are reading from.

It's important that our decoder has a single consistent type, which means we would need to represent our possible results as a sum type.

-- represents the different possible tables
type TableEntry
    = ModelTableEntry ModelTableFields
    | UserTableEntry  UserTableFields
    | ... 

-- we will use this alias as a constructor with `Decode.map3`
type alias ModelTableFields =
    { id       : Int
    , model    : String
    , appLabel : String
    }

type alias UserTableFields =
    { id : Int
    , ...
    }

tableContentDecoder : Decoder (List TableEntry)
tableContentDecoder =
    Decode.value 
        |> Decode.andThen 
            \value ->
                let
                    tryAt field = 
                        Decode.decodeValue
                            (Decode.at ["result"] <| 
                                Decode.list <|
                                Decode.at [field] Decode.string)
                            value
                in  
                    -- check the results of various attempts and use
                    -- the appropriate decoder based on results
                    case ( tryAt "model", tryAt "name", ... ) of
                        ( Ok _, _, ... ) ->
                            decodeModelTable

                        ( _, Ok _, ... ) ->
                            decodeUserTable

                        ...

                        (_, _, ..., _ ) ->
                            Decode.fail "I don't know what that was!"

-- example decoder for ModelTableEntry
-- Others can be constructed in a similar manner but, you might
-- want to use NoRedInk/Json.Decode.Pipline for more complex data
decodeModel : Decoder (List TableEntry)
decodeModel  =
    Decode.list <|
       Decode.map3 
           (ModelTableEntry << ModelTableFields)
           (Decode.field "id" Decode.int)
           (Decode.field "model" Decode.string)
           (Decode.field "app_label" Decode.string) 

decodeUser : Decoder (List TableEntry)
decodeUser = 
    ...

It is fair to say that this is a lot more work than most other languages would make you do to parse JSON. However, this comes with the benefit of being able to use outside data without worrying about exceptions.

One way of thinking about it is that Elm makes you do all the work upfront. Where other languages might let you get up and running faster but, do less to help you get to a stable implementation.