Create list of dicts within a dict in TCL

Question

I am forced to use TCL for something and I need to create a json string like this:

{ "mainKey": "mainValue", "subKey": [{"key1":"value1"},{"key2":"value2"}]}

So I am trying to do this:

set subDict1 [dict create key1 value1]
set subDict2 [dict create key2 value2]
set subDictList [list $subDict1 $subDict2]
set finalDict [dict create mainKey mainValue subKey $subDictList]

When I convert this dict to json, I get:

{"mainKey":"mainValue", "subKey":{"key1 value1":{"key2":"value2"}}}

instead of the required:

{ "mainKey": "mainValue", "subKey": [{"key1":"value1"},{"key2":"value2"}]}

What am I doing wrong?

slebetman · Accepted Answer

First you have to understand that TCL is a very typeless language. What exactly are list and dicts in tcl?

In Tcl a list is a string that is properly formatted where each member of the list is separated by spaces (space, tab or newline) and if the data contained by an item contains spaces they can be escaped either by:

using backslash escaping:
```
"this is a list\ of\ four\ items"
```
using "" grouping:
```
{this is a "list of four items"}
```
using {} grouping:
```
{this is a {list of four items}}
```

Note that internally, once a string has been parsed as a list, Tcl uses a different internal data structure to store the list for speed. But semantically it is still a string. Just like HTML is a specially formatted string or JSON is a specially formatted string Tcl takes the attitude that lists are nothing but specially formatted strings.

So, what are dicts? In Tcl dicts are lists with even number of elements. That's it. Nothing special. A dict is therefore also semantically a string (though as mentioned above, once tcl sees you using that string as a dict it will compile it to a different data structure for optimizing speed).

Note again the core philosophy in tcl: almost all data structures (with the exception of arrays) are merely strings that happens to be formatted in a way that has special meaning.

This is the reason you can't auto-convert tcl data structures to JSON - if you ask Tcl to guess what the data structure is you end up with whatever the programmer who wrote the guessing function want it to be. In your case it looks like it defaults to always detecting lists with even number of elements as dicts.

So how can you generate JSON correctly?

There are several ways to do this. You can of course use custom dedicated for loops or functions to convert your data structure (which again, is just a specially formatted string) to JSON.

Several years ago I've written this JSON compiler:

# data is plain old tcl values
# spec is defined as follows:
# {string} - data is simply a string, "quote" it if it's not a number
# {list} - data is a tcl list of strings, convert to JSON arrays
# {list list} - data is a tcl list of lists
# {list dict} - data is a tcl list of dicts
# {dict} - data is a tcl dict of strings
# {dict xx list} - data is a tcl dict where the value of key xx is a tcl list
# {dict * list} - data is a tcl dict of lists
# etc..
proc compile_json {spec data} {
    while [llength $spec] {
        set type [lindex $spec 0]
        set spec [lrange $spec 1 end]

        switch -- $type {
            dict {
                lappend spec * string

                set json {}
                foreach {key val} $data {
                    foreach {keymatch valtype} $spec {
                        if {[string match $keymatch $key]} {
                            lappend json [subst {"$key":[
                                compile_json $valtype $val]}]
                            break
                        }
                    }
                }
                return "{[join $json ,]}"
            }
            list {
                if {![llength $spec]} {
                    set spec string
                } else {
                    set spec [lindex $spec 0]
                }
                set json {}
                foreach {val} $data {
                    lappend json [compile_json $spec $val]
                }
                return "$$[join $json ,]$$"
            }
            string {
                if {[string is double -strict $data]} {
                    return $data
                } else {
                    return ""$data""
                }
            }
            default {error "Invalid type"}
        }
    }
}

(See http://wiki.tcl.tk/JSON for the original implementation and discussion of JSON parsing)

Because tcl can never correctly guess what your "string" is I've opted to supply a format string to the function in order to correctly interpret tcl data structures. For example, using the function above to compile your dict you'd call it like this:

compile_json {dict subKey list} finalDict

I've begged the tcllib maintainers to steal my code because I still believe it's the correct way to handle JSON in tcl but so far it's still not in tcllib.

BTW: I license the code above as public domain and you or anyone may claim full authorship of it if you wish.

Create list of dicts within a dict in TCL

Answers (2)

So how can you generate JSON correctly?

Related Questions