Translate Tcl List to Python List

Question

I try to translate Tcl list to python list.

There are two problems:

• If a list in the original list contains only a single list, the translation is incorrect. For example, {{12 34}} is not correctly translated.
• The option to convert all numbers to a type does not work.

Python 3 code:

import tkinter


class TclInterpreter(object):
    def __init__(self):
        self._tcl = tkinter.Tcl()

    def eval(self, tcl_cmd):
        return self._tcl.eval(tcl_cmd)


class TclPyListTranslator(object):
    def __init__(self, tcl):
        self._tcl = tcl

    def to_py(self, tcl_list, dtype=str):
        # convert a Tcl List to python list, also convert elements of each leaf
        # node to dtype
        self._tcl.eval("set tcl_list %s" % tcl_list)
        numItems = int(self._tcl.eval("llength $tcl_list"))
        if numItems > 1:
            result = [self._tcl.eval("lindex $tcl_list %d" % i) for i in range(
                numItems)]
            for i in range(numItems):
                result[i] = self.to_py("{" + result[i] + "}", dtype)
        else:
            result = dtype(self._tcl.eval("lindex $tcl_list %d" % 0))
        return result


inter = TclInterpreter()
translator = TclPyListTranslator(inter)
tcl_list = "{12 {{12 34}} {56 {78 {11 12} 10}}}"

# prints ['12', '12 34', ['56', ['78', ['11', '12'], '10']]]
# The '12 34' is incorrect
print(translator.to_py(tcl_list))

# does not run
print(translator.to_py(tcl_list, int))

Answer 1

Python parser:

def add_element(cache, element):
    if element != '':
        cache[-1].append(element)
    return ''

def parse(tcl_list):
    """ Parse TCL list to Python list """    
    out = []
    cache = [out]
    element = ''
    escape = False
    for char in tcl_list:
        if escape:
            element += char
            escape = False
        elif char == "\\":
            escape = True
        elif char in [" ", "\t", "\r", "\n"]:
            element = add_element(cache, element)
        elif char == "{":
            a = []
            cache[-1].append(a)
            cache.append(a)
        elif char == "}":
            element = add_element(cache, element)
            cache.pop()
        else:
            element += char
    return out[0]
import pprint
pprint.pprint(
    parse("{ 12 apple {100} {} {{12 34}} \n {56\n { \\{78 {11 12 11} 10}}}"))

output:

['12',
 'apple',
 ['100'],
 [],
 [['12', '34']],
 ['56', ['{78', ['11', '12', '11'], '10']]]

Answer 2

I had a need to do this today and used the accepted answer as a starting point, however, it did not take into account strings containing spaces. For example: {hello world "foo bar"} would result in ['hello', 'world', '"foo', 'bar"'] rather than ['hello', 'world', 'foo bar']

Here is a modified implementation:

class TCLListParser(object):

    NO_ESCAPE = 0
    SINGLE_ESCAPE = 1
    STRING_ESCAPE = 2

    def __init__(self):
        self._out = None
        self._buffer = None
        self._stack = None

    def _flush(self):
        if self._buffer is not None:
            self._stack[-1].append(self._buffer)
        self._buffer = None

    def _add_char(self, char):
        if self._buffer is None:
            self._buffer = char
        else:
            self._buffer += char

    def parse(self, tcl_list):
        self._out = []
        self._stack = [self._out]
        self._buffer = None

        escape = self.NO_ESCAPE

        for char in tcl_list:
            # Single escapes
            if escape & self.SINGLE_ESCAPE:
                self._add_char(char)
                escape &= ~self.SINGLE_ESCAPE
            elif char == '\\':
                escape |= self.SINGLE_ESCAPE
            # Strings with spaces, like "hello world"
            elif char == '"':
                escape ^= self.STRING_ESCAPE
            else:
                if escape & self.STRING_ESCAPE:
                    self._add_char(char)
                elif char in [" ", "\t", "\r", "\n"]:
                    self._flush()
                elif char == "{":
                    _ = []
                    self._stack[-1].append(_)
                    self._stack.append(_)
                elif char == "}":
                    self._flush()
                    self._stack.pop()
                else:
                    self._add_char(char)
        return self._out[0]

parser = TCLListParser()
pprint.pprint(parser.parse('{ 12 "big apple" {100} {} {{12 34}} \n {56\n { \\{78 {11 12 11} 10}}}'))

result:

['12',
 'big apple',
 ['100'],
 [],
 [['12', '34']],
 ['56', ['{78', ['11', '12', '11'], '10']]]

Answer 3

The easiest way of handling this is to get the code on the Tcl side (which natively understands Tcl lists) to generate the string form of the Python value and then eval that within Python. However, the complex part is that Tcl's type system is really utterly different to that of Python (to the point where I don't intend to explain it as it's an intensely complex and technical argument), making deciding where the leaves of the nested list structure non-trivial. Some assumptions are required. With those assumptions, we can do a pretty decent job in not too much code.

The Tcl side code you need is something like this (in the case where you need leaves that are integers):

proc toPythonList {value} {
    if {[string is integer -strict $value]} {
        return $value
    }
    set result "\["
    foreach item $value {
        append result [toPythonList $item] ", "
    }
    append result "\]"
    return $result
}

That then means you can do this (and I've added a very simple-minded version of the adaptation for different types of leaves):

class TclPyListTranslator(object):
    def __init__(self, tcl):
        self._tcl = tcl
        self._tcl.eval("""
            proc isLeaf.int {value} {
                string is integer -strict $value
            }
            proc isLeaf.str {value} {
                expr {![string match "{*}" $value]}
            }
            proc toPythonLeaf.int {value} { return $value }
            proc toPythonLeaf.str {value} { return "\"$value\"" }
            proc toPythonList {value dtype} {
                if {[isLeaf.$dtype $value]} {
                    return [toPythonLeaf.$dtype $value]
                }
                set result "\["
                foreach item $value {
                    append result [toPythonList $item] ", "
                }
                append result "\]"
                return $result
            }
        """)

    def to_py(self, tcl_list, dtype=str):
        # convert a Tcl List to python list
        return eval(self._tcl.eval("toPythonList %s %s" % (tcl_list, dtype.__name__))

WARNING: The code above should work, but I can't test it as I don't have tkinter configured in any of my Python interpreters. The pieces work on their own though, so I'm reasonably confident.