CODEWITHSUNDEEP
pythonparsingabstract-syntax-treepyparsing
Hassan Hayat
Hassan Hayat

Reputation: 1065

Convert a simple command into an AST in python

I would like a function in Python that converts a string command into an AST (Abstract Syntax Tree).

The syntax for the command is as follows:

commandName(3, "hello", 5.0, x::int)

A command can accept any number of comma separated values that can be either

  1. Integers
  2. Strings
  3. Floats
  4. Types

Suppose the function is called convert_to_ast, then

convert_to_ast('commandName(3, "hello", 5.0, x::int)')

Should yield the following AST:

{ 
  'type': 'command',
  'name': 'commandName',
  'args': [{
    'type': 'int',
    'value': 3
  }, {
    'type': 'str',
    'value': 'Hello'
  }, {
    'type': 'float',
    'value': 5.0
  }, {
    'type': 'var',
    'kind': 'int',
    'name': 'x
  }]

Upvotes: 0

Views: 780

Answers (1)

mgilson
mgilson

Reputation: 310307

Seems like you could just evaluate the string and then pick off the types from there:

>>> items = ast.literal_eval('(404.5, "Hello", 5)')
>>> [{'type': type(item).__name__, 'value': item} for item in items]
[{'type': 'float', 'value': 404.5}, {'type': 'str', 'value': 'Hello'}, {'type': 'int', 'value': 5}]

Of course, if you want to do more interesting things, you can access the AST directly:

>>> ast.dump(ast.parse('(404.5, "Hello", 5)'))
"Module(body=[Expr(value=Tuple(elts=[Num(n=404.5), Str(s='Hello'), Num(n=5)], ctx=Load()))])"
>>> ast.parse('(404.5, "Hello", 5)').body[0].value.elts
[<_ast.Num object at 0x107fa1250>, <_ast.Str object at 0x107fa1290>, <_ast.Num object at 0x107fa12d0>]

For a more general thing than parsing a tuple (as you've added to the question), we still can use python's AST to parse this (as long as your syntax is valid python). In this case, we'll create an ast.NodeVisitor which will pull out the information that we as it visits each node of the python AST that we care about. In this case, we care about Call, Num, Str and Name nodes:

import ast

class Parser(ast.NodeVisitor):

    def __init__(self):
        self.calls = []
        self.current_command = None

    def visit_Call(self, node):
        name = node.func.id
        self.current_command = {
            'type': 'command',
            'name': name,
            'args': []
        }
        self.calls.append(self.current_command)
        for arg in node.args:
            self.visit(arg)
        self.current_command = None

    def visit_Num(self, node):
        if not self.current_command:
            return
        args = self.current_command['args']
        arg = {
            'type': type(node.n).__name__,
            'value': node.n
        }
        args.append(arg)

    def visit_Str(self, node):
        if not self.current_command:
            return
        args = self.current_command['args']
        arg = {
            'type': 'str',
            'value': node.s
        }
        args.append(arg)

    def visit_Name(self, node):
        if not self.current_command:
            return
        args = self.current_command['args']
        arg = {
            'type': 'type',
            'kind': node.id 
        }
        args.append(arg)


S = 'commandName(3, "hello", 5.0, int)'

tree = ast.parse(S)
p = Parser()
p.visit(tree)
print p.calls

Upvotes: 5

Related Questions