CODEWITHSUNDEEP
pythonparsingcompiler-construction
intull
intull

Reputation: 141

Adding a new feature in the python syntax base

I want to add a new functionality in python, purely for experimental purposes where I would like to extend the decorator syntax. Currently decorators can be applied on functions and classes. I would like to also use decorators on loops (for loop for example) and also to blocks of code.

Example 1: 

    @foo
    for i in range(20):
        # do something
        # and something more

Example 2: 

    @foo
        # there's a block starting from an indent here.
        # there's some code now
        # do something
        # and something more

Now although this is the basic idea, my requirement is to modify the body of the decorator. For example, I want to change the loop a bit based on the decorator applied to it. I can use the AST module for this.

The problem is I do not want to completely add a new syntax and its complete implementation. I just want to parse with the new syntax, access the parse tree and the decorator's body, operate on it and insert that into the body of the program, removing the decorator thus changing the program which had a new syntax to a syntax that python has right now.

Any idea on how I would go about doing this?

Upvotes: 2

Views: 101

Answers (1)

BrenBarn
BrenBarn

Reputation: 251365

You can't do that without adding new syntax. Decorators themselves don't have a "body" as such. Decorators can apply to functions or to classes and that's it. See near the top of http://docs.python.org/2/reference/grammar.html :

decorated: decorators (classdef | funcdef)

If you want something else, it can't be a decorator, it has to be your own syntax that looks like a decorator.

You could write some kind of preprocessor that parses your syntax and transforms into valid Python. One possibility is the parser module. It has facilities for parsing basic Python elements like suites (i.e., blocks). You can see a simple example in the documentation. The ast module also provides this functionality. But these modules don't provide a way to parse decorators independently of class/function defs; the decorators are essentially considered as part of the class/function defition.

Even if you manage to parse your particular construct, you will probably have to do substantial trickery to create the AST. The problem is that you can't just "access the parse tree" and "modify the AST" because the program as you've written won't have a normal Python parse tree since it can't be parsed as valid Python. So you'll have to try to stitch together your own AST by patching together your custom code with ordinary Python-parsed code.

Upvotes: 2

Related Questions