CODEWITHSUNDEEP
pythonabstract-syntax-tree
Chris
Chris

Reputation: 93

How do you turn an unquoted Python function/lambda into AST? 2.6

This seems like it should be easy, but I can't find the answer anywhere - nor able to derive one myself. How do you turn an unquoted python function/lambda into an AST?

Here is what I'd like to be able to do.

import ast
class Walker(ast.NodeVisitor):
    pass
    # ...

# note, this doesnt work as ast.parse wants a string
tree = ast.parse(lambda x,y: x+y)

Walker().visit(tree)

Upvotes: 8

Views: 3431

Answers (5)

dan
dan

Reputation: 51

The Meta library allows you to recover the source in many cases, with some exceptions such as comprehensions and lambdas.

import meta, ast
source = '''
a = 1
b = 2
c = (a ** b)
'''

mod = ast.parse(source, '<nofile>', 'exec')
code = compile(mod, '<nofile>', 'exec')

mod2 = meta.decompile(code)
source2 = meta.dump_python_source(mod2)

assert source == source2

Upvotes: 5

Ants Aasma
Ants Aasma

Reputation: 54882

If you only get access to the function/lambda you only have the compiled python bytecode. The exact Python AST can't be reconstructed from the bytecode because there is information loss in the compilation process. But you can analyze the bytecode and create AST's for that. There is one such analyzer in GeniuSQL. I also have a small proof of concept that analyzes bytecode and creates SQLAlchemy clauseelements from this.

The process I used for analyzing is the following:

  1. Split the code into a list of opcodes with potential arguments.
  2. Find the basic blocks in the code by going through the opcodes and for every jump create a basic block boundary after the jump and before the jump target
  3. Create a control flow graph from the basic blocks.
  4. Go through all the basic blocks with abstract interpretation tracking stack and variable assignments in SSA form.
  5. To create the output expression just get the calculated SSA return value.

I have pasted my proof of concept and example code using it. This is non-clean quickly hacked together code, but you're free to build on it if you like. Leave a note if you decide to make something useful from it.

Upvotes: 6

Alex Martelli
Alex Martelli

Reputation: 881675

In general, you can't. For example, 2 + 2 is an expression -- but if you pass it to any function or method, the argument being passed is just the number 4, no way to recover what expression it was computed from. Function source code can sometimes be recovered (though not for a lambda), but "an unquoted Python expression" gets evaluated so what you get is just the object that's the expression's value.

What problem are you trying to solve? There may be other, viable approaches.

Edit: tx to the OP for clarifying. There's no way to do it for lambda or some other corner cases, but as I mention function source code can sometimes be recovered...:

import ast
import inspect

def f():
  return 23

tree = ast.parse(inspect.getsource(f))

print ast.dump(tree)

inspect.getsource raises IOError if it can't get the source code for whatever object you're passing it. I suggest you wrap the parsing and getsource call into an auxiliary function that can accept a string (and just parses it) OR a function (and tries getsource on it, possibly giving better errors in the IOError case).

Upvotes: 10

nosklo
nosklo

Reputation: 222862

You can't generate AST from compiled bytecode. You need the source code.

Upvotes: 2

Ned Batchelder
Ned Batchelder

Reputation: 375584

Your lambda expression is a function, which has lots of information, but I don't think it still has source code associated with. I'm not sure you can get what you want.

Upvotes: 0

Related Questions