CODEWITHSUNDEEP
pythonexpression-treesabstract-syntax-treedslbytecode
Jason Baker
Jason Baker

Reputation: 198577

Is there any way to programmatically generate Python bytecode?

I want to hack around with the Python interpreter and try creating a small DSL . Is there any module where I can do something like this theoretical code (similar to LINQ expression trees)? 

expression_tree = Function(
    Print(
        String('Hello world!')
    )
 )
compile_to_bytecode(expression_tree)

Or would it just be easier to just generate Python source code? Could this be made easier by using C or SWIG or Cython?

Upvotes: 12

Views: 5557

Answers (6)

Mark Rushakoff
Mark Rushakoff

Reputation: 258188

In Python 2.X, you would typically approach this with the compiler module and its ast sub-module (but note this module is deprecated since version 2.6). In Python 3.X, you would use just ast.

Both offer a compile() function that will go from source/AST to "a code object that can be executed by the exec statement or eval()."

Upvotes: 5

Wojciech Kaczmarek
Wojciech Kaczmarek

Reputation: 2342

Updating for Python3 - there's also very interesting assembler zachariahreed/byteasm.

Actually the only one working for me in Py3. It has very nice & clean API:

>>> import byteasm, dis
>>> b = byteasm.FunctionBuilder()
>>> b.add_positional_arg('x')
>>> b.emit_load_const('Hello!')
>>> b.emit_load_fast('x')
>>> b.emit_build_tuple(2)
>>> b.emit_return_value()
>>> f = b.make('f')
>>> f
<function f at 0xb7012a4c>
>>> dis.dis(f)
  1           0 LOAD_CONST               0 ('Hello!')
              3 LOAD_FAST                0 (x)
              6 BUILD_TUPLE              2
              9 RETURN_VALUE
>>> f(108)
('Hello!', 108)

Upvotes: 1

Alex Martelli
Alex Martelli

Reputation: 881635

Working via ast and compiling the tree into bytecode, as another answer suggests, is probably simplest; generating sources and compiling those, almost as good.

However, to explore lower-level approaches, check out the links from this page; I've found byteplay especially useful (currently doesn't work on 2.6 nor 3.*, only 2.4 or 2.5, but I think fixing it for 2.6 should be easy, as currently discussed in its tracker). I have not used Phil Eby's similarly-featured BytecodeAssembler, but given the author's reputation I'm sure it's worth checking it out!

Upvotes: 13

Lennart Regebro
Lennart Regebro

Reputation: 172249

It's easier to generate Python code and run it. If you do that you can also debug it more easily since there is actual source for the debugger to display. See also Malte Borchs article in the July issue of Python magazine, where he talks about this amongst other things.

Upvotes: 2

ACoolie
ACoolie

Reputation: 1439

Fernando Meyer recently wrote a blog post explaining how to use the # coding directive to specify your own extensions to Python. Example (the actual format definition is in pyspec.py and tokenizer.py):

# coding: pyspec

class Bow:
    def shot(self):
        print "got shot"

    def score(self):
        return 5

describe Bowling:
    it "should score 0 for gutter game":
        bowling = Bow()
        bowling.shot()
        assert that bowling.score.should_be(5)

Upvotes: 1

Nope
Nope

Reputation: 36000

Check out the disassembler module found here:

http://docs.python.org/library/dis.html

Upvotes: 0

Related Questions