The optimal way to set a breakpoint in the Python source code while debugging CPython by GDB

Question

I use GDB to understanding how CPython executes the test.py source file and I want to stop the CPython when it starts the execution of opcode I am interested.

OS: Ubuntu 18.04.2 LTS
Debugger: GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git

---

The first problem - many CPython's .py own files are executed before my test.py gets its turn, so I can't just break at the _PyEval_EvalFrameDefault - there are many of them, so I should distinguish my file from others.

The second problem - I can't set the condition like "when the filename is equal to the test.py", because the filename is not a simple C string, it is the CPython's Unicode object, so the standard GDB string functions can't be used for comparing.

At this moment I do the next trick for breaking the execution at the needed line of test.py source:

For example, I have the source file:

x = ['a', 'b', 'c']

# I want to set the breakpoint at this line.

for e in x:
    print(e)

I add the binary left shift operator to the code:

x = ['a', 'b', 'c']

# Added for breakpoint   
a = 12
b = 2 << a

for e in x:
    print(e)

And then, track the BINARY_LSHIFT opcode execution in the Python/ceval.c file by this GDB command:

break ceval.c:1327

I have chosen the BINARY_LSHIFT opcode, because of its seldom usage in the code. Thus, I can reach the needed part of .py file quickly - it happens once in the all other .py modules executed before my test.py.

I look the more straightforward way of doing the same, so the questions:

1. Can I catch the moment the test.py starts executing? I should mention, what the test.py filename is appearing on different stages: parsing, compilation, execution. So, it also will be good to can break the CPython execution at the any stage.
2. Can I specify the line of the test.py, where I want to break? It is easy for .c files, but is not for .py files.

Answer 1

My idea would be to use a C-extension, to make setting C-breakpoints possible in a python-script (similar to pdb.set_trace() or breakpoint() since Python3.7), which I will call cbreakpoint.

Consider the following python-script:

#example.py
from cbreakpoint import cbreakpoint

cbreakpoint(breakpoint_id=1)
print("hello")
cbreakpoint(breakpoint_id=2)

It could be used as follows in gdb:

>>> gdb --args python example.py
[gdb] b cbreakpoint
[gdb] run

Now, the debuger would stops at cbreakpoint(breakpoint_id=1) and cbreakpoint(breakpoint_id=2).

Here is proof of concept, written in Cython to avoid the otherwise needed boilerplate-code:

#cbreakpoint.pyx
cdef extern from *:
    """
    long long last_breakpoint_id = -1;
    void cbreakpoint(long long breakpoint_id){
         last_breakpoint_id = breakpoint_id;
    }
    """
    void c_cbreakpoint "cbreakpoint"(long long breakpoint_id)


def cbreakpoint(breakpoint_id = 0):
    c_cbreakpoint(breakpoint_id)

which can be build inplace via:

cythonize -i cbreakpoint.pyx

If Cython isn't installed, I have uploaded a version which doesn't depend on Cython (too much code for this post) on github.

It is also possible to break conditionally, given the breakpoint_id, i.e.:

>>> gdb --args python example.py
[gdb] break src/cbreakpoint.c:595 if breakpoint_id == 2
[gdb] run

will break only after hello was printed - at cbreakpoint with id=2 (while cbreakpoint with id=1 will be skipped). Depending on Cython version the line can vary, but can be found out once gdb stops at cbreakpoint.

---

It would also do something similar without any additional modules:

1. add breakpoint or import pdb; pdb.set_trace() instead of cbreakpoint
2. gdb --args python example.py + run
3. When pdb interrupts the program, hit Ctrl+C in order to interrupt in gdb.
4. Activate breakpoints in gdb.
5. continue in gdb and then in pdb (i.e. c+enter twice).

A small problem is, that after that the breakpoints might be hit while in pdb, so the first method is a little bit more robust.