Debugging Pybind11 extension with Visual Studio Code (MacOS)

Question

I've been using pybind11 recently, and now that I'm getting the hang of it, I'm thrilled with it. It's an awesome piece of work. The final piece of the tool puzzle for doing pybind11 is the debug part. I've got command line debugging with lldb working using the following guide:

Debugging pybind11 with lldb

I've spent some time trying to get debug working with Visual Studio Code, with limited success. The first issue is that in order to set up an attach configuration, you need to specify the python executable (not the process id). I don't know how this is supposed to work if you have more than one active python process, which often happens.

Putting that aside, I set up a launch configuration to point to the ipython executable, which is the most convenient thing to use. When I try to start debugging, I get this:

Can anyone explain this?

If I change the executable to plain Python, I get this in the Debug Console:

Could not initialize Python interpreter - only native expressions will be available.
Launching: /Users/andy/anaconda3/envs/SciPy37/bin/python

But if go to the Terminal window, I can successfully enter Python expressions, and trigger breakpoints set in the code. Hooray! But there is one more problem. When one of my extensions needs to load another dylib, it can't find it.

>>> import block_test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/andy/Dropbox (Personal)/Developer/AudioDev/GenericDSP/Common/SciPy/BlockTest/build/block_test.cpython-37m-darwin.so, 2): Symbol not found: _vDSP_vsmul
  Referenced from: /Users/andy/Dropbox (Personal)/Developer/AudioDev/GenericDSP/Common/SciPy/BlockTest/build/block_test.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Users/andy/Dropbox (Personal)/Developer/AudioDev/GenericDSP/Common/SciPy/BlockTest/build/block_test.cpython-37m-darwin.so

This makes some sense because_vDSP_vsmul is part of an Apple DSP accelerate library. But what I don't get is that I don't have this problem when I use the command line debug technique noted at the beginning of this post. Clearly this is somehow related to how dylibs are found, but why would this be different from the command line situation?

Any help on these issues would be great. Getting this debug stuff working in Visual Studio Code is the missing piece to absolutely amazing interoperability between Python and C++

Answer 1

I had issues achieving this in macOS 10.14.5 using LLDB. What worked for me was installing the "vadimcn.vscode-lldb" extension for VS Code with the following configurations:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "lldb",
            "request": "attach",
            "name": "Attach pid",
            "pid": "${command:pickProcess}", // use ${command:pickProcess} to pick other users' processes,
        },
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "stopOnEntry": true,
            "env": {
                "PYTHONPATH": "${workspaceFolder}/build"
            }
        },
        {
            "type": "lldb",
            "request": "launch",
            "name": "LLDB Python",
            "program": "/usr/local/bin/python3",
            "args": [
                "${file}"
            ],
            "cwd": "${workspaceFolder}",
            "stopOnEntry": false,
            "env": {
                "PYTHONPATH": "${workspaceFolder}/build"
            },
        },
    ]
}

Either launch python with lldb (LLDB Python) or I first launch a python debug session (Python: Current File) which I can attach to (Attach pid) afterward. The second option has the benefit of being able to set breakpoints in both python and C++.

Answer 2

I've stumbled upon this SO post How to configure VS Code to be able to step into a shared library (.so) that is loaded when debugging a Python script? and instead of having to attach to an already running debugger, I was able to launch gdb from within VSCode:

    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) Launch 1123",
            "type": "cppdbg",
            "request": "launch",
            "program": "/usr/bin/python3",
            "args": ["${file}"],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

From this launch.json setup, I am able to step through my C++ code where my breakpoints are set and debug my C++ code. However, as mentioned in the comments from the previous SO post I am not sure if there is a method to flow between python and C/C++ code during a debugging session. If attempted to place breakpoints on my python file, I too "get an unfilled circle instead of a red dot when you create them".

Answer 3

I figured out how to get attach to process debugging to work with Visual Studio Code with iPython and on Mac OS Catalina. The following code in launch.json will work:

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "type": "lldb",
      "request": "launch",
      "name": "Debug",
      "program": "path_to/bin/python",
      "args": [],
      "cwd": "${workspaceFolder}/build"
    },
    { 
      "name": "Debug Attach",
      "type": "lldb",
      "request": "attach",
      "program": "path_to/bin/python",
      "processId": "${command:pickProcess}",
      "MIMode": "gdb"
  },
  ]
}

First, before trying to start debugging, in a Terminal window, start IPython so you have a process to attach to.

I was initially running into problems with VSC complaining about not having a "program" entry in the json file, which seems irrelevant when attaching to process. It may in fact be completely irrelevant, but you have to add it to get things to work. I have not tried putting in a bogus entry for the program argument to see whether the value matter at all.

One other detail - sometime the breakpoints in VSC don't appear to be enabled, you get an unfilled circle instead of a red dot when you create them. But it seems you will hit the breakpoints regardless. However, based on my previous experience, it is best to import the package in iPython under test after attaching to the process, for best results.