CODEWITHSUNDEEP
pythondebuggingipythonjupyter-notebookspyder
P i
P i

Reputation: 30684

How to debug external .py functions run from Jupyter/IPython notebook

My Jupyter/IPython notebook executes functions in an external .py.

I need to set breakpoints within these functions, inspect variables, single step, etc.

It just isn't practical to use a combination of print statements and throwing exceptions to early-exit a cell.

I need some kind of workflow.

Is it possible to hook up some third-party editor/IDE to view the .py and somehow connect it to the Python runtime Jupyter/IPython is using?

So that if I set a breakpoint in my external .py using my IDE and execute a cell in the notebook which encounters said breakpoint, I can continue to navigate manually from within the IDE.

EDIT: I've found https://pypi.python.org/pypi/ipdb https://www.quora.com/What-are-your-favorite-tricks-for-IPython-Notebook

EDIT https://www.youtube.com/watch?v=Jb2HHOahvcE <-- this video is getting close to what I'm after, I just can't quite see how to put it all together. That video demonstrates spyder which is an IDE with an IPython prompt... I wonder if maybe I can run my notebook through the prompt and debug it.

EDIT: It looks as though PyCharm does exactly what I'm after: https://www.jetbrains.com/help/pycharm/2016.1/tutorial-using-ipython-jupyter-notebook-with-pycharm.html

EDIT: I'm in the middle of trying to get PyCharm to behave. I will provide the details in an answer if I sort it out.

Upvotes: 13

Views: 6965

Answers (3)

David Taieb
David Taieb

Reputation: 206

We've just released the first version of a Python Visual Debugger for Jupyter Notebooks which could be of interest. You can find the details here.
(Disclaimer: I'm the lead developer)

Upvotes: 5

Daniel
Daniel

Reputation: 12026

Is it possible to hook up some third-party editor/IDE to view the .py and somehow connect it to the Python runtime Jupyter/IPython is using?

Yes, it's possible.

Using Emacs as your third party Python IDE.

You can open a file side-by-side with the iPython runtime. It is possible to send definitions from the external file directly to the prompt using key shortcuts, and set breakpoints, single step and inspect variables from the iPython prompt using the standard python debugger pdb.

Auto-completion works fine. Simply writing import XXX makes Emacs aware of all functions within the XXX module.

Here is an example screenshot: Spacemacs

And these are the steps I followed to get to the screenshot.*

  1. Open file playground.py on Emacs.
  2. Press C-c C-p to fire up an iPython process and connect file to it. Note that the iPython process is fully functional. I can call all magic methods on it.
  3. Press M-m m d b to include the breakpoint in the line I want (in form of a pdb import)
  4. Press C-M-x to send the definition to the iPython prompt. Note in the screenshot that I am calling unique_everseen without ever having typed it into the prompt. The definition was sent directly from the file (that is why we have an empty cell 2, as a visual feedback that Emacs did something).
  5. Execute the function from iPython prompt so as to trigger the breakpoint. I get information telling me what line I am and enter the debugger automatically.
  6. I inspect variables to see what is going on.
  7. I press n to step to the next instruction.
  8. I click on the filename (displayed in red on the prompt) and it takes my cursor directly to that line within the playground.py file. Note the black triangles next to the line numbers telling me where the stepper is in. Pretty nice.

Now, I haven't even scratched the surface of what Emacs can offer. If this interests you I'd suggest looking into it. It's really powerful and has a great, very helpful and active community.

To get all Python configuration running out of the box without having to configure anything yourself simply install Spacemacs, a popular distribution of Emacs (what anaconda is to Python, Spacemacs is to Emacs). The install instructions can be found under the link.

Using Emacs to edit iPython notebooks.

A package called EIN (Emacs iPython Notebooks) "provides a IPython Notebook client and integrated REPL in Emacs." To get it working you open your iPython notebook server from any terminal with ipython notebook, and call the function ein:notebooklist.

More information can be found under the project's page here, and on the Spacemacs iPython-Notebook layer page here.

In particular, from EIN documentation:

  1. Copy/paste cells, even to/from different notebooks.
  2. Console integration: You can easily connect to a kernel via the console application. This enables you to start debugging in the same kernel. It is even possible to connect to a console over ssh.
  3. An IPython kernel can be "connected" to any Emacs buffer. This enables you to evaluate a buffer or buffer region using the same kernel as the notebook. Notebook goodies such as tooltip help, help browser and code completion are available in these buffers.
  4. Jump to definition (go to the definition by hitting M-. over an object).

All functionality described above for the simple iPython + file workflow is available for iPython notebooks (*.ipynb).

Screenshots here.

Upvotes: 6

Vinod
Vinod

Reputation: 549

In Jupyter, you can use python debugger by adding below two lines for a breakpoint. 

import pdb
pdb.set_trace()

Code execution will pause at this step and will provide you text box for debugging python code. I have attached screenshot for same.

You can refer to pdb documentation for the operations you can perform apart from printing your variables

PDB in Jupyter Notebook

Upvotes: 6

Related Questions