Connecting Spyder to Docker Containers

Question

I've started working at a company that develops code using docker containers, which I have had no experience with thus far. The nature of my work is Data Science-y, and so I find Spyder to be an invaluable tool for such work.

I would like to connect spyder to the docker containers that are being used by my colleagues, but I am not sure how to, or if this is even possible. I was not able to find helpful material on this that I could comprehend.

I considered ditching Spyder in favor of VS Code, as it has the ability to connect to docker containers. But my best attempts at trying to recreate Spyder's functionality in VS Code were only partially successful.

Given the popularity of both Spyder and Docker, I thought this would be a straightforward thing to do. Anyway, I would greatly appreciate any information you might have on this topic. I suppose I could consider other IDEs if you are aware of any that can do this. The key features I need are the ability to launch an interactive python environment which allows me to run scripts in the docker, keep the variables stored after the script runs, using these variables to find where things are going wrong and easily create plots, and possibly also have access to a debugger like Spyder's.

I obviously don't want to bloat the Dockerfile and install Spyder inside the container, I'd like something to run on the outside but be able to connect into the docker container and use the python environment defined there.

The following two links were not helpful to me: Connect Spyder to a console in a docker container on a remote host Connecting Spyder to Remote Jupyter Notebook in a Docker Container

Answer 1

I'm not sure anyone cares, but I can briefly describe my solution. You can set up ssh in the docker container and then use Spyder's "connect to an existing kernel" feature to use Spyder as a sort of frontend for an ipython kernel running in the docker container. This feature allows you to connect to remote ipython kernels through ssh. It's a bit of an annoyance to setup, so if you can get by with just ipython sessions and ipdb inside the container, that's probably the way to go. But if you're really stuck trying to debug something and want the full Spyder frontend, this works reliably, in my limited experience.

If you do try this, just note that some versions of Spyder seem to simply crash anytime you try to connect to any existing kernel, in a docker container or not. So if the latest version doesn't work, try some other ones...

EDIT: I'm now especially not sure anyone cares, but I've abandoned Spyder since posting this question. It took too long to actually start getting work done when trying to rely on Spyder inside docker, which itself was buggy in its ability to connect to remote kernels and with respect to its debugger functionality. I got inconsistent experiences on Windows/Ubuntu. Instead, if I don't need to do any data visualizations, I just use ipython in the container.

If I need to do data visualizations, I usually do whatever I need to do in docker to get the data I need saved to a file. Then, I have a conda environment that I use to analyze the data in the file. Even outside of docker I've stopped using Spyder. Instead I write my files in vim, run them at the command line, and I use breakpoint() in my code for debugging; I set export PYTHONBREAKPOINT=ipdb.set_trace or IPython.embed depending on if I want ipdb to step around through the code, or if I just need to try some code out at a particular spot in the program in an interactive shell with all the program's variables defined at that point. In fact, you can launch an ipython shell from ipdb by issuing interact. This has been working well for me, and may work for others depending on the nature of the work/company. It is very much freeing not having to rely on bulky tools like Spyder.