Notebooks not persistent for Jupyter in Docker container?

Question

Running jupyter in a docker container is a good solution for me, but I'm having trouble getting the notebook files to persist as advertised in the documentation here.

The docs say that after the session is closed and the server shutsdown, the .ipynb (notebook) files should be persisted in the ./work directory however, for me they are not. I have created notebooks both in the root directory and also in the /work directory which appears in the Jupyter Home page, but neither are to be found after shutdown, and if I restart the server, they are not in the directory list anymore either. I've tried launching the container in two ways -- first as suggested by the docs (substituting latest for the image tag):

docker run -p 8888:8888 jupyter/scipy-notebook:latest

and second by creating a docker-compose.yml file which allows me to capture the command text options and avoid the token security (which I don't need) as follows:

version: '3'

  services: # jupyter notebook
    jupyter_notebook:
    image: jupyter/scipy-notebook
    volumes:
      - ./work:/work
    ports:
      - "8888:8888"
    command: "start.sh jupyter notebook --NotebookApp.token=''"

I am running under Ubuntu 18.04.1 LTS with docker 18.06.1-ce I am expecting to find the notebook (at least the one I created in the /work folder) in the host system's ./work folder which is in the directory where I launched docker (or docker-compose), but nothing is there.

Here is a session transcript:

   s@VC66:ls -la
   -rw-r--r-- 1 steve steve  232 Nov  7 22:45 docker-compose.yml
   drwxr-xr-x 2 steve steve 4096 Nov  7 21:34 work

   s@VC66:~/sambashare/jupyter$ cat docker-compose.yml 

   version: '3'

     services:
       jupyter_notebook:
         image: jupyter/scipy-notebook
         volumes:
           - ./work:/work
         ports:
           - "8888:8888"
         command: "start.sh jupyter notebook --NotebookApp.token=''"

   s@VC66:~/sambashare/jupyter$ docker-compose up

   Creating network "jupyter_default" with the default driver
   Creating jupyter_jupyter_notebook_1 ... done
   Attaching to jupyter_jupyter_notebook_1
   jupyter_notebook_1  | Container must be run with group "root" to update passwd file
   jupyter_notebook_1  | Executing the command: jupyter notebook --NotebookApp.token=
   jupyter_notebook_1  | [I 16:08:40.454 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
   jupyter_notebook_1  | [W 16:08:40.597 NotebookApp] All authentication is disabled.  Anyone who can connect to this server will be able to run code.
   jupyter_notebook_1  | [I 16:08:40.625 NotebookApp] JupyterLab extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
   jupyter_notebook_1  | [I 16:08:40.625 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] Serving notebooks from local directory: /home/jovyan
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] The Jupyter Notebook is running at:
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] http://(62b087792f87 or 127.0.0.1):8888/
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
   jupyter_notebook_1  | [I 16:08:58.820 NotebookApp] 302 GET / (172.21.0.1) 0.48ms
   jupyter_notebook_1  | [I 16:09:07.941 NotebookApp] Creating new file in /work
   jupyter_notebook_1  | [I 16:09:17.360 NotebookApp] Saving file at /work/untitled.txt
   jupyter_notebook_1  | [I 16:09:24.725 NotebookApp] Shutting down on /api/shutdown request.
   jupyter_notebook_1  | [I 16:09:24.727 NotebookApp] Shutting down 0 kernels
   jupyter_jupyter_notebook_1 exited with code 0
   s@VC666:~/sambashare/jupyter$ ls work
   s@VC66:~/sambashare/jupyter$ ls
   docker-compose.yml  work

As you can see, it says it saved "untitled.txt" in the /work dir, but upon exiting there's nothing in there.

So to further refine the problem here, I altered the docker-compose.yml file to run a simple python script to create a file in the /work dir and see if it persists. It does!

command: "python3 /work/test.py"  # rather than start.sh...

here's the python test.py script:

   import os
   import pytz
   from datetime import datetime

   dir = "/work"
   if not os.path.isdir(dir):
      dir = "" # to test outside docker container...

   nyc_time = datetime.now( pytz.timezone("America/New_York"))
   fname = os.path.join(dir,"test.txt")
   f = open(fname, 'w')
   f.write(f"Test time is {nyc_time}\n")
   f.close()
   exit()

This time, after docker-compose up, the work folder contains "test.txt" which contains

Test time is 2018-11-09 11:55:28.472581-05:00

So the docker container mounting the /work dir seems fine -- problem might be something the jupyter image is doing on shutdown perhaps?

Answer 1

I think your misconception is about the docker container using /work. AFAIKs it is /home/jovyan/work instead.

So you can solve your trouble by e.g. this volume mapping

mkdir -P /your-jupyter/work
docker run -p 8888:8888 -e JUPYTER_ENABLE_LAB=yes -v /your-jupyter/work:home/jovyan/work jupyter/scipy-notebook

HTH.