Jupyter and IPython Kernel fails to start on RHEL 7.6

Question

I am stuck... I am trying to set up a Jupyter server on a newly built enterprise Linux box. I have both Miniconda and Anaconda routes, with Python 3.7 and 3.6. I have tried installing and reinstalling Jupyter / IPython separately with pip and conda.

Inevitably, the browser-based Jupyter notebook fails to execute any code, it is stuck in the In [*] state. This indicates that the kernel is malfunctioning.

Indeed,

$ jupyter console --debug
[ZMQTerminalIPythonApp] Searching ['/home/scadmin', '/home/scadmin/.jupyter', '/opt/conda/a3/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[ZMQTerminalIPythonApp] Looking for jupyter_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /opt/conda/a3/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /home/scadmin/.jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_config in /home/scadmin
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /usr/local/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /opt/conda/a3/etc/jupyter
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /home/scadmin/.jupyter
[ZMQTerminalIPythonApp] Loaded config file: /home/scadmin/.jupyter/jupyter_console_config.py
[ZMQTerminalIPythonApp] Looking for jupyter_console_config in /home/scadmin
[ZMQTerminalIPythonApp] Connection File not found: /run/user/1008/jupyter/kernel-19936.json
[ZMQTerminalIPythonApp] Starting kernel: ['/opt/conda/a3/bin/python', '-m', 'ipykernel_launcher', '-f', '/run/user/1008/jupyter/kernel-19936.json']
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:47202
[ZMQTerminalIPythonApp] connecting shell channel to tcp://127.0.0.1:47589
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:47589
[ZMQTerminalIPythonApp] connecting iopub channel to tcp://127.0.0.1:42874
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:42874
[ZMQTerminalIPythonApp] connecting stdin channel to tcp://127.0.0.1:32777
[ZMQTerminalIPythonApp] Connecting to: tcp://127.0.0.1:32777
[ZMQTerminalIPythonApp] connecting heartbeat channel to tcp://127.0.0.1:44354
Traceback (most recent call last):
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_console/ptshell.py", line 341, in init_kernel_info
    reply = self.client.get_shell_msg(timeout=1)
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_client/client.py", line 77, in get_shell_msg
    return self.shell_channel.get_msg(*args, **kwargs)
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_client/blocking/channels.py", line 57, in get_msg
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/a3/bin/jupyter-console", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_core/application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/conda/a3/lib/python3.7/site-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-116>", line 2, in initialize
  File "/opt/conda/a3/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_console/app.py", line 142, in initialize
    self.init_shell()
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_console/app.py", line 115, in init_shell
    confirm_exit=self.confirm_exit,
  File "/opt/conda/a3/lib/python3.7/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_console/ptshell.py", line 287, in __init__
    self.init_kernel_info()
  File "/opt/conda/a3/lib/python3.7/site-packages/jupyter_console/ptshell.py", line 344, in init_kernel_info
    raise RuntimeError("Kernel didn't respond to kernel_info_request")
RuntimeError: Kernel didn't respond to kernel_info_request
$ [IPKernelApp] WARNING | Parent appears to have exited, shutting down.

Any help and ideas will be appreciated...

Answer 1

Came down to the firewall policy dropping TCP traffic by default, except for specified ports.

Adding this to /etc/sysconfig/iptables followed by systemctl restart iptables fixed the issue.

-A INPUT -p tcp -m tcp -s localhost --dport 32768:65535 -m state --state NEW -m comment --comment "Jupyter ZeroMQ" -j ACCEPT

Note that Jupyter documentation asks for port range 49152:65535 for the ZeroMQ communication. In fact, the port range it picks is wider, therefore start at 32768.