Use Django OAuth2 provider with JupyterHub

Question

I'm attempting to run a Django web application that pairs with a JupyterHub server, where users enter via the web app and are then granted access to a notebook server once they've signed in. To facilitate this, I'm attempting to use OAuth2, where Django provides the authentication and JupyterHub verifies users against that.

I'm using django-oauth-toolkit to provide the authentication service and linking against it using the Generic OAuthenticator. A docker-compose reference implementation is available here. Currently, the authorize redirect works, but some part of the token retrieval process throws the following error:

jupyterhub_1  | [I 2018-01-07 18:53:41.763 JupyterHub log:124] 302 GET /hub/oauth_login?next= → http://localhost:8000/o/authorize?client_id=5hICA5iNiBhBuROGzxGJqGQ7Ur7yH8dHi53aPLB5&response_type=code&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D&redirect_uri=http%3A%2F%2Flocalhost%3A8001%2Fhub%2Foauth_callback (@172.22.0.1) 3.98ms
django_1      | [07/Jan/2018 18:53:41] "GET /o/authorize?client_id=5hICA5iNiBhBuROGzxGJqGQ7Ur7yH8dHi53aPLB5&response_type=code&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D&redirect_uri=http%3A%2F%2Flocalhost%3A8001%2Fhub%2Foauth_callback HTTP/1.1" 301 0
django_1      | [07/Jan/2018 18:53:41] "GET /o/authorize/?client_id=5hICA5iNiBhBuROGzxGJqGQ7Ur7yH8dHi53aPLB5&response_type=code&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D&redirect_uri=http%3A%2F%2Flocalhost%3A8001%2Fhub%2Foauth_callback HTTP/1.1" 200 3159
django_1      | [07/Jan/2018 18:53:42] "POST /o/authorize/?client_id=5hICA5iNiBhBuROGzxGJqGQ7Ur7yH8dHi53aPLB5&response_type=code&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D&redirect_uri=http%3A%2F%2Flocalhost%3A8001%2Fhub%2Foauth_callback HTTP/1.1" 302 0
jupyterhub_1  | [W 2018-01-07 18:53:42.959 JupyterHub log:124] 405 POST /o/token (@127.0.0.1) 9.08ms
jupyterhub_1  | [E 2018-01-07 18:53:42.961 JupyterHub web:1590] Uncaught exception GET /hub/oauth_callback?code=Rz9OLMKqO0QBne5evvJJjusEFjEhto&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D (172.22.0.1)
jupyterhub_1  |     HTTPServerRequest(protocol='http', host='localhost:8001', method='GET', uri='/hub/oauth_callback?code=Rz9OLMKqO0QBne5evvJJjusEFjEhto&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D', version='HTTP/1.1', remote_ip='172.22.0.1', headers={'X-Forwarded-Host': 'localhost:8001', 'Accept-Encoding': 'gzip, deflate, br', 'X-Forwarded-Port': '8001', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36', 'Upgrade-Insecure-Requests': '1', 'Cache-Control': 'max-age=0', 'Referer': 'http://localhost:8000/o/authorize/?client_id=5hICA5iNiBhBuROGzxGJqGQ7Ur7yH8dHi53aPLB5&response_type=code&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D&redirect_uri=http%3A%2F%2Flocalhost%3A8001%2Fhub%2Foauth_callback', 'X-Forwarded-For': '172.22.0.1', 'X-Forwarded-Proto': 'http', 'Host': 'localhost:8001', 'Connection': 'close', 'Accept-Language': 'en-US,en;q=0.9', 'Cookie': '_xsrf=2|159cb5ee|fde7d35de59d079ff7b5b4e029156a50|1509298869; GUID_8800=6XuSYLOxGOHGBer7Ks3o; csrftoken=zDh7M6uxWbz8G83FZHPz29PBxRsj79m9x70bYc8PBOsOJAY3F9uNq60g2nHOpP56; sessionid=wfia2uydbydieqahawxlsk2rz45uhjoc; oauthenticator-state="2|1:0|10:1515351221|20:oauthenticator-state|120:ZXlKdVpYaDBYM1Z5YkNJNklDSWlMQ0FpYzNSaGRHVmZhV1FpT2lBaVlXUTBORGMzTUdWbVptWTVORE15T0dFek9EQmxOVGhqTUdJNVlXUTBaVGNpZlE9PQ==|c1315c25a514c4e01d49edeb1a0b4f9595c88b9f309ec041d31eca10c6510030"'})
jupyterhub_1  |     Traceback (most recent call last):
jupyterhub_1  |       File "/opt/conda/lib/python3.5/site-packages/tornado/web.py", line 1511, in _execute
jupyterhub_1  |         result = yield result
jupyterhub_1  |       File "/opt/conda/lib/python3.5/site-packages/oauthenticator/oauth2.py", line 182, in get
jupyterhub_1  |         user = yield self.login_user()
jupyterhub_1  |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 407, in login_user
jupyterhub_1  |         authenticated = yield self.authenticate(data)
jupyterhub_1  |       File "/opt/conda/lib/python3.5/site-packages/jupyterhub/auth.py", line 227, in get_authenticated_user
jupyterhub_1  |         authenticated = yield self.authenticate(handler, data)
jupyterhub_1  |       File "/opt/conda/lib/python3.5/site-packages/oauthenticator/generic.py", line 101, in authenticate
jupyterhub_1  |         resp = yield http_client.fetch(req)
jupyterhub_1  |     tornado.httpclient.HTTPError: HTTP 405: Method Not Allowed
jupyterhub_1  |     
jupyterhub_1  | [E 2018-01-07 18:53:42.965 JupyterHub log:116] {
jupyterhub_1  |       "X-Forwarded-Host": "localhost:8001",
jupyterhub_1  |       "Accept-Encoding": "gzip, deflate, br",
jupyterhub_1  |       "X-Forwarded-Port": "8001",
jupyterhub_1  |       "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
jupyterhub_1  |       "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36",
jupyterhub_1  |       "Upgrade-Insecure-Requests": "1",
jupyterhub_1  |       "Cache-Control": "max-age=0",
jupyterhub_1  |       "Referer": "http://localhost:8000/o/authorize/?client_id=5hICA5iNiBhBuROGzxGJqGQ7Ur7yH8dHi53aPLB5&response_type=code&state=eyJuZXh0X3VybCI6ICIiLCAic3RhdGVfaWQiOiAiYWQ0NDc3MGVmZmY5NDMyOGEzODBlNThjMGI5YWQ0ZTcifQ%3D%3D&redirect_uri=http%3A%2F%2Flocalhost%3A8001%2Fhub%2Foauth_callback",
jupyterhub_1  |       "X-Forwarded-For": "172.22.0.1",
jupyterhub_1  |       "X-Forwarded-Proto": "http",
jupyterhub_1  |       "Host": "localhost:8001",
jupyterhub_1  |       "Connection": "close",
jupyterhub_1  |       "Accept-Language": "en-US,en;q=0.9",
jupyterhub_1  |       "Cookie": "_xsrf=2|159cb5ee|fde7d35de59d079ff7b5b4e029156a50|1509298869; GUID_8800=6XuSYLOxGOHGBer7Ks3o; csrftoken=zDh7M6uxWbz8G83FZHPz29PBxRsj79m9x70bYc8PBOsOJAY3F9uNq60g2nHOpP56; sessionid=wfia2uydbydieqahawxlsk2rz45uhjoc; oauthenticator-state=\"2|1:0|10:1515351221|20:oauthenticator-state|120:ZXlKdVpYaDBYM1Z5YkNJNklDSWlMQ0FpYzNSaGRHVmZhV1FpT2lBaVlXUTBORGMzTUdWbVptWTVORE15T0dFek9EQmxOVGhqTUdJNVlXUTBaVGNpZlE9PQ==|c1315c25a514c4e01d49edeb1a0b4f9595c88b9f309ec041d31eca10c6510030\""
jupyterhub_1  |     }

All necessary code and configurations are available via the source link above. Am I using/configuring the authentication incorrectly? Have I hit a bug in one of the underlying libraries (I'm especially suspicious of JupyterHub since it's in early active development)?

EDIT: I've posted this as an issue on the OAuthenticator github, and am still getting no help. I can't find anyone else that has linked up these two services before, but they share the OAuth2 protocol and so should work with each other. I've dug throught the source code of both the JupyterHub Generic OAuthenticator and the Django OAuth2 provider, and can't figure out why this error would get thrown. Can anyone help me figure this out?

Answer 1

Turned out to be a few minor bugs I had to work through to get this to work:

1. The URL used to get a token is relative to the JupyterHub server, NOT relative to the client/browser like the authorization url is. In the Docker Compose example provided, the django authentication server is "localhost:8000" relative to the client, but "django:8000" relative to the JupyterHub server.
2. The appropriate hostnames (both "localhost" and "django") therefore needed to be in the Django app's ALLOWED_HOSTS list.

3. I'm not positive it's necessary, but I also added the middleware suggested by the oauth-toolkit documentation:

   MIDDLEWARE = [ ..., 'oauth2_provider.middleware.OAuth2TokenMiddleware', ]

4. JupyterHub also expects a userdata URL to get a username. This must be provided in the OAUTH2_USERDATA_URL environment variable (again, using a URL relative to the JupyterHub server), and that URL must return a JSON blob with, at least, a 'username' key.

Complete diffs from the previous code to the working example are available at this commit (as well as a complete, minimal example in that repository).