Can't get virtualenv working due to SSL error

Question

I'm trying to use virtualenv on my Mac (El Capitan) but something must have changed recently, as it's no longer working, and complains about SSL problems.

It's the first time I've had to use this project in about 9 months, and in that time, I've not changed anything on my system regarding Python.

In an effort to get it working, over the past day or so I've uninstalled virtualenv, manually deleted some files (I found these instructions in a GitHub issue, but I can't find them again to provide a link, but if I remember correctly, it was just deleting broken symlinks and directories relating to virtualenv,) and then uninstalled Python via Brew, leaving me with a system that had no Python, pip, or virtualenv libraries or executables on it.

Then I ran brew install python2 to get Python installed. Then I ran pip install virtualenv which gave me the same SSL error as before. I manually downloaded virtualenv from the URL pip was trying to download it from, and then pip installed it locally using the downloaded file. This worked. However, when I then run virtualenv venv, I still get the same SSL error.

I've seen issues offering all kinds of advice, from downloading DigiCertHighAssuranceEVRootCA.pem and using it via the --cert option, creating a pip.ini file with trusted-host=files.pythonhosted.org in it, using the --trusted-host option, using a different network that isn't my company's corporate network, but none of it so far has worked for me.

I am consistently getting the following SSL error whenever I try to pip install anything, or, in this specific example, create a new virtualenv:

New python executable in /Users/apl/Projects/fdm/FDM-fordele/venv/bin/python2.7
Also creating executable in /Users/apl/Projects/fdm/FDM-fordele/venv/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /Users/apl/Projects/...e/venv/bin/python2.7 - setuptools pip wheel:
  Looking in links: /usr/local/lib/python2.7/site-packages, /usr/local/lib/python2.7/site-packages/virtualenv_support, /usr/local/lib/python2.7/site-packages/virtualenv_support
Collecting setuptools
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl?X-OpenDNS-Session=_0a895c9402cc904c260a26007e7f555cbe499270f445_8d79d41f_
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl?X-OpenDNS-Session=_0a895c9402cc904c260a26007e7f555cbe499270f445_8d79d41f_
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl?X-OpenDNS-Session=_0a895c9402cc904c260a26007e7f555cbe499270f445_8d79d41f_
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl?X-OpenDNS-Session=_0a895c9402cc904c260a26007e7f555cbe499270f445_8d79d41f_
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl?X-OpenDNS-Session=_0a895c9402cc904c260a26007e7f555cbe499270f445_8d79d41f_
Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org.x.0a895c9402cc904c260a26007e7f555cbe49.9270f445.id.opendns.com', port=443): Max retries exceeded with url: /s/files.pythonhosted.org/packages/96/06/c8ee69628191285ddddffb277bd5abdf769166e7a14b867c2a172f0175b1/setuptools-40.4.3-py2.py3-none-any.whl?X-OpenDNS-Session=_0a895c9402cc904c260a26007e7f555cbe499270f445_8d79d41f_ (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),))

Is there some other way I can get this to work? This project cannot be deployed (using an Ansible role called ansistrano) because of this error.

EDIT:

I just tried uninstalling/removing Virtualenv, Pip and Python again, reinstalling Python, then manually installing pip with https://bootstrap.pypa.io/get-pip.py, which gives:

Collecting pip
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl?X-OpenDNS-Session=_0cf307740aff804587084980d2ecedab4cc89270f445_80300092_
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl?X-OpenDNS-Session=_0cf307740aff804587084980d2ecedab4cc89270f445_80300092_
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl?X-OpenDNS-Session=_0cf307740aff804587084980d2ecedab4cc89270f445_80300092_
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl?X-OpenDNS-Session=_0cf307740aff804587084980d2ecedab4cc89270f445_80300092_
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),)': /s/files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl?X-OpenDNS-Session=_0cf307740aff804587084980d2ecedab4cc89270f445_80300092_
Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org.x.0cf307740aff804587084980d2ecedab4cc8.9270f445.id.opendns.com', port=443): Max retries exceeded with url: /s/files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl?X-OpenDNS-Session=_0cf307740aff804587084980d2ecedab4cc89270f445_80300092_ (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)'),))

Answer 1

So it turns out that this is because the company I work for use Cisco's Umbrella DNS security system, that inserts itself in all DNS requests. This breaks SSL in Python, apparently, even though I can successfully manually download the file with curl.

By adding

151.101.1.63 files.pythonhosted.org

to my hosts file, I was able to get a system that behaved exactly as expected. Hope this helps others in the future who also are forced to go through Umbrella.