How to disable "check_hostname" using Requests library and Python 3.8.5?

Question

using latest Requests library and Python 3.8.5, I can't seem to "disable" certificate checking on my API call. I understand the reasons not to disable, but I'd like this to work.

When i attempt to use "verify=True", the servers I connect to throw this error:

(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))

When i attempt to use "verify=False", I get:

Error making PS request to [<redacted server name>] at URL https://<redacted server name/rest/v2/api_endpoint: Cannot set verify_mode to CERT_NONE when check_hostname is enabled.

I don't know how to also disable "check_hostname" as I haven't seen a way to do that with the requests library (which I plan to keep and use).

My code:

self.ps_server = server
self.ps_base_url = 'https://{}/rest/v2/'.format(self.ps_server)
url = self.ps_base_url + endpoint

response = None
try:
    if req_type == 'POST':
        response = requests.post(url, json=post_data, auth=(self.ps_username, self.ps_password), verify=self.verify, timeout=60)
        return json.loads(response.text)
    elif req_type == 'GET':
        response = requests.get(url, auth=(self.ps_username, self.ps_password), verify=self.verify, timeout=60)
        if response.status_code == 200:
            return json.loads(response.text)
        else:
            logging.error("Error making PS request to [{}] at URL {} [{}]".format(server, url, response.status_code))
            return {'status': 'error', 'trace': '{} - {}'.format(response.text, response.status_code)}
    elif req_type == 'DELETE':
        response = requests.delete(url, auth=(self.ps_username, self.ps_password), verify=self.verify, timeout=60)
        return response.text
    elif req_type == 'PUT':
        response = requests.put(url, json=post_data, auth=(self.ps_username, self.ps_password), verify=self.verify, timeout=60)
        return response.text
except Exception as e:
    logging.error("Error making PS request to [{}] at URL {}: {}".format(server, url, e))
    return {'status': 'error', 'trace': '{}'.format(e)}

Can someone shed some light on how I can disable check_hostname as well, so that I can test this without SSL checking?

Answer 1

If you have pip-system-certs, it monkey-patches requests as well. Here's a link to the code: https://gitlab.com/alelec/pip-system-certs/-/blob/master/pip_system_certs/wrapt_requests.py

After digging through requests and urllib3 source for awhile, this is the culprit in pip-system-certs:

ssl_context = ssl.create_default_context()
ssl_context.load_default_certs()
kwargs['ssl_context'] = ssl_context

That dict is used to grab an ssl_context later from a urllib3 connection pool but it has .check_hostname set to True on it.

As far as replacing the utility of the pip-system-certs package, I think forking it and making it only monkey-patch pip would be the right way forward. That or just adding --trusted-host args to any pip install commands.

EDIT:

Here's how it's normally initialized through requests (versions I'm using):

https://github.com/psf/requests/blob/v2.21.0/requests/adapters.py#L163

def init_poolmanager(self, connections, maxsize, block=DEFAULT_POOLBLOCK, **pool_kwargs):
    """Initializes a urllib3 PoolManager.
    This method should not be called from user code, and is only
    exposed for use when subclassing the
    :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`.
    :param connections: The number of urllib3 connection pools to cache.
    :param maxsize: The maximum number of connections to save in the pool.
    :param block: Block when no free connections are available.
    :param pool_kwargs: Extra keyword arguments used to initialize the Pool Manager.
    """
    # save these values for pickling
    self._pool_connections = connections
    self._pool_maxsize = maxsize
    self._pool_block = block

    # NOTE: pool_kwargs doesn't have ssl_context in it
    self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize,
                                   block=block, strict=True, **pool_kwargs)

And here's how it's monkey-patched:

def init_poolmanager(self, *args, **kwargs):
    import ssl
    ssl_context = ssl.create_default_context()
    ssl_context.load_default_certs()
    kwargs['ssl_context'] = ssl_context
    return super(SslContextHttpAdapter, self).init_poolmanager(*args, **kwargs)