CODEWITHSUNDEEP
pythonsslopensslpython-requests
NerdSec
NerdSec

Reputation: 105

cURL queries work, but python-request fails

Summary

We are using a custom CA in our enviroment. I have imported the CA certificates in the trust store by downloading the Base64 certs and updating the trust store using update-ca-certificates.

I am able to run cURL queries to my REST API, however the request library fails with an SSL error when running the same.

What I have tried

I have tried specifying the root ca certificate file path to the library, but got the same error. How do I troubleshoot this issue? Setting verify to false is not an option.

Code Run

Works

curl -X GET https://api.me.com/admin/ -H 'Authorization: Token 4ae5'

Doesn't work

requests.get('https://api.me.com/admin/', headers={'Authorization': 'Token 4ae5'}, verify='/etc/ssl/certs/root.pem')

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 844, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/util/ssl_.py", line 325, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 377, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 752, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 988, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 633, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/usr/local/lib/python3.5/dist-packages/urllib3/connectionpool.py", line 630, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 480, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 588, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 447, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

Upvotes: 1

Views: 5876

Answers (3)

StillAyo
StillAyo

Reputation: 21

To add on, for the python requests to work, you may need to create the certificate chain file:

cat your_cert.crt root_cert.crt > chain.pem

You can then directly pass this in your requests call:

chain_cert_path = 'chain.pem'
response = requests.get(url, verify=chain_cert_path)

Upvotes: 0

Martin Hepp
Martin Hepp

Reputation: 1585

The reason is that Python Requests uses certificates from the python-certifi package., not those of the underlying operating system.

certifi includes all CA certificates from Mozilla, and it is also relatively straightforward to add missing certificates to it.

See this post for details on adding certificates to certifi: https://stackoverflow.com/a/66111417/516699

The good thing is also that you do not tweak the SSL of the underlying system, just that of the Python environment your are working in.

Upvotes: 1

Steffen Ullrich
Steffen Ullrich

Reputation: 123250

... requests.get(..., verify='/etc/ssl/certs/root.pem')

Given your description of using update-ca-certificates it looks like you are using Debian or similar (i.e. Ubuntu). In this case the relevant path should be /etc/ssl/certs/ca-certificates.crt.

Upvotes: 3

Related Questions