Python requests throwing "certificate verify failed: unable to get local issuer certificate" error

Question

When trying to call an API using python Requests I'm getting certificate verify failed: unable to get local issuer certificate error.

I have added the SSL certificate in cacert.pem of certifi. This did not fix the problem.

data = {'foo':'bar'}
url = 'hostname'
r = requests.post(url, data=data)

print(r) 

The above code only works when I put verify=False in the above code.

I have also extracted SSL certificate of API using the below code and added it to certifi cacert.pem but that too did not work.

import ssl, socket
myhostname = 'hostname'
myctx = ssl.create_default_context()
myctx.check_hostname = False
myctx.verify_mode = ssl.CERT_NONE
s = myctx.wrap_socket(socket.socket(), server_hostname=myhostname)
s.connect((myhostname, 443))

bcert = s.getpeercert(binary_form=True)
cert = ssl.DER_cert_to_PEM_cert(bcert)

Below is the error that is getting thrown

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connection.py", line 361, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 377, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)

Answer 1

Try running the program code below:

# install_certifi.py
#
# sample script to install or update a set of default Root Certificates
# for the ssl module.  Uses the certificates provided by the certifi package:
#       https://pypi.python.org/pypi/certifi

import os
import os.path
import ssl
import stat
import subprocess
import sys

STAT_0o775 = ( stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
             | stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
             | stat.S_IROTH |                stat.S_IXOTH )


def main():
    openssl_dir, openssl_cafile = os.path.split(
        ssl.get_default_verify_paths().openssl_cafile)

    print(" -- pip install --upgrade certifi")
    subprocess.check_call([sys.executable,
        "-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"])

    import certifi

    # change working directory to the default SSL directory
    os.chdir(openssl_dir)
    relpath_to_certifi_cafile = os.path.relpath(certifi.where())
    print(" -- removing any existing file or link")
    try:
        os.remove(openssl_cafile)
    except FileNotFoundError:
        pass
    print(" -- creating symlink to certifi certificate bundle")
    os.symlink(relpath_to_certifi_cafile, openssl_cafile)
    print(" -- setting permissions")
    os.chmod(openssl_cafile, STAT_0o775)
    print(" -- update complete")

if __name__ == '__main__':
    main()

Answer 2

I have also extracted SSL certificate of API

My guess is that you used leaf server certificate as CA, and that's not how cert validation works. The SSL library on your side is trying to build a trust chain. Typical server certificate chain looks like this:

(CA ->) Intermediate CA -> server certificate

The CA cert is not sent in the TLS handshake with the server. It's your ssl client's job to find a matching trusted CA for the chain.

You have to add CA to CA_BUNDLE not leaf server cert.

How to get the CA cert? Try in your browser and click on the lock, the CA cert will be upmost in the chain if it's found.

You can also inspect the cert and find out who signed it (issuer attribute). Maybe you'll be able to find that issuer's cert in some public CA database.