CODEWITHSUNDEEP
pythonerror-handlingpython-requests
KM Zohir
KM Zohir

Reputation: 71

HTTPSConnectionPool SSL Error certificate verify failed

I'm working on web scraping some particular websites and therefor I use the python 3 requests package and beautifulsoup. While processing a test over some websites I got this error :

requests.exceptions.SSLError: HTTPSConnectionPool(host='autoglassbodyrepair.lawshield.co.uk', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))


import requests as rq
import bs4

current_url = 'autoglassbodyrepair.lawshield.co.uk'
try:
   req = rq.get(current_url)
except rq.exceptions.RequestException as e:
   print(e)
else:
   soup = bs4.BeautifulSoup(r.content, "html.parser")
   text = soup.findAll(text = True)

When I try over my browser it shows me that the certificate is expired yet I can process to the page with https barred and turned to red. What I want is if there's an exception which won't allow me to access the page I'd just ignore it and get to the next page to process but if there's no exception I'd process the current page and ignore those SSl certificate.

Thanks in advance for your help !

Upvotes: 1

Views: 18217

Answers (1)

KM Zohir
KM Zohir

Reputation: 71

I got it, it just need to ignore the certificate as the code below, you would get a warning as insecure connection.

req = rq.get(current_url, verify = False)

Upvotes: 5

Related Questions