CODEWITHSUNDEEP
python-3.xpython-requests
Varun G
Varun G

Reputation: 39

how to avoid SSL certificate error while scraping a webpage?

I am trying to scrape petrol prices from below page using requests, but am getting below error.

requests.exceptions.SSLError: HTTPSConnectionPool(host='www.goodreturns.in', port=443): Max retries exceeded with url: /petrol-price-in-karnataka-s17.html (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1131)')))

below is the code which I have tried.

import requests
r = requests.get('https://www.goodreturns.in/petrol-price-in-karnataka-s17.html',verify=False)

Upvotes: 1

Views: 2332

Answers (1)

Gprj
Gprj

Reputation: 32

You need to pass headers along with your request to make it authentic. Pass headers parameters as following:

import requests
headers = {'Host': 'www.goodreturns.in',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0'}
r = requests.get('https://www.goodreturns.in/petrol-price-in-karnataka-s17.html', headers = headers)

Upvotes: -1

Related Questions