CODEWITHSUNDEEP
pythonpython-3.xweb-scrapingbeautifulsoup
BellamyPT
BellamyPT

Reputation: 35

Python WebScraping - I can navigate the website but the request returns '403'

I want to Webscrape the website in the code, but I get a 403 code. I have searched all the topics regarding getting over this problem, and used the recommendations for defining the user agent, however it is still not working.

This is the code that I have tried.

from requests import get

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36'}
idealista = 'https://www.idealista.pt/comprar-casas/lisboa-distrito/pagina-1'
response = get(idealista, headers=headers)
print(response)

It is returning "Response [403]". How can I get the 200 code?

Upvotes: 0

Views: 324

Answers (1)

Mr_U4913
Mr_U4913

Reputation: 1354

If you still get forbidden, you should try adding more headers.

headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0',
           'referer': 'https://www.idealista.pt/comprar-casas/lisboa-distrito/pagina-1' }
request = requests.get('https://www.idealista.pt/comprar-casas/lisboa-distrito/pagina-1', headers=headers)

output 

<Response [200]>

Upvotes: 3

Related Questions