CODEWITHSUNDEEP
pythonpython-requests
silpa
silpa

Reputation: 57

Page is giving 403 response when tried to get the data

I'm trying to request this page https://health.usnews.com/best-hospitals/rankings/cancer using Python(2.7) requests module. But it's giving 403 response(It's working fine on my local machine but not working on the server).

requested the page bypassing headers and cookies in the request. But got 403 response. Also, tried the Session object as well as suggested in Python requests - 403 forbidden - despite setting `User-Agent` headers

>>> requests.get('https://health.usnews.com/best-hospitals/rankings/cancer')
<Response [403]>
>>> requests.get('https://health.usnews.com/best-hospitals/rankings/cancer', headers=h)
<Response [403]>

How can we get the proper response from that page?

Thank you in advance!

Upvotes: 0

Views: 116

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195528

User-Agent in headers is needed when making request:

import requests

url = 'https://health.usnews.com/best-hospitals/rankings/cancer'
headers = {'User-Agent':'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0'}

txt = requests.get(url, headers=headers).text
print(txt)

Prints:

<!doctype html>
<html class="no-js" lang="">
    <head>
... and so on.

Upvotes: 1

Related Questions