CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Arslan Aziz
Arslan Aziz

Reputation: 213

I scrape the review of post but they don't scrape

I scrape the review of post but they don't scrape solve it I am very thankful

import requests
from bs4 import BeautifulSoup


headers ={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36'
}

r =requests.get('https://www.realpatientratings.com/botox-cosmetic') 
soup=BeautifulSoup(r.content, 'lxml')
tag = soup.find_all('p',class_='text')
for u in tag:
    print(u.text)

Upvotes: 0

Views: 35

Answers (1)

Joaquin
Joaquin

Reputation: 2091

After checking xhr requests I found out that you're getting the incorrect page. Try:

import requests
from bs4 import BeautifulSoup


headers ={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36'
}

r =requests.get('https://www.realpatientratings.com/reviews/procreviewfilters?type=surgical&star=&procedureId=147&sort=new&location=&state=0&within=0') 
soup=BeautifulSoup(r.content, 'lxml')
tag = soup.find_all('p',class_='text')
for u in tag:
    print(u.text)

Just changed https://www.realpatientratings.com/botox-cosmetic to https://www.realpatientratings.com/reviews/procreviewfilters?type=surgical&star=&procedureId=147&sort=new&location=&state=0&within=0

Upvotes: 1

Related Questions