CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsouppython-requests-html
Ganesh
Ganesh

Reputation: 33

How to Extract the texts behind the "Read More" in reviews?

This is the inspected source code image of the review

If i use that class in the python for extraction. It just return "READ MORE" string, It cant get the text behind the "READ MORE" My code is,

extra = []
e = j.find_all('span', {'class' : '_1EPkIx'})
extra.append([k.get_text() for k in e])

and it returns

[['READ MORE', 'READ MORE', 'READ MORE', 'READ MORE', 'READ MORE', 'READ MORE', 'READ MORE', 'READ MORE']]

Do you have any idea about this???

Upvotes: 0

Views: 78

Answers (1)

woblob
woblob

Reputation: 1377

This kind of javascript events are handled by selenium but not beautifulsoup.

import selenium
selenium.webdriver.find_element_by_css_selector('span._1EPkIx').click()

after that you can fetch webdriver.page_source and there suppose to be whole text

Upvotes: 1

Related Questions