CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsouplxml
Robby
Robby

Reputation: 401

scraping onclick

I am trying to scrape the reviews on this webpage.

http://www.tripadvisor.com/Hotel_Review-g294265-d2309275-Reviews-The_Forest_by_Wangz-Singapore.html

The only problem in each review is "More", which loads more text on OnClick event.

For example:

<span class="taLnk hvrIE6 tr147826763 moreLink" onclick = " ta.util.cookie.setPIDCookie(2247); ta.call('ta.servlet.Reviews.expandReviews', event,this,'review_147826763', '1', 2247)">
More </span>

How to scrap the complete review text using LXML/BeautifulSoup?

Upvotes: 1

Views: 1231

Answers (1)

user104365
user104365

Reputation:

This probably isn't the kind of answer you're looking for, but I've started looking at PhantomJS, which gives you a headless, scriptable, webkit browser. I'd bet it's an easier path than whatever ajax reverse engineering rabbit hole you're about to go down...

Upvotes: 1

Related Questions