CODEWITHSUNDEEP
pythonseleniumweb-scraping
swifty
swifty

Reputation: 1282

Selenium - How much am I hitting the server?

I am using selenium to scrape information from a website.

The page is behind a login so I can't provide an example - but basically I need to gather data on approx. 800 fields on the one page.

Currently it boils down to me navigating to the correct page and then running

for i in driver.find_elements_by_xpath('//*[@id]'): some_list.append(i.get_attribute('textContent'))

My question is;

Or is the full page 'cached' and then I am simply reading the values that are already loaded?

Just want to make sure I'm being kind to the other party and not doing 800 calls for get_attribute!

Thanks.

Upvotes: 1

Views: 64

Answers (1)

Simon N
Simon N

Reputation: 337

get_attribute is retrieving data from sources already downloaded. You are not making requests to a web server when you execute that command

Upvotes: 5

Related Questions