CODEWITHSUNDEEP
seleniuminternet-explorerdomviewstate
Eternity Tomorrow
Eternity Tomorrow

Reputation: 1

Selenium aspx results

I'm new to selenium, so apologies if the question isn't well formed. I'm looking at automating some data gathering. There is an ASP.NET site, where you can request data about a specific topic. When the data query completes the web page is updated with the results. When I use selenium to get the page_source, I don't see the data. I believe it's encoded in the __VIEWSTATE. Using IE DOM Explorer, the data is there in plain text.

Is there a way to pull the data from the DOM Explorer? Basically, I'd be happy to get a dump and parse it myself.

Any help would be greatly appreciated.

Upvotes: 0

Views: 659

Answers (1)

Eternity Tomorrow
Eternity Tomorrow

Reputation: 1

I think I figured this out. Thanks for all the input. I got clued in to the answer from this: Scraping dynamic content through Selenium?

What I ended up doing is first get the list of iframes on the page

driver.find_elements_by_tag_name("iframe")

Then switch to each iframe one at a time. In my case the first iframe was the one with the relevant information:

driver.switch_to.frame(driver.find_elements_by_tag_name("iframe")[0])

Once I was in the relevant frame then I could do:

html = driver.page_source

And that would give me the expected result.

It appears like the frame context is important. Once you're in the frame context then you could scrape the data.

Upvotes: 0

Related Questions