CODEWITHSUNDEEP
pythonselenium
apogne
apogne

Reputation: 3645

Selenium: how to get the entire html as a string?

I am using Selenium with python. See the following code:

from selenium.webdriver.common.keys import Keys
import selenium.webdriver
driver = selenium.webdriver.Firefox()
driver.get("http://finance.yahoo.com/q?s=APP")

Now, I want to do one simple thing: get the html of that web as a string from the driver. Then, I can use BeautifulSoup to parse it. Does anyone know this?

Actually, I don't how to access information from this driver, e.g., to get the stock price of apple in this case.

I am totally new to Selenium. A good tutorial for it is highly appreciated.

Thank you!

Upvotes: 1

Views: 3021

Answers (2)

Abdul Majeed
Abdul Majeed

Reputation: 2781

Have a look at following code.

from selenium.webdriver.common.keys import Keys
import selenium.webdriver
driver = selenium.webdriver.Firefox()
driver.get("http://finance.yahoo.com/q?s=APP")
page_html = driver.page_source

In page_html you will have html of opened page.

Upvotes: 2

Gregory Kremler
Gregory Kremler

Reputation: 146

You're looking for page_source.

To continue along with your example: 

soup = BeautifulSoup(driver.page_source)

As another commenter noted however, you could use a library like requests to the same effect:

r = requests.get('http://finance.yahoo.com/q?s=APP')
soup = BeautifulSoup(r.content)

Upvotes: 1

Related Questions