CODEWITHSUNDEEP
python
Shooting Chuang
Shooting Chuang

Reputation: 83

Python How to open an URL and get source code at the same time?

What I had try are as following:

1)

response = urllib2.urlopen(url)
html = response.read()

In this way, I can't open the url in browser.

2)

webbrowser.open(url)

In this way, I can't get source code of the url.

So, how can I open an URL and get source code at the same time?

Thanks for your help.

Upvotes: 0

Views: 1562

Answers (1)

Tom Pitts
Tom Pitts

Reputation: 600

Have a look at BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/

You can request a website and then read the HTML source code from it:

import requests
from bs4 import BeautifulSoup

r = requests.get(YourURL)

soup = BeautifulSoup(r.content)
print soup.prettify()

If you want to read JavaScript, look into Headless Browsers.

Upvotes: 1

Related Questions