CODEWITHSUNDEEP
pythonhtmlbeautifulsoup
Time4Boom
Time4Boom

Reputation: 55

Beautiful Soup returning empty html

So this is my second question regarding Beautiful Soup (sorry, im a beginner)

I was trying to fetch data from this website:

https://www.ccna8.com/ccna4-v6-0-final-exam-full-100-2017/

My Code:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

url = 'https://www.ccna8.com/ccna4-v6-0-final-exam-full-100-2017/'

uClient = uReq(url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "lxml")

print(page_soup)

But for some reason it returns an empty string.

I've been searching for similar threads and apparently it has something to do with the website using external api's , but this website doesn't.

network of website

Upvotes: 2

Views: 599

Answers (2)

johnII
johnII

Reputation: 1433

It seems that the content-type of the response if gzip so you need to handle that before you can process the html response.

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
import gzip

url = 'https://www.ccna8.com/ccna4-v6-0-final-exam-full-100-2017/'

uClient = uReq(url)
page_html = gzip.decompress(uClient.read())
uClient.close()
page_soup = soup(page_html, "lxml")
print(page_soup)

Upvotes: 2

Rakesh
Rakesh

Reputation: 82775

try using requests module

Ex:

import requests
from bs4 import BeautifulSoup as soup

url = 'https://www.ccna8.com/ccna4-v6-0-final-exam-full-100-2017/'

uClient = requests.get(url)
page_soup = soup(uClient.text, "lxml")
print(page_soup)

Upvotes: 2

Related Questions