CODEWITHSUNDEEP
pythonbeautifulsoup
Hrithik Puri
Hrithik Puri

Reputation: 316

Beautiful Soup is giving none, though value is present

Why my following code is giving an output NONE

from bs4 import BeautifulSoup
import urllib3
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
urllib3.disable_warnings()
url = "https://www.amazon.ae/dp/B07N62ZGWQ/ref=br_msw_pdt-5?_encoding=UTF8&smid=ABO0A2K2SKD5O&pf_rd_m=A2KKU8J8O8784X&pf_rd_s=&pf_rd_r=GZ376KPZWEXC0TBHXPKY&pf_rd_t=36701&pf_rd_p=da8f558b-6ee9-4705-8e31-6a4c345b29e8&pf_rd_i=desktop"
http = urllib3.PoolManager()
r = http.request('GET', url)
content = r.data.decode('utf-8')
soup = BeautifulSoup(content,'html.parser')
print(soup.find('span', {'id':"priceblock_ourprice"}))

I tried using soup.find(id="priceblock_ourprice"), soup.select("#priceblock_ourprice"), but both gave None output .

ID is present on the page and has a value as well.

enter image description here

Upvotes: 1

Views: 1269

Answers (1)

Kafels
Kafels

Reputation: 4069

First, I altered urllib3 to requests because urllib3 was giving an exception

See the code below:

import requests
from bs4 import BeautifulSoup

url = "https://www.amazon.ae/dp/B07N62ZGWQ/ref=br_msw_pdt-5?_encoding=UTF8&smid=ABO0A2K2SKD5O&pf_rd_m=A2KKU8J8O8784X&pf_rd_s=&pf_rd_r=GZ376KPZWEXC0TBHXPKY&pf_rd_t=36701&pf_rd_p=da8f558b-6ee9-4705-8e31-6a4c345b29e8&pf_rd_i=desktop"

response = requests.request('GET', url)
content = response.content.decode('utf-8')
soup = BeautifulSoup(content, 'html.parser')
table = soup.find('table', attrs={'class': 'a-lineitem'})
price_blocks = table.find_all('span', attrs={'id': 'priceblock_ourprice'})
prices = [block.text for block in price_blocks]
print(prices)

Output console:

['AED26.00']

Upvotes: 2

Related Questions