CODEWITHSUNDEEP
pythonseleniumbeautifulsoup
Wicaledon
Wicaledon

Reputation: 810

Python - Item Price Web Scraping for Target

I'm trying to get any item's price from Target website. I did some examples for this website using selenium and Redsky API but now I tried to wrote bs4 code below:

import requests
from bs4 import BeautifulSoup

url = "https://www.target.com/p/ruffles-cheddar-38-sour-cream-potato-chips-2-5oz/-/A-14930847#lnk=sametab"
r= requests.get(url)
soup = BeautifulSoup(r.content, "lxml")

price = soup.find("div",class_= "web-migration-tof__PriceFontSize-sc-14z8sos-14 elGGzp")
print(price)

But it returns me None .

I tried soup.find("div",{'class': "web-migration-tof__PriceFontSize-sc-14z8sos-14 elGGzp"})

What am I missing?

I can accept any selenium code or Redsky API code but my priority is bs4

Upvotes: 0

Views: 947

Answers (3)

cruisepandey
cruisepandey

Reputation: 29372

use .text

price = soup.find("div",class_= "web-migration-tof__PriceFontSize-sc-14z8sos-14 elGGzp")
print(price.text)

Upvotes: 0

chitown88
chitown88

Reputation: 28630

The page is dynamic. The data is rendered after the initial request is made. You can use selenium to load the page, and once it's rendered, then you can pull out the relevant tag. API though is always the preferred way to go if it's available.

from selenium import webdriver
from bs4 import BeautifulSoup

driver = webdriver.Chrome('C:/chromedriver_win32/chromedriver.exe')

# If you don't want to open a browser, comment out the line above and uncomment below
#options = webdriver.ChromeOptions()
#options.add_argument('headless')
#driver = webdriver.Chrome('C:/chromedriver_win32/chromedriver.exe', options=options)

url = "https://www.target.com/p/ruffles-cheddar-38-sour-cream-potato-chips-2-5oz/-/A-14930847#lnk=sametab"
driver.get(url)
r = driver.page_source
soup = BeautifulSoup(r, "lxml")

price = soup.find("div",class_= "web-migration-tof__PriceFontSize-sc-14z8sos-14 elGGzp")
print(price.text)

Output:

$1.99

Upvotes: 2

Prophet
Prophet

Reputation: 33361

You are simply using wrong locator.
Try this

price_css_locator = 'div[data-test=product-price]'

or in XPath style

price_xpath_locator = '//div[@data-test="product-price"]'

With bs4 it should be something like this:

soup.select('div[data-test="product-price"]')

to get the element get you just need to add .text

price = soup.select('div[data-test="product-price"]').text
print(price)

Upvotes: 0

Related Questions