CODEWITHSUNDEEP
pythonweb-scraping
Toma Tomov
Toma Tomov

Reputation: 1704

Why does my scraping script returns empty result

I am practicing here and my goal is to retrieve these data from the page in the url variable:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

url = "https://www.newegg.com/global/bg-en/PS4-Accessories/SubCategory/ID-3142"

# opening connection, grabing the page
uClient = uReq(url)
page_html = uClient.read()
uClient.close()

# html parser
page_soup = soup(page_html, "html.parser")

# grabs each product
containers = page_soup.findAll("div", {"class": "item-container"})

for container in containers:
    brand = container.select("div.item-info")[0].a.img["title"]
    name = container.findAll("a", {"class": "item-title"})[0].text.strip()
    shipping = container.findAll("li", {"class": "price-ship"})[0].text.strip()

    print("brand " + brand)
    print("name " + name)
    print("shipping " + shipping)

Nothing more I can say for it :) I just simple as that but I still can't get it why no data is retrieved. Will be thankful for every advice!

Upvotes: 0

Views: 123

Answers (1)

santamanno
santamanno

Reputation: 636

You are invoking the find_all method with wrong arguments.

You should use the argument "class_" properly, according to the documentation found here:

https://www.crummy.com/software/BeautifulSoup/bs4/doc/#searching-by-css-class

Upvotes: 1

Related Questions