CODEWITHSUNDEEP
htmlpython-3.xweb-scrapingbeautifulsoup
mistersenpai
mistersenpai

Reputation: 59

beautiful soup unable to find elements from website

It's my first time working with web scraping so cut me some slack. I'm trying to pull the "card_tag" from a website. I triple checked that the card tag is inside their respected tags as seen in the code.

import requests
from bs4 import BeautifulSoup


result = requests.get("https://www.anime-planet.com/users/mistersenpai/anime/dropped")
src = result.content
soup = BeautifulSoup(src, features="html.parser")


urls = []
for div_tag in soup.find_all('div id="siteContainer"'):
    ul_tag = div_tag.find("ul class")
    li_tag = ul_tag.find("li")
    card_tag = li_tag.find("h3")
    urls.append(card_tag)




print(urls)

When I go to print the url list it outputs nothing. You can see the thing I'm looking for by visiting the link as seen in the code and inspecting element on "Blood-C". As you can see it's listed in the tag I'm trying to find, yet my code can't seem to find it. Any help would be much appreciated.

Upvotes: 0

Views: 891

Answers (1)

chitown88
chitown88

Reputation: 28630

just minor syntax you need to change with the tags and attributes.

import requests
from bs4 import BeautifulSoup


result = requests.get("https://www.anime-planet.com/users/mistersenpai/anime/dropped")
src = result.content
soup = BeautifulSoup(src, features="html.parser")


urls = []
containers = soup.find_all('div', {'id':'siteContainer'})
for div_tag in containers:
    ul_tag = div_tag.find("ul", {'data-type':'anime'})
    li_tag = ul_tag.find_all("li")
    for each in li_tag:
        card_tag = each.find("h3")
        urls.append(card_tag)
        print(card_tag)

Also, you could just skip all that and go straight to those <h3> tags with the class attribute cardName:

import requests
from bs4 import BeautifulSoup


result = requests.get("https://www.anime-planet.com/users/mistersenpai/anime/dropped")
src = result.content
soup = BeautifulSoup(src, features="html.parser")


urls = []
for card_tag in soup.find_all('h3', {'class':'cardName'}):
    print(card_tag)
    urls.append(card_tag)

Output:

<h3 class="cardName">Black Butler</h3>
<h3 class="cardName">Blood-C</h3>
<h3 class="cardName">Place to Place</h3>

Upvotes: 2

Related Questions