CODEWITHSUNDEEP
pythonbeautifulsoup
jjyoh
jjyoh

Reputation: 426

multiple class with beautifulsoup

I'm trying to scrap html data with beautifulsoup and I'd like to know how do you scrap a class who owns a class. (double class). Here my html code to understand : 

<span class="phone-contacts">
  <span class = "phone">
    <span class = "label">
      phone
     </span>
    <span class = "value">
       <a class="tel" href="tel:+41XXXXXX">
        021 XXX XX XX
       </a>
     </span>
  </span>
  <span class = "mobile">
    <span class = "label">
     Mobile 
    </span>
    <span class = "value">
      <a class="tel" href="tel:+41XXXXXX">
        079 XXX XX XX
      </a>
    </span>
  </span>
</span>

You can see that the last class who defines phone and mobile is "tel", and that's my problem, I want to take mobile and phone in a dictionary separetly like this:

def bot_get_data(item_url):
    source_code = requests.get(item_url)
    plain_text = source_code.text
    soup = BeautifulSoup(plain_text, "html.parser")
    name_company = soup.find_all("h1")
    phone_number = soup.find_all("a", {"class": "phone"}, {"class": "tel"})
    #my problem is here : I have to find a way to go in a class who owns another
    mobile_number = soup.find_all("a",{"class": "mobile"}, {"class": "tel"})
    site_name = soup.find_all("a", {"class": "redirect"})
    email_name = soup.find_all("a", href=re.compile('mailto'))

    name_data = []
    phone_data = []
    mobile_data = []
    site_data = []
    mail_data = []

    for item in name_company:
        name_data.append(item.string)
        print(item.string) 

    for num in phone_number:
        phone_data.append(num.string)
        print(num.string) 

    for mob in mobile_number:
        mobile_data.append(mob.string)
        print(mob.string)

    for site in site_name:
        site_data.append(site.string)
        print(site.string)


    for email in email_name:
        mail_data.append(email.string)
        print(email.string)

Is there someone who knows how to do that with beautifulsoup ?

Thanks =)

Upvotes: 0

Views: 826

Answers (1)

Muhammad Tahir
Muhammad Tahir

Reputation: 5184

Use BeautifulSoup.select()

tel = soup.select(".phone .value .tel")[0].text.strip()
mob = soup.select(".mobile .value. .tel")[0].text.strip()

Upvotes: 1

Related Questions