CODEWITHSUNDEEP
pythonseleniumwebdriverbeautifulsoup
Mwspencer
Mwspencer

Reputation: 1183

Scrape a class within a class

I want to scrape class_="href" with in the class_="_e4d". Basically looking to scrape a class within a class using BeautifulSoup. 

from bs4 import BeautifulSoup
import selenium.webdriver as webdriver

url = ("https://www.google.com/search?...")

def get_related_search(url):
    driver = webdriver.Chrome("C:\\Users\\John\\bin\\chromedriver.exe")
    driver.get(url)
    soup = BeautifulSoup(driver.page_source)
    relate_result = soup.find_all("p", class_="_e4b")
    return relate_result[0]

relate_url = get_related_search(url)
print(relate_url)

Results: markup_type=markup_type)) p class="_e4b"}{a href="/search?...a}{/p}

I now want to scrape the href result. I am not sure what the next step would be. Thanks for the help.

Note: I replaced <> with {} since it was not showing up as html script

Upvotes: 3

Views: 2228

Answers (1)

alecxe
alecxe

Reputation: 473753

You can actually find this inner a element in one go with a CSS selector:

links = soup.select("p._e4b a[href]")
for link in links:
    print(link['href'])

p._e4b a[href] would locate all a elements having the href attribute inside the p elements having _e4b class.

Upvotes: 5

Related Questions