CODEWITHSUNDEEP
python-3.xweb-scrapingselenium-chromedriver
Jcbyte
Jcbyte

Reputation: 237

How to get Chrome webdriver to scrape salary value and not "Glassdoor est"

The output of the salary_estimate variable is '$39K-$72K (Glassdoor est.)' The goal is to retrieve salary value such as '$39K-$72K' and not including '(Glassdoor est.)'

from selenium import webdriver
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(
    executable_path="C:/webdrivers/chromedriver", options=options)
url = 'https://www.glassdoor.com/Job/data-scientist-jobs-SRCH_KO0,14_IP2.htm'
driver.get(url)
salary_estimate = driver.find_element_by_xpath('.//span[@class="css-1uyte9r css-hca4ks e1wijj242"]').text

enter image description here

enter image description here

Upvotes: 0

Views: 154

Answers (1)

Arthur Pereira
Arthur Pereira

Reputation: 1559

You have to get rid of the child nodes you don't want. And you can do that executing a javascript with your driver.

Append this to your code:

salary = driver.execute_script('return arguments[0].firstChild.textContent;', salary_estimate).strip()
print(salary)

This will ensure you get only the first element text.

Upvotes: 1

Related Questions