CODEWITHSUNDEEP
pythonselenium-webdriverweb-scraping
S. Ard
S. Ard

Reputation: 5

Selenium in Python: Do not know how to click on next page if link is not present

Hi I'm new to python and have been giving a shot at web scraping. This is what my code looks like:

print('What is the term to be searched?') 

term = input()

browser = webdriver.Firefox(executable_path ='/usr/local/bin/geckodriver')
browser.get('https://google.com/search?q=' + term)

try:    
   link = browser.find_element_by_link_text("Hello")

# (I want to make it so the first page has no links of interest, so I can proceed to clicking the next page)    
except NoSuchElementException:    
   nextPage = browser.find_element_by_css_selector('Css of next page button)    
   nextPage.click()

So the code above has giving me errors so I tried just taking out the 'link=browser.find_element' and searching the term and proceeding to the next page by Css_selector, but I got Traceback error.

Any help would be appreciated. Thanks

Here is the code message:

Traceback (most recent call last):
  File "/Users/shaun/Documents/Python/mwpractice.py", line 17, in <module>
    nextPage = browser.find_element_by_css_selector('''<span class="csb ch" style="background:url(/images/nav_logo242_hr.png) no-repeat;background-position:-74px 0;background-size:167px;width:20px"></span>''')
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 498, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 832, in find_element
    'value': value})['value']
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: Given css selector expression "<span class="csb ch" style="background:url(/images/nav_logo242_hr.png) no-repeat;background-position:-74px 0;background-size:167px;width:20px"></span>" is invalid: InvalidSelectorError: '<span class="csb ch" style="background:url(/images/nav_logo242_hr.png) no-repeat;background-position:-74px 0;background-size:167px;width:20px"></span>' is not a valid selector: "<span class="csb ch" style="background:url(/images/nav_logo242_hr.png) no-repeat;background-position:-74px 0;background-size:167px;width:20px"></span>"

Upvotes: 0

Views: 378

Answers (1)

Mateusz Ostaszewski
Mateusz Ostaszewski

Reputation: 329

You have to read error messages more carefully.

selenium.common.exceptions.InvalidSelectorException: Message: Given css selector expression ..."

It is a very clear message - the CSS selector provided by you is wrong.

If you want to click next page button, you can use xpath:

browser.find_element_by_xpath('//*[@id="pnnext"]/span[2]').click()

Correct CSS selector for next page button is:

#pnnext > span:nth-child(2)

Upvotes: 1

Related Questions