CODEWITHSUNDEEP
pythonhtmlcssseleniumselenium-webdriver
Patrick
Patrick

Reputation: 75

Python Selenium fails to select search bar

I am trying to use selenium to search for courses on https://catalog.swarthmore.edu/ and scrape the results. All the selectors I have attempted to use fail, and when I print them out they return empty arrays. Why did these selectors fail, and what is the correct one? I get the "#keyword" and "span input" from hovering over and clicking on the search field with the chrome SelectorGadget extension, and I obtained the "span.show.clearfix input" and ".show.clearfix input" from examining the HTML with Chrome Devtools.

#import scrapy
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import json

browserOptions = Options()
browserOptions.add_argument('--headless')
driver = webdriver.Chrome('C:/WebDriver/bin/chromedriver.exe', options=browserOptions)

def css(selector):
    return driver.find_elements_by_id(selector)

driver.get("https://catalog.swarthmore.edu/")
print(driver.title)
search = css("span.show.clearfix input")#css("#keyword")#get search field
print(search)
print(css("span input"))
print(css(".show.clearfix input"))
print(css("#keyword"))

search.send_keys("ANCH 028")#enter your search.
search.submit()
search.clear()#clear search field
driver.quit()

Error message: Traceback (most recent call last): File "getDescWJSON.py", line 31, in search.send_keys("ANCH 028")#enter your search. AttributeError: 'list' object has no attribute 'send_keys'

Upvotes: 1

Views: 62

Answers (3)

Praveen
Praveen

Reputation: 106

This worked for me:-

driver.get('https://catalog.swarthmore.edu/')
btn = driver.find_element_by_xpath('''/html/body/table/tbody/tr[3]/td[1]/table/tbody/tr[2]/td[2]/table/tbody/tr[1]/td/form/fieldset/span[3]/input[1]''')
btn.click() #you have to click it first
btn.send_keys("ANCH 028")

Upvotes: 0

Arundeep Chohan
Arundeep Chohan

Reputation: 9969

You could simply do this to send the keys to that element. Wait for the element to come up after page load and send keys to the element.

search=WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//input[@id='keyword']"))).send_keys("ANCH 028")

Import

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC

Upvotes: 1

Mike67
Mike67

Reputation: 11342

The page has multiple elements with id=keyword so the results is a list. To send the keys, select the first element from the list:

Try this code:

search = css('keyword')[0]  #("span.show.clearfix input")#css("#keyword")#get search field

You should probably change the name of the function css since it actually searches by element id.

Upvotes: 2

Related Questions