CODEWITHSUNDEEP
pythonhtmlseleniumweb-scraping
Coercer_
Coercer_

Reputation: 39

How can I select a search suggestion using selenium? The site prevents me from just clicking on submit, requires a selection

I'm trying to make searching for temporary apartments a bit easier on myself, but a website with listings for these apartments requires me to select a suggestion from their drop down list before I can click on submit. No matter how complete the entry in the search box might be.

The ultimate hope here is that I can get forward to the search results and then extract contact information from each listing. I was able to extract the data I need from a listing using Beautiful soup and Requests, but I had to paste in the URL for that specific listing into my code. I didn't get that far. If anyone has a suggestion on how to perhaps circumvent the landing page to get to the relevant listings, please let me know.

I tried just splicing the town name and the state name into the address bar by looking at how it's written after a successful search but that didn't work.

The site is Mein Monteurzimmer.

Here is my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.select import Select


driver = webdriver.Firefox()


webpage = r"https://mein-monteurzimmer.de"
print('Prosim vnesi zeljeno mesto') #Please enter the town to search
searchterm = input()

driver.get(webpage)

sbox = driver.find_element_by_xpath("/html/body/main/cpagearea/section/div[2]/div/section[1]/div/div[1]/section/form/div/input")
sbox.send_keys(searchterm)

ddown = driver.find_element_by_xpath("/html/body/main/cpagearea/section/div[2]/div/section[1]/div/div[1]/section/form/div")
ddown.select_by_value(1)

webdriver.wait(2)

#select = driver.find_element_by_xpath("/html/body/main/cpagearea/section/div[2]/div/section[1]/div/div[1]/section/form/div")

submit = driver.find_element_by_xpath("/html/body/main/cpagearea/section/div[2]/div/section[1]/div/div[1]/section/form/button")
submit.click

When I inspect the search box I can't find anything related to the suggestions until I enter a text. Then I can't click on the HTML code because that dismisses the suggestions. It's quite frustrating.

Here's a screenshot:

Inspecting suggestions

So I'm blindly trying to select something.

The error here is:

AttributeError: 'FirefoxWebElement' object has no attribute 'select_by_value'

I tried something with select, but that doesn't work with the way I tried this.

I am stumped and the solutions I could find were specific for other sites like Google or Amazon and I couldn't make sense if it.

Does anyone know how I could make this work?

Here's the code for getting information out of a listing, which I'll have to expand on to get the other data:

import bs4, requests

def getMonteurAddress(MonteurUrl):
    res = requests.get(MonteurUrl)
    res.raise_for_status()

    soup = bs4.BeautifulSoup(res.text, 'html.parser')
    elems = soup.select('section.c:nth-child(4) > div:nth-child(2) > div:nth-child(2) > dl:nth-child(1) > dd:nth-child(2)')
    return elems[0].text.strip()




address = getMonteurAddress('https://mein-monteurzimmer.de/105742/monteurzimmer/deggendorf-monteurzimmer-deggendorf-pensionfelix%40googlemailcom')
print('Naslov je ' + address) #print call to see if it gets the right data

Upvotes: 1

Views: 815

Answers (1)

rahul rai
rahul rai

Reputation: 2326

As you can see once you type in, there is a list of divs creating. Now you need to get the a valid locator for these divs. To get the locator for these created divs you need to inspect elements in debug pause mode ( F12--> Source Tab --> F8).

Try below code to select first matching address as you typed.

sbox = driver.find_element_by_xpath("//input[@placeholder='Adresse, PLZ oder Ort eingeben']")
sbox.send_keys(searchterm)
addessXpath = "//div[contains(text(),'"+searchterm+"')]"
driver.find_element_by_xpath(addessXpath).click()

Note : If there are more than one matching address , first one will be selected.

Upvotes: 1

Related Questions