CODEWITHSUNDEEP
pythonseleniumweb-scrapingbeautifulsoup
noobCoder
noobCoder

Reputation: 94

How to scrape the search suggestion using selenium

I'm trying to scrape the search suggestion from the Crunchbase. I find the search box and send the words to the website but I want to scrape the search suggestion result without send ENTER key but When i type manually I'm getting the search suggestion but through selenium I'm not getting search suggestions.

Here is What I've done so far

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
d = webdriver.Chrome('chromedriver')
books = ['netflix']
for book in books:
  d.get('https://www.crunchbase.com/home')
  e = d.find_element_by_id('mat-input-0')
  e.send_keys(book)
  sleep(5)

You can see here there is no search suggestions when using Selenium enter image description here

You can see the search suggestion when manually type enter image description here

Upvotes: 1

Views: 247

Answers (1)

jizhihaoSAMA
jizhihaoSAMA

Reputation: 12672

Because this element need to be clicked to show this page,like the code below:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
d = webdriver.Chrome('chromedriver')
books = ['netflix']
for book in books:
  d.get('https://www.crunchbase.com/home')
  e = d.find_element_by_id('mat-input-0')
  e.click()
  e.send_keys(book)
  sleep(5)

enter image description here

Upvotes: 2

Related Questions