CODEWITHSUNDEEP
javascriptpythonselenium
tonymgarner
tonymgarner

Reputation: 19

Python Selenium Can't see data

OK, so I'm new to programming with Python and HTML/Javascript but I have managed to scrape a website or two so far. However, I have come across this site and its driving me round the bend.

My Code is:

#import libraries
from urllib.request import urlopen as ureq
from bs4 import BeautifulSoup as soup
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By



#set chromepath to include .exe
chrome_path = r"C:\Program Files\Python37\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome(chrome_path)

#set variables
myurl='http://www.ctbar.org/search/newsearch.asp'
driver.get(myurl) # loads the 1st page into the browser

element = driver.find_elements(By.ID, 'SearchResultsGrid')

print (element)
print('Done')

driver.quit()

The only output I get is:

[] Done

What bit(s) am I getting wrong??

Please help before I give up in frustration:-)

Upvotes: 2

Views: 97

Answers (3)

Ivan Bryzzhin
Ivan Bryzzhin

Reputation: 2155

The element you are finding located inside of iframe, you have to switch context to it first:

driver.switch_to.frame(driver.find_element_by_id('SearchResultsFrame'))
# driver.switch_to_frame is deprecated
element = driver.find_elements(By.ID, 'SearchResultsGrid')

Upvotes: 1

Satish Michael
Satish Michael

Reputation: 2015

you are getting a blank list because the driver couldn't find the element match the locator strategy (By.ID, 'SearchResultsGrid').

It's inside a frame, you need to switch to the frame first and then find the element

Upvotes: 1

Luke
Luke

Reputation: 774

The element you are looking for is contained within an iframe, SearchResultsFrame.

Try this:

driver.switch_to_frame('SearchResultsFrame')
element = driver.find_elements(By.ID, 'SearchResultsGrid')

Upvotes: 1

Related Questions