Reputation: 31
I am new to Selenium and I am trying to scrape all the book list info stored in book_list, which will then be showed in pandas dataframe.
Here is the code that's the script I am using :
from selenium import webdriver
import pandas as pd
import time
url = "https://book.naver.com/"
query = 'python'
driver = webdriver.Chrome()
driver.get(url)
driver.find_element_by_xpath('//*[@id="nx_query"]').send_keys(query)
time.sleep(4.5)
driver.find_element_by_xpath('//*[@id="search_btn"]').click()
time.sleep(4.5)
driver.find_element_by_xpath('//*[@id="content"]/div[2]/ul/li[2]/a').click()
time.sleep(4.5)
books = driver.find_elements_by_xpath('//*[@id="searchBiblioList"]/li[1]/dl')
book_list = []
for book in books:
title = book.driver.find_element_by_xpath('.//*[@id="searchBiblioList"]/li[1]/dl/dt/a').text
desp = book.driver.find_element_by_xpath('.//*[@id="searchDescrition_17854273"]').text
book_item = {
'title' : title,
'desp' : desp
}
book_list.append(book_item)
df = pd.DataFrame(book_list)
print(df)
This throws an error :
AttributeError Traceback (most recent call last)
<ipython-input-6-f96a3c3baaf4> in <module>()
4
5 for book in books:
----> 6 title = book.driver.find_element_by_xpath('.//*[@id="searchBiblioList"]/li[1]/dl/dt/a').text
7 desp = book.driver.find_element_by_xpath('.//*[@id="searchDescrition_17854273"]').text
8 book_item = {
AttributeError: 'WebElement' object has no attribute 'driver'
Any help would be appreciated.
Upvotes: 0
Views: 650
Reputation: 19
easy.
this is your books
:
books = driver.find_elements_by_xpath('//*[@id="searchBiblioList"]/li[1]/dl')`
it has been a driver object generator.
then,you use for loop to get each book
, it has been the seleium object, so you don't need to use book.driver
, just use book.find_element_by_xpath
, just like below:
for book in books:
title = book.find_element_by_xpath('.//*[@id="searchBiblioList"]/li[1]/dl/dt/a').text
desp = book.find_element_by_xpath('.//*[@id="searchDescrition_17854273"]').text
book_item = {
'title' : title,
'desp' : desp
}
Hope it's helpful, Goood Luck.
Upvotes: 1