CODEWITHSUNDEEP
pythonseleniumweb-scraping
Kalakutas
Kalakutas

Reputation: 134

With selenium pick up data

i have problem with pick up data with selenium webdriver. Im getting error code and text (title)

DevTools listening on ws://127.0.0.1:53770/devtools/browser/99e38459-8949-448a-9d5c-bea5b9d8c93c
[8136:23432:1119/174813.994:ERROR:edge_auth_errors.cc(376)] EDGE_IDENTITY: Get Default OS Account failed: Error: Primary Error: kTokenRequestFailed, Secondary Error: kTokenFetchUserInteractionRequired, Platform error: -2138701818, hex:80860006, Error string: Error code: 0x80860006, error message:Error

Stromvergleich von VERIVOX – Tarif stressfrei wechseln und sparen.
PS C:\python\web craper verevox>

My code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time 
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

PATH =".\msedgedriver.exe"
driver = webdriver.Edge(PATH)

driver.get("https://www.verivox.de/stromvergleich/vergleich/#/?plz=10555&persons=on&usage=3500&bonus=OnlyCompliant&profile=H0&product=electricity&source=1&q=WzYsMCwxLDEsMSwxLDEsMiwyMCwwLDEsNzQxMTIyLCI3MTA4NSIsMSwyNDAsMjQwLDM1MDAsMCwwLDAsOTk5LC0xLC0xLC0xLDAsMCwiVG90YWxDb3N0cyIsIkFzY2VuZGluZyIsIk5vbmUiLDM4LCJBbm51YWxDb3N0VmlldyIsMF0%3D&partnerid=1")
print(driver.title)

try:
    main = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.CLASS_NAME, "nrg-main-container"))
    )
    
    articles = main.find_elements_by_tag_name("li")
    for article in articles:
        header = article.find_element_by_class_name("result-item")
        print(header.text)



finally:
    driver.quit()

I want to pick up data from top 20 header info who set on class name result item. Picture bellow enter image description here

Upvotes: 0

Views: 806

Answers (1)

KunduK
KunduK

Reputation: 33384

Use WebDriverWait() and wait for visibility_of_all_elements_located() and following css selector.

allheader=WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,"li[class='result-item'] .result-name-area>.result-name")))
for header in allheader:
     print("Header: " + header.text)

Console output:

Header: Easy24 Strom
Header: Vollstrom sicher
Header: Natur24 Strom
Header: eprimoStrom 24
Header: E.ON Strom Extra
Header: Strom Zuhause+(24
Header: Super Schlau Fuxx
Header: Q.ENERGY Eco24 Plus
Header: MeinSmartTarif Strom
Header: Doppel Plus
Header: innogy direkt Strom
Header: ÖkoStrom Relax 24
Header: E.ON Strom 24
Header: Easy12 Extra Strom
Header: innogy direktStrom12
Header: Spar Smart Sofort 12
Header: E.ON Strom Extra
Header: eprimo PrimaKlima
Header: Lidl-Strom Extra
Header: R(H)EINPOWER Strom

Upvotes: 1

Related Questions