CODEWITHSUNDEEP
pythonhtmlseleniumweb-scrapingautomation
João Weckerle
João Weckerle

Reputation: 73

Selenium Python code to Loop through items select box

I'm trying to loop through some pages and get the data from each page, but I keep getting the same error after the second page.

Here is a snippet of the code:

driver = webdriver.Chrome('C:\ProgramData\Anaconda3\chromedriver.exe')
driver.get('https://cvmweb.cvm.gov.br/SWB/Sistemas/SCW/CPublica/InfDiario/CPublicaInfDiario.aspx?PK_PARTIC=121145&SemFrame=')
select = Select(driver.find_element(By.XPATH,'/html/body/form/table[1]/tbody/tr[1]/td/select'))



options = select.options
for index in range(0, len(options)):
    print(index)
    time.sleep(10)
    select.select_by_index(index)
    time.sleep(5)

And I get the following error:

`

StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=107.0.5304.88)
Stacktrace:
Backtrace:
    Ordinal0 [0x009AACD3+2075859]
    Ordinal0 [0x0093EE61+1633889]
    Ordinal0 [0x0083B7BD+571325]
    Ordinal0 [0x0083E374+582516]
    Ordinal0 [0x0083E225+582181]
    Ordinal0 [0x0083E4C0+582848]
    Ordinal0 [0x0086A9A2+764322]
    Ordinal0 [0x0086AE1B+765467]
    Ordinal0 [0x00861A81+727681]
    Ordinal0 [0x00887364+881508]
    Ordinal0 [0x008615BF+726463]
    Ordinal0 [0x00887534+881972]
    Ordinal0 [0x0089B56A+963946]
    Ordinal0 [0x00887136+880950]
    Ordinal0 [0x0085FEFD+720637]
    Ordinal0 [0x00860F3F+724799]
    GetHandleVerifier [0x00C5EED2+2769538]
    GetHandleVerifier [0x00C50D95+2711877]
    GetHandleVerifier [0x00A3A03A+521194]
    GetHandleVerifier [0x00A38DA0+516432]
    Ordinal0 [0x0094682C+1665068]
    Ordinal0 [0x0094B128+1683752]
    Ordinal0 [0x0094B215+1683989]
    Ordinal0 [0x00956484+1729668]
    BaseThreadInitThunk [0x76DCFA29+25]
    RtlGetAppContainerNamedObjectPath [0x770B7BBE+286]
    RtlGetAppContainerNamedObjectPath [0x770B7B8E+238]

`

Upvotes: 1

Views: 194

Answers (1)

Eugeny Okulik
Eugeny Okulik

Reputation: 1424

I suppose the page is reloaded after selecting an element from the select box. And every time the page is reloaded, Selenium looses connection to the elements that were found. So you need to find elements again after selecting an option.
If select box doesn't change its position on the page, it should work:

driver = webdriver.Chrome('C:\ProgramData\Anaconda3\chromedriver.exe')
driver.get('https://cvmweb.cvm.gov.br/SWB/Sistemas/SCW/CPublica/InfDiario/CPublicaInfDiario.aspx?PK_PARTIC=121145&SemFrame=')
select = Select(driver.find_element(By.XPATH,'/html/body/form/table[1]/tbody/tr[1]/td/select'))



options = select.options
for index in range(0, len(options)):
    print(index)
    time.sleep(10)
    select.select_by_index(index)
    time.sleep(5)
    select = Select(driver.find_element(By.XPATH, '/html/body/form/table[1]/tbody/tr[1]/td/select'))```

Upvotes: 1

Related Questions