CODEWITHSUNDEEP
python-2.7seleniumweb-scraping
Xonshiz
Xonshiz

Reputation: 1367

Element not found in cache - Selenium (Python)

I just wrote a simple webscraping script to give me all the episode links on a particular site's page. The script was working fine, but, now it's broke. I didn't change anything.

Try this URL (For scraping ) :- http://www.crunchyroll.com/tabi-machi-late-show

Now, the script works mid-way and gives me an error stating, ' Element not found in the cache - perhaps the page has changed since it was looked up'

I looked it up on internet and people said about using the 'implicit wait' command at certain places. I did that, still no luck.

UPDATE : I tried this script in a demote desktop and it's working there without any problems.

Here's my script :-

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import os
import time
from subprocess import Popen

#------------------------------------------------

try:
    Link = raw_input("Please enter your Link : ")
    if not Link:
        raise ValueError('Please Enter A Link To The Anime Page. This Application Will now Exit in 5 Seconds.')
except ValueError as e:
    print(e)
    time.sleep(5)
    exit()

print 'Analyzing the Page. Hold on a minute.'
driver = webdriver.Firefox()
driver.get(Link)

assert "Crunchyroll" in driver.title
driver.implicitly_wait(5) # <-- I tried removing this lines as well. No luck.
elem = driver.find_elements_by_xpath("//*[@href]")
driver.implicitly_wait(10) # <-- I tried removing this lines as well. No luck.
text_file = open("BatchLink.txt", "w")
print 'Fetching The Links, please wait.'
for elem in elem:
    x = elem.get_attribute("href")
    #print x
    text_file.write(x+'\n')         


print 'Links have been fetched. Just doing the final cleaning now.'
text_file.close()

CleanFile = open("queue.txt", "w")
with open('BatchLink.txt') as f:
    mylist = f.read().splitlines()
    #print mylist
    with open('BatchLink.txt', 'r') as inF:
     for line in inF:
        if 'episode' in line:
            CleanFile.write(line)

print 'Please Check the file named queue.txt'
CleanFile.close()
os.remove('BatchLink.txt')
driver.close()

Here's a screenshot of the error (might be of some help) : http://i.imgur.com/SaANlsg.png

Upvotes: 0

Views: 904

Answers (1)

Leon Barkan
Leon Barkan

Reputation: 2703

Ok i didn't work with python but know the problem

you have variable that you init -> elem = driver.find_elements_by_xpath("//*[@href]")

after that you doing some things with it in loop before you finishing the loop try to init this variable again 

elem = driver.find_elements_by_xpath("//*[@href]")

The thing is that the DOM is changes and you loosing the element collection.

Upvotes: 2

Related Questions