CODEWITHSUNDEEP
javascriptpythonselenium
Lovethenakedgun
Lovethenakedgun

Reputation: 845

Handle erroneous JS when loading page in Selenium

I was scraping a site recently, and ran into this:

<img onerror="onErrorImg(this)" onload="onLoadImg(this);" src="example.png">

However, the site in question does not define a function named onErrorImg, so this produces an error in the console if images are disabled. I'd like to disable images in this case as it's being run over a metered connection. Without changing the rest of my code, if I enable images in the browser, the page loads correctly & can be scraped.

I'm going to contact the site owners about this, but in the event it isn't rectified in a timely manner: Is there a straightforward workaround for cases like this? If I could inject a JS script into the DOM as it's being loaded, I could define a function that is just an empty block, but beyond that I'm not really sure how to approach this.

Edit: I'm not sure what the process is that creates the DOM elements for that page, but if you use requests to fetch the URL, it primarily only returns JS. This makes me assume that the whole of the page is generated via JS / AJAX, so this function error'ing out seems to cause the majority of the page not to load.

Upvotes: 0

Views: 81

Answers (1)

ewwink
ewwink

Reputation: 19154

try with this simple solution, it overwriting the function

driver.get('https://.....')

driver.execute_script('''
    window.onErrorImg = function(){}
    window.onLoadImg = function(){}
''')

if it not working you have to create extension that overwrite the function early.

Upvotes: 0

Related Questions