CODEWITHSUNDEEP
javascriptpythonselenium
Greg Kaleka
Greg Kaleka

Reputation: 1990

Getting the "text" from a script element using Selenium

I'm working on a web scraper, and the site I'm scraping has a script element on the page that looks like this:

<script type="text/javascript">
                        jQuery(window).load(function($) {
                        Morris.Line({
                          element: 'mpr-graph',
                          data: [
                            {'date': '25-04-2017','y':'1.05'},
                            {'date': '25-04-2017','y':'1.50'},
                            ...

What I want:

I want to get to the data property of the object passed to Morris.Line so that I can turn the data into something usable.

I've managed to select the correct element as a Selenium WebElement using the surrounding div's id and the tag name script, but now I'm stuck.

Is there a way to get the text of a script element using Selenium? The text property is empty since it only returns the text shown on the page for a given element.

What I've tried:

Since I was able to get the text in the browser console by grabbing the text property of the element, I tried using execute_script.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script');")

This returns a WebElement, so we're back to square 1, but at least we know it's working so we can move on to:

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').text;")

I thought this might work since it works in the browser console, but Selenium returns nothing.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').innerHTML;")

As above.

Upvotes: 3

Views: 4941

Answers (1)

JeffC
JeffC

Reputation: 25596

You should be able to use an XPath to find the SCRIPT tag based on its contents

script_text = driver.find_element_by_xpath("//script[contains(.,'mpr-graph')]").text

If for some reason that isn't specific enough (more than one SCRIPT tag contains "mpr-graph") then you can adjust it to whatever text is unique among SCRIPT tags.

Upvotes: 5

Related Questions