CODEWITHSUNDEEP
javascriptpythonselenium
Osuynonma
Osuynonma

Reputation: 499

Returning javascript data from Python Selenium Web Driver

I'm trying to run a javascript file inside a Python file using Selenium WebDriver and get the return value from the javascript function. The following attempts below should work from what I've looked up online. I've worked on this for a few hours to no avail. Any help would be appreciated.

Python Code:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
import os


driver = webdriver.Chrome(executable_path='chromedriver')
driver.get('example.com')

# Clean up the text inside the <p> tags
js_file = 'clean_text.js'
with open(js_file, 'r') as f:
    script = f.read()

# Attempt 1
text = driver.execute_script(script)

# Attempt 2
text = driver.execute_script(f'return {script}')

# Attempt 3
text = WebDriverWait(driver, 20).until(lambda driver: driver.execute_script(script))

# Attempt 4
text = WebDriverWait(driver, 20).until(lambda driver: driver.execute_script(f'return {script}'))

The attempts either return None or a selenium.common.exceptions.TimeoutException

Javascript Code:

$(document).ready(function() {
  // Collect the text within the nested <p> tags
  var text = [];

  // get all nested <p> tags
  let paragraphs = $(`div#id div[data-test-id=value] p`);

  // Replace all whitespaces with a single white space
  paragraphs.each(function(index) {
    let original_text = $(this).text();
    let cleaned_text = original_text.trim().replace(/\s\s+/g, ' ');
    $(this).text(cleaned_text);
    text.push(cleaned_text);
  });

  return text;
});

Given furas' answer below this JS code worked for Attempt 1 & 4. I needed to add "return" at the start, "();" at the end & remove the jQuery.

return function cleanText() {
  // Collect the text within the nested <p> tags
  var text = [];

  // get all nested <p> tags
  let paragraphs = $(`div#id div[data-test-id=value] p`);

  // Replace all whitespaces with a single white space
  paragraphs.each(function(index) {
    let original_text = $(this).text();
    let cleaned_text = original_text.trim().replace(/\s\s+/g, ' ');
    $(this).text(cleaned_text);
    text.push(cleaned_text);
  });

  return text;
}();

Upvotes: 0

Views: 1067

Answers (1)

furas
furas

Reputation: 142651

All problem can be because you run function() inside ready() and it return sends text to ready(), not to you. You have to directly run function to get its result.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
import os
import time

driver = webdriver.Chrome()# executable_path='chromedriver')
driver.get('http://quotes.toscrape.com/js/')

#script = '$(document).ready(function() { return "Hello World"; })';
script = 'function test() { return "Hello World"; })()';

time.sleep(3)

# Attempt 1
text = driver.execute_script(script)
print(text)  # None

# Attempt 2
text = driver.execute_script(f'return {script}')
print(text)  # Hello World

# Attempt 3
#text = WebDriverWait(driver, 20).until(lambda driver: driver.execute_script(script))
#print(text)  # error

# Attempt 4
text = WebDriverWait(driver, 20).until(lambda driver: driver.execute_script(f'return {script}'))
print(text)  # Hello World

EDIT:

if you really need to check if document is ready then you should do it inside your function. You could try to use while-loop with variable $.isReady

In JQuery, how do I check if the DOM is ready?

Upvotes: 2

Related Questions