CODEWITHSUNDEEP
ajaxunit-testingselenium
Zombies
Zombies

Reputation: 25912

How to get selenium to wait for ajax response?

How can I get selenium to wait for something like a calendar widget to load? Right now I am just doing a Thread.sleep(2500) after exporting the testcase to a junit program.

Upvotes: 70

Views: 118376

Answers (18)

tinyhare
tinyhare

Reputation: 2401

Use driver.get_log('performance'), you can get message about 'Network.requestWillBeSent', 'Network.responseReceived', 'Network.loadingFinished'.

and all of them has a 'requestId' that stand for a request.

If you have the URL of the ajax , you can get requestId from 'Network.requestWillBeSent' or 'Network.responseReceived', and wait the requestId appear in the 'Network.loadingFinished' message.

import json
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait


options = webdriver.ChromeOptions()
options.set_capability("goog:loggingPrefs", {"performance": "ALL"})
driver = webdriver.Chrome(options=options)

def wait_ajax_finish(driver, url, start_time=0, timeout=50):
    request_id = ''

    def check_finish(driver):
        nonlocal request_id
        for log in driver.get_log('performance'):
            if start_time * 1000 > log['timestamp'] or 'message' not in log:
                continue
            log_entry = json.loads(log['message'])
            try:
                method = log_entry['message']['method']
                params = log_entry['message']['params']
                # get requestId 
                if not request_id and \
                        method in ['Network.requestWillBeSent', 'Network.responseReceived'] and \
                        params['request']['method'] in ['POST', 'GET']:
                    if method == 'Network.requestWillBeSent' and  url in params['request']['url']:
                        requestId = params['requestId']
                    if method == 'Network.responseReceived' and url in params['response']['url']:
                        requestId = params['requestId']
                # check requestId 
                if request_id and method in ['Network.loadingFinished'] and request_id == params['requestId']:
                    request_id = ''
                    return params['requestId']
            except Exception as e:
                pass
    
    return WebDriverWait(driver, timeout).until(check_finish)

# the url may accessed by default
driver.get('https://xxx.yyy.com/home')

button = driver.find_element(......)
start_time = time.time() # We should use logs after this time
button.click()

request_id = wait_ajax_finish(driver, 'https://xxx.yyy.com/get/xxxx', start_time, 120)
time.sleep(1)
# I advise sleep 1 or more seconds  before do other things.
# because the data need some time to render to the web page by js.

# you can get the result by the request_id
print(driver.execute_cdp_cmd("Network.getResponseBody", {"requestId": request_id}))

Upvotes: 0

erdemkiiliic
erdemkiiliic

Reputation: 11

It is possible to capture this by writing a custom HTTP listener in JavaScript as follows. For those who wish to examine this script, you can check out this link: https://gist.github.com/erdemkiiliic/3e3caa666ca1c950c4db3e94a8d3f049

public static void waitForAjaxToFinish() {
  String customScriptForHTTPListener = "if(void 0===window.jsXMLHTTPListener){let i={isCallFinishedStatus:1,pointerSize:0,pointerFinishSize:0,isCallFinished:function(){return 1===this.isCallFinishedStatus},setCallFinished:function(i=0){this.isCallFinishedStatus=i},bindEvents:function(i,e){var t=e.XMLHttpRequest.prototype.send;e.XMLHttpRequest.prototype.send=function(){i.setCallFinished(0);var s=this;i.pointerSize++;var n=e.setInterval(function(){4==s.readyState&&(i.pointerFinishSize++,i.pointerSize===i.pointerFinishSize?i.setCallFinished(1):i.setCallFinished(0),clearInterval(n))},1);return t.apply(this,[].slice.call(arguments))}},init:function(){this.bindEvents(this,window)}};window.jsXMLHTTPListener=i,window.jsXMLHTTPListener.init();let e=window.jsXMLHTTPListener.isCallFinished();console.log(e);return e;}else{let t=window.jsXMLHTTPListener.isCallFinished();console.log(t);return t;}";

  WebDriverWait wait = new WebDriverWait(driver, 10);

  wait.until(new ExpectedCondition < Boolean > () {
    public Boolean apply(WebDriver driver) {
      return ((JavascriptExecutor) driver).executeScript(
          "return document.readyState").equals("complete") &&
        ((JavascriptExecutor) driver).executeScript(
          customScriptForHTTPListener).equals(true);
    }
  });
}

This script returns either true or false. If there is no active XHR request at that moment, it returns true.

Upvotes: 0

erdemkiiliic
erdemkiiliic

Reputation: 11

The solution below can be used for cases where jQuery is not available.

By actively monitoring the logs, we can ensure that all files and responses have been completed with certainty.

I have tested this solution for Chrome, but you should also ensure it is supported in other browsers. For example, Safari does not support this method.

Also, don't forget to set the capability for these logs to be visible.

 LoggingPreferences preferences = new LoggingPreferences();
    preferences.enable(LogType.PERFORMANCE, Level.ALL);
    ChromeOptions chromeOptions = new ChromeOptions();
    chromeOptions.setCapability("goog:loggingPrefs", preferences);

You should use with WebDriverWait instance. After that, just call the waitUntilPageIsReady() method.

public class IsPageReady implements ExpectedCondition<Boolean> {
private Logger log = LogManager.getLogger(IsPageReady.class);

@NullableDecl
@Override
public Boolean apply(@NullableDecl WebDriver webDriver) {
    try {
        if (webDriver == null)
            return false;
        LogEntries logs = WebDriverManager.getInstance().getWebDriver().manage().logs().get(LogType.PERFORMANCE);
        return ready = logs.getAll().isEmpty();
    } catch (Exception ex) {
        log.warn("There was a problem checking if the page was ready!");
    }
    return false;
}
}

just call:

public void waitUntilPageIsReady() {
    getWebDriverWait().until(new IsPageReady());
}

Upvotes: 0

Roberto Petrilli
Roberto Petrilli

Reputation: 865

In my case the issue seemed due to ajax delays but was related to internal iframes inside the main page. In seleminum it is possible to switch to internal frames with:

    driver.switchTo().frame("body");
    driver.switchTo().frame("bodytab");

I use java. After that I was able to locate the element

    driver.findElement(By.id("e_46")).click();

Upvotes: 0

toutpt
toutpt

Reputation: 5220

With webdriver aka selenium2 you can use implicit wait configuration as mentionned on https://www.selenium.dev/documentation/en/webdriver/waits/#implicit-wait

Using Java:

WebDriver driver = new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("http://somedomain/url_that_delays_loading");
WebElement myDynamicElement = driver.findElement(By.id("myDynamicElement"));

Or using python:

from selenium import webdriver

ff = webdriver.Firefox()
ff.implicitly_wait(10) # seconds
ff.get("http://somedomain/url_that_delays_loading")
myDynamicElement = ff.find_element_by_id("myDynamicElement")

Upvotes: 4

BatBold
BatBold

Reputation: 19

Below is my code for fetch. Took me while researching because jQuery.active doesn't work with fetch. Here is the answer helped me proxy fetch, but its only for ajax not fetch mock for selenium

public static void customPatchXMLHttpRequest(WebDriver driver) {
    try {
        if (driver instanceof JavascriptExecutor) {
            JavascriptExecutor jsDriver = (JavascriptExecutor) driver;
            Object numberOfAjaxConnections = jsDriver.executeScript("return window.openHTTPs");
            if (numberOfAjaxConnections instanceof Long) {
                return;
            }
            String script = "  (function() {" + "var oldFetch = fetch;"
                    + "window.openHTTPs = 0; console.log('starting xhttps');" + "fetch = function(input,init ){ "
                    + "window.openHTTPs++; "

                    + "return oldFetch(input,init).then( function (response) {"
                    + "      if (response.status >= 200 && response.status < 300) {"
                    + "          window.openHTTPs--;  console.log('Call completed. Remaining active calls: '+ window.openHTTPs); return response;"
                    + "      } else {"
                    + "          window.openHTTPs--; console.log('Call fails. Remaining active calls: ' + window.openHTTPs);  return response;"
                    + "      };})" + "};" + "var oldOpen = XMLHttpRequest.prototype.open;"
                    + "XMLHttpRequest.prototype.open = function(method, url, async, user, pass) {"
                    + "window.openHTTPs++; console.log('xml ajax called');"
                    + "this.addEventListener('readystatechange', function() {" + "if(this.readyState == 4) {"
                    + "window.openHTTPs--; console.log('xml ajax complete');" + "}" + "}, false);"
                    + "oldOpen.call(this, method, url, async, user, pass);" + "}" +

                    "})();";
            jsDriver.executeScript(script);
        } else {
            System.out.println("Web driver: " + driver + " cannot execute javascript");
        }
    } catch (Exception e) {
        System.out.println(e);
    }
}

Upvotes: -1

neonidian
neonidian

Reputation: 1263

This works like a charm for me :

public void waitForAjax() {

    try {
        WebDriverWait driverWait = new WebDriverWait(driver, 10);

        ExpectedCondition<Boolean> expectation;   
        expectation = new ExpectedCondition<Boolean>() {

            public Boolean apply(WebDriver driverjs) {

                JavascriptExecutor js = (JavascriptExecutor) driverjs;
                return js.executeScript("return((window.jQuery != null) && (jQuery.active === 0))").equals("true");
            }
        };
        driverWait.until(expectation);
    }       
    catch (TimeoutException exTimeout) {

       // fail code
    }
    catch (WebDriverException exWebDriverException) {

       // fail code
    }
    return this;
}

Upvotes: 2

Artem Kislicyn
Artem Kislicyn

Reputation: 51

This work for me 

public  void waitForAjax(WebDriver driver) {
    new WebDriverWait(driver, 180).until(new ExpectedCondition<Boolean>(){
        public Boolean apply(WebDriver driver) {
            JavascriptExecutor js = (JavascriptExecutor) driver;
            return (Boolean) js.executeScript("return jQuery.active == 0");
        }
    });
}

Upvotes: 5

Bill Hileman
Bill Hileman

Reputation: 2836

If the control you are waiting for is an "Ajax" web element, the following code will wait for it, or any other Ajax web element to finish loading or performing whatever it needs to do so that you can more-safely continue with your steps.

    public static void waitForAjaxToFinish() {

    WebDriverWait wait = new WebDriverWait(driver, 10);

    wait.until(new ExpectedCondition<Boolean>() { 
        public Boolean apply(WebDriver wdriver) { 
            return ((JavascriptExecutor) driver).executeScript(
                    "return jQuery.active == 0").equals(true);
        }
    }); 

}

Upvotes: 0

llatinov
llatinov

Reputation: 104

As mentioned above you can wait for active connections to get closed:

private static void WaitForReady() {
    WebDriverWait wait = new WebDriverWait(webDriver, waitForElement);
    wait.Until(driver => (bool)((IJavaScriptExecutor)driver).ExecuteScript("return jQuery.active == 0"));
}

My observation is this is not reliable as data transfer happens very quickly. Much more time is consumed on data processing and rendering on the page and even jQuery.active == 0 data might not be yet on the page.

Much wiser is to use an explicit wait for element to be shown on the page, see some of the answers related to this.

The best situation is if your web application have some custom loader or indication that data is being processed. In this case you can just wait for this indication to hide.

Upvotes: 1

Hrabosch
Hrabosch

Reputation: 1583

I wrote next method as my solution (I hadn't any load indicator):

public static void waitForAjax(WebDriver driver, String action) {
       driver.manage().timeouts().setScriptTimeout(5, TimeUnit.SECONDS);
       ((JavascriptExecutor) driver).executeAsyncScript(
               "var callback = arguments[arguments.length - 1];" +
                       "var xhr = new XMLHttpRequest();" +
                       "xhr.open('POST', '/" + action + "', true);" +
                       "xhr.onreadystatechange = function() {" +
                       "  if (xhr.readyState == 4) {" +
                       "    callback(xhr.responseText);" +
                       "  }" +
                       "};" +
                       "xhr.send();");
}

Then I jsut called this method with actual driver. More description in this post.

Upvotes: 2

Cirem
Cirem

Reputation: 870

Here's a groovy version based on Morten Christiansen's answer.

void waitForAjaxCallsToComplete() {
    repeatUntil(
            { return getJavaScriptFunction(driver, "return (window.jQuery || {active : false}).active") },
            "Ajax calls did not complete before timeout."
    )
}

static void repeatUntil(Closure runUntilTrue, String errorMessage, int pollFrequencyMS = 250, int timeOutSeconds = 10) {
    def today = new Date()
    def end = today.time + timeOutSeconds
    def complete = false;

    while (today.time < end) {
        if (runUntilTrue()) {
            complete = true;
            break;
        }

        sleep(pollFrequencyMS);
    }
    if (!complete)
        throw new TimeoutException(errorMessage);
}

static String getJavaScriptFunction(WebDriver driver, String jsFunction) {
    def jsDriver = driver as JavascriptExecutor
    jsDriver.executeScript(jsFunction)
}

Upvotes: 1

George Kargakis
George Kargakis

Reputation: 5556

The code (C#) bellow ensures that the target element is displayed:

        internal static bool ElementIsDisplayed()
        {
          IWebDriver driver = new ChromeDriver();
          driver.Url = "http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp";
          WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
          By locator = By.CssSelector("input[value='csharp']:first-child");
          IWebElement myDynamicElement = wait.Until<IWebElement>((d) =>
          {
            return d.FindElement(locator);
          });
          return myDynamicElement.Displayed;
        }

If the page supports jQuery it can be used the jQuery.active function to ensure that the target element is retrieved after all the ajax calls are finished:

 public static bool ElementIsDisplayed()
    {
        IWebDriver driver = new ChromeDriver();
        driver.Url = "http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp";
        WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
        By locator = By.CssSelector("input[value='csharp']:first-child");
        return wait.Until(d => ElementIsDisplayed(d, locator));
    }

    public static bool ElementIsDisplayed(IWebDriver driver, By by)
    {
        try
        {
            if (driver.FindElement(by).Displayed)
            {
                //jQuery is supported.
                if ((bool)((IJavaScriptExecutor)driver).ExecuteScript("return window.$ != undefined"))
                {
                    return (bool)((IJavaScriptExecutor)driver).ExecuteScript("return $.active == 0");
                }
                else
                {
                    return true;
                }
            }
            else
            {
                return false;
            }
        }
        catch (Exception)
        {
            return false;
        }
    }

Upvotes: 2

Nimo
Nimo

Reputation: 8304

If using python, you may use this function, which clicks the button and waits for the DOM change:

def click_n_wait(driver, button, timeout=5):
    source = driver.page_source
    button.click()
    def compare_source(driver):
        try:
            return source != driver.page_source
        except WebDriverException:
            pass
    WebDriverWait(driver, timeout).until(compare_source)

(CREDIT: based on this stack overflow answer)

Upvotes: 16

Amir Shenouda
Amir Shenouda

Reputation: 2185

I had a similar situation, i wanted to wait for ajax requests so that the loading panel would have disappeared, I have inspected the html before and after the requests, found that there is a div for the ajax loading panel, the dix is displayed during the ajax request, and hidden after the request ends. I have created a function to wait for the panel to be displayed, then wait for it to be hidden 



public void WaitForModalPanel()
{
    string element_xpath = ".//*[@id='ajaxLoadingModalPanelContainer'  and not(contains(@style,'display: none'))]";
    WebDriverWait wait = new WebDriverWait(driver, new TimeSpan(0, 2, 0));
    wait.Until(ExpectedConditions.ElementIsVisible(By.XPath(element_xpath)));
    element_xpath = ".//*[@id='ajaxLoadingModalPanelContainer' and contains(@style,'DISPLAY: none')]";
    wait.Until(ExpectedConditions.ElementExists(By.XPath(element_xpath)));
}

Check this for more details

Upvotes: 1

Samuel Tian
Samuel Tian

Reputation: 787

For those who is using primefaces, just do:

selenium.waitForCondition("selenium.browserbot.getCurrentWindow().$.active==0", defaultWaitingPeriod);

Upvotes: -2

Morten Christiansen
Morten Christiansen

Reputation: 19580

A more general solution than waiting for an element would be to wait for all the connections to the server to close. This will allow you to wait for all ajax calls to finish, even if they don't have any callback and thus don't affect the page. More details can be found here.

Using C# and jQuery, I have created the following method to wait for all AJax calls to complete (if anyone have more direct ways of accessing JS variables from C#, please comment):

internal void WaitForAjax(int timeOut = 15)
{
    var value = "";
    RepeatUntil(
        () => value = GetJavascriptValue("jQuery.active"), 
        () => value == "0", 
        "Ajax calls did not complete before timeout"
    );
}

internal void RepeatUntil(Action repeat, Func<bool> until, string errorMessage, int timeout = 15)
{
    var end = DateTime.Now + TimeSpan.FromSeconds(timeout);
    var complete = false;

    while (DateTime.Now < end)
    {
        repeat();
        try
        {
            if (until())
            {
                complete = true;
                break;
            }
        }
        catch (Exception)
        { }
        Thread.Sleep(500);
    }
    if (!complete)
        throw new TimeoutException(errorMessage);
}

internal string GetJavascriptValue(string variableName)
{
    var id = Guid.NewGuid().ToString();
    _selenium.RunScript(String.Format(@"window.$('body').append(""<input type='text' value='""+{0}+""' id='{1}'/>"");", variableName, id));
    return _selenium.GetValue(id);
}

Upvotes: 28

Neil Aitken
Neil Aitken

Reputation: 7854

I would use

waitForElementPresent(locator)

This will wait until the element is present in the DOM.

If you need to check the element is visible, you may be better using

waitForElementHeight(locator)

Upvotes: 46

Related Questions