Getting the old html after an click event in selenium handler java

Question

I have created a custom handler in interactive selenium as below

public class SomeHandler implements InteractiveSeleniumHandler {
  private static final Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
  public String processDriver(WebDriver driver) {
    String accumulatedData = "";
    try {
      Configuration conf = NutchConfiguration.create();
      new WebDriverWait(driver, conf.getLong("libselenium.page.load.delay", 3));
      WebElement more = driver.findElement(By.className("ulBlueLinks"));
      more.click();
      LOG.error("before collecting data:");
      JavascriptExecutor jsx = (JavascriptExecutor) driver;
      jsx.executeScript("document.body.innerHTML=document.body.innerHTML;");
  accumulatedData =  driver.findElement(By.tagName("body")).getAttribute("innerHTML");      
    } 
    catch (Exception e) {
      LOG.error(StringUtils.stringifyException(e));
    }
    return accumulatedData;
  }
  public boolean shouldProcessURL(String URL) {
  return true;
  }
}

The issue is some times I get the whole data (the data including after click event) and some time it is not getting the dynamic data.

Note: I am able to see the click event in the browser. I am using the below:

1. Firefox 61
2. Selenium 3.13
3. Apache Nutch 1.14

Answer 1

After click wait for sometime to get the page loaded.

you could wait for document.readyState = complete which will wait till the page gets loaded.

((JavascriptExecutor) driver).executeScript("return document.readyState").equals("complete");