Iterating through website table (Element is not attached to the page document)

Question

I have a code that iterates through website table. Table is dynamic and items are loaded into it on user scroll.

I have parameters:

public class UserTableRow
{
    private readonly IWebElement row;

    public string Username => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l0 r0')]")).Text;
    public string Firstname => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l1 r1')]")).Text;
    public string Lastname => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l2 r2')]")).Text;
    public string Type => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l3 r3')]")).Text;
    public string Crew => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l4 r4')]")).Text;
    public string JobTitle => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l5 r5')]")).Text;
    public string DefaultPrice => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l6 r6')]")).Text;
    public string Future => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l7 r7')]")).Text;
    public string Language => row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l8 r8')]")).Text;

    public override string ToString()
    {
        return "SharePrice: " + Username.ToString() + ": " + Firstname.ToString();
    }

    public UserTableRow(IWebElement row)
    {
        try
        {
            this.row = row;
        }
        catch (Exception)
        {

            throw;
        }
    }
}

Here is the method itself for adding data into List.

    public static IEnumerable AddItemsToList(IWebDriver driver)
    {

        IReadOnlyCollection rows = new List();

        List DataHere = new List();

        driver.FindElement(By.XPath("//*[@id=\"disabled_users_show_label\"]")).Click();

        Thread.Sleep(7000);

        driver.FindElement(By.XPath("//*[@id=\"users_table\"]/div[5]/div/div[1]")).Click();

        for (int i = 0; i < 11; i++)
        {
            rows = driver.FindElements(By.CssSelector("#users_table .slick-viewport .slick-row"));

            // Now we will iterate through cells in table and compare to what we already have in the list
            foreach (IWebElement cell in rows)
            {
                rows = driver.FindElements(By.CssSelector("#users_table .slick-viewport .slick-row"));

                // Add data to our object
                DataHere.Add(new UserTableRow(cell));

                //We need to select table first to be able to scroll down. We do it directly here
                Actions actions = new Actions(driver);
                actions.SendKeys(Keys.ArrowDown).Build().Perform();

            }

        }

        List noDupes = DataHere.Distinct().ToList();

        return noDupes;
    }

It goes fine but at some point (I don't know why) it catches an error saying:

OpenQA.Selenium.StaleElementReferenceException: 'stale element reference: element is not attached to the page document (Session info: chrome=84.0.4147.105)'

Pointing at row.FindElement(By.XPath(".//div[contains(@class, 'slick-cell l0 r0')]")).Text;

Any ideas how to add try catch to Parameters so code will continue execution until adding to list has been finished?

The reason I am calling

rows = driver.FindElements(By.CssSelector("#users_table .slick-viewport .slick-row"));

Inside the loop is I am trying to update table values to iterate through after scrolling down. This is a tricky table that unloads items and loads other items from database on the scroll. So I am trying to mimic user scroll with keyboard and catch table items to List.

It would be perfect to compare to list items on the flight. However I was thinking to get all to the List (even duplicates) and then later make other list where I will eliminate duplicates wit something like:

    List noDupes = DataHere.Distinct().ToList();

HTML:


    
    
        
            Username:
                
            
            Firstname:
                
            
            Lastname:
                
            
            Type:
                
            
            Crew:
                
            
            Job title:
                
            
            Defaultprice class:
                
            
            Description:
                
            
            Language:
                
            
        
    
    
        
        
    
    
        
    
    
        
            
                john.smith
                John
                Smith
                Contractor
                Microsoft
                Sales manager
                A
                
                en
            
            
                robert.geits
                Robert
                Geits
                Staff
                Google
                Project manager
                B
                
                de
            
            
                amir.rooney
                Amir
                Rooney
                Staff
                Microsoft
                Sales manager
                A
                
                en
            
            # ETC # ETC # ETC # ETC # ETC # ETC # ETC # ETC # ETC # ETC

Paperclip Bob · Accepted Answer

You are refreshing rows inside the foreach, but rows is the iterable object of the foreach. That I guess isn't illegal (after all, it compiled) but seems asking for trouble. My opinion is the stale element reference is somehow related to rows being reassigned within its own foreach loop.

However, after you assign rows the 2nd time, you never use it inside the foreach. It almost seems like it is an unintentional statement that you can delete. After all, why bother assigning rows inside the foreach and then never use it inside the foreach?

If you are assigning rows a 2nd time because you are comparing to what's already been processed, I would give the 2nd rows assignment a different variable name (say, rows_B). But as I said, I don't see it happening here. You just assign rows inside foreach and never use it.

Or, if the reason for the 2nd rows assignment because the table is dynamically changing while you're processing the table? If so, I think the best you can do is catch the error and try to recover from whatever changed on you midstream. I know that sounds vague, but really, if your input data is changing while you're trying to process that input the best you could ever hope for is a best-effort effort.

For example, you could create a 2nd rows variable (say, rows_b) to refresh within the foreach loop. Then compare rows_b with the original rows. If they are identical - awesome, just keep processing.

If they differ, I would keep track of which rows you've processed to that point, break out of the loop and start over. During the foreach you'd check if a row had been previously processed and if so skip to the next iteraton.

If this is what you're trying to do - and want to see a code example - let me know. I didn't want to write the code if it isn't even what you're trying to accomplish ;-)

Iterating through website table (Element is not attached to the page document)

Answers (1)

Related Questions