CODEWITHSUNDEEP
c#selenium
Arif YILMAZ
Arif YILMAZ

Reputation: 5866

How to iterate all links in a website with Selenium in C#

I am very new to Selenium and trying to do a small project to get images from website's pages.

It throws StaleElementReferenceException in the second loof of foreach. It throws the exception in the if statement. I know that after GoToUrl(), it cannot use GetAttribute() but how am I supposed to iterate all webpages?

driver.Navigate().GoToUrl("http://www.xxxxxxxx.com/"); // dummy web address
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));
IList<IWebElement> results = driver.FindElements(By.CssSelector(".list-menu > li > ul > li > a"));

foreach (IWebElement result in results)
{
    if (result.GetAttribute("href").Length>0) // It throws EXCEPTION here *******
    {
        driver.Navigate().GoToUrl(result.GetAttribute("href"));
        driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(20));

        //serie-list-product-four-column
        IList<IWebElement> gridResults = driver.FindElements(By.CssSelector(".serie-list-product-four-column > li > a"));
        foreach (IWebElement gridResult in gridResults)
        {

        }
    }
}

Upvotes: 1

Views: 1351

Answers (2)

MasterJoe
MasterJoe

Reputation: 2335

Ayilmaz, if you are using an ide, then you can check if there is back() method in navigate. There is ! Alternately, you can google "selenium navigate api". Then, go to official documentation here http://docs.seleniumhq.org/docs/03_webdriver.jsp. Look under: Navigation: History and Location.

Another thing to try is create two browsers. One browser to hold page with all links and other page to open each link. Why go backward and forward and keep reloading the main page ? The disadvantage is that you need more memory to create any additional browsers.

You could try to create a new window using or a new tab using "Actions" instead of creating a new browser. In my limited experience, managing windows and tabs is not easy and it can be unreliable in chrome (I created two tabs with unique window handles and chrome was not able to switch from new tab to previous tab !!!).

Upvotes: 0

Pseudo Sudo
Pseudo Sudo

Reputation: 1412

You need to navigate back to the page that the link is on before you can navigate to the next link. The stale link exception is thrown because the link you are attempting to navigate to is not on the page that the selenium driver is on. Just add 'driver.Navigate().Back()' where needed.

This could be done very elegantly with a recursive function.

I'd be happy to share a recursive function I made to perform this task with you, but I'm currently on mobile.

Upvotes: 2

Related Questions