CODEWITHSUNDEEP
c#asp.net-coreselenium-webdriverselenium-chromedriver.net-5
Konstantin Konstantinov
Konstantin Konstantinov

Reputation: 1413

How to run numerous web acceptance tests on a single machine

Let's say that I want to run so called web acceptance tests on a modern (as of the day of the question) machine, which, let's say, routinely has somewhere between 16 and 128 logical cores. The number could be different in each particular case, but let's stick with this range for now.

By web acceptance test I mean a test, which opens a web page in one of the browsers (Chrome / Firefox / Edge / ... ) using a driver (e.g. chromedriver / geckodriver, etc...), manipulates a web page in whatever way the test wants, and then "collects" some output (e.g. - does the web page has this or that element OR did it navigate to this or that expected page). The actual detail are irrelevant.

Given that such tests naturally spend most of the time waiting (so that to be sure that once they want to manipulate some web page [element] then it has been loaded for sure) it is then seems reasonable to assume that if I have N logical cores on the machine, then I should be able to spawn at least N of such web acceptance tests.

A typical C# code to do that can be summarized as follows:

namespace WebAcceptanceTests
{
    public static class Chrome
    {
        public static async Task Run(
            Uri uri, 
            Func<ChromeDriver, Task> manipulate, 
            Action<ChromeDriver> validate)
        {
            var chromeDriverService = ChromeDriverService.CreateDefaultService();
            chromeDriverService.HideCommandPromptWindow = true;
            var options = new ChromeOptions();

            // To make Chrome window invisible.
            options.AddArgument("--headless");
            using var driver = new ChromeDriver(chromeDriverService, options);

            try
            {
                driver.Manage().Window.Maximize();
                driver.Navigate().GoToUrl(uri);

                await manipulate(driver);
                validate(driver);
            }
            finally
            {
                driver.Quit();
            }
        }
    }
}

where manipulate performs some "manipulation" of the page (e.g. attempts to click some buttons / enters some text / etc...) and validate performs some validation (e.g. if manipulate entered user name and password and then clicked login button, then did the site actually transitioned to logged in page). The actual details of what these manipulate and validate do are irrelevant. However, manipulate is a lengthy process because the site needs to load the page and do some "work" here or there. Therefore, we can model it by a method, which just waits and does nothing, e.g.:

public static async Task Manipulate(ChromeDriver driver)
{
    // Do some useful stuff here instead of just waiting.
    await Task.Delay(60_000);
}

However, if I start spawning such drivers then very quickly (with under 10 drivers created) some of the created drivers start producing weird errors like:

OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL http://localhost:52382/session timed out after 60 seconds.

The test server machine that I am getting these errors has 16 cores and enough RAM to open hundreds of Chrome tabs without any problems, yet a small number of chrome drivers (less than 10) seems not working in parallel.

How do I make many Chrome drivers work in parallel? Ideally I'd want to open (3-4X the number of cores) drivers because they will mostly wait and do nothing.

Upvotes: 0

Views: 151

Answers (1)

AlfeG
AlfeG

Reputation: 1473

We achieve this using NUnit parallel run, parallelizable by fixture.

Allocate driver during OneTimeSetup. Do whatever test need in single fixture. On OneTimeTearDown, dispose driver. We do this in base class that all web acceptance test fixtures are inherit

    [Parallelizable(ParallelScope.Fixtures)]
    public abstract class WebDriverTest
    {
        protected IDriver driver;

        [OneTimeSetup]
        public void PrepareDriver()
        {
            // ...
            this.driver = new ChromeDriver(chromeDriverService, options);
            // ...
        }

        [OneTimeTearDown]
        public void CleanupDriver()
        {
            this.driver.Dispose();
        }

        [TearDown]
        public void ScreenshotForFailedTest()
        {
            var testStatus = GetTestStatus();

            if (!string.IsNullOrEmpty(testStatus) && testStatus.Equals("Failed"))
            {
                this.driver.TakeScreenshot(); // extension method with a take screenshot functionality
                // log more details if needed
            }
        }
    }

[OneTimeTearDown] is executed even if there is failed tests

As bonus we take screen

Using this snippet we run around 500 smoke tests against Chrome in 5-6 minutes on each commit.

Upvotes: 0

Related Questions