CODEWITHSUNDEEP
pythonpython-3.xmultithreadingseleniumparallel-processing
Stoopid Newbie
Stoopid Newbie

Reputation: 51

How To Run Multiple Selenium Drivers Parallelly?

How can I run multiple selenium drivers parallelly to login into a instagram using text file inputs (email:password format).

I would like to open n number of drivers and do in paralel to every one of them and not one by one.

I've tried threading but it was just stuck at refreshing the same selenium driver (maybe I did it wrong).

Should I use threading or multiprocessing ?

I'm new to this so I honestly don't know how to do it here is the snippet of my current code for doing it in one driver

executable_path = "chromedriver.exe"
options = Options()
options.add_argument("--start-maximized")
driver = webdriver.Chrome(executable_path=executable_path, options=options)
file = open("list.txt", "r")
for line in file:
    pieces = line.split(":")
    myemail = pieces[0]
    mypass = pieces[1]
    driver.get('https://www.instagram.com/accounts/login/')
    time.sleep(5)
    element = driver.find_elements_by_css_selector('form input')[0]
    element.send_keys(myemail)
    element1 = driver.find_elements_by_css_selector('form input')[1]
    element1.send_keys(mypass)

Upvotes: 4

Views: 7911

Answers (2)

Aqua 4
Aqua 4

Reputation: 871

It is recommeneded to use multi-processing with python,

Below is an example to open different urls parallely

import multiprocessing as mp


def worker(url):
    executable_path = "chromedriver.exe"
    options = Options()
    options.add_argument("--start-maximized")
    driver = webdriver.Chrome(executable_path=executable_path, options=options)
    driver.get(url)
    # do the necessary operations
    # task1; task2; task3
    # close instance after completed
    driver.quit()


url_list = ["https://github.com/Aqua-4/auto-insta/blob/master/refresh_db.py","https://stackoverflow.com/questions/59706118/how-to-run-multiple-selenium-drivers-parallelly"]
if __name__ == '__main__':
    p = mp.Pool(mp.cpu_count())
    p.map(worker, url_list)

This will open up as many selenium instances as the number of cores on your device, also you can mp.cpu_count() change this number, but it is not recommended to increase it above the number of cores on the device

Upvotes: 4

Serhii Matvienko
Serhii Matvienko

Reputation: 312

Not sure how it's should be on Python), but on C# it will be looks like this:

private IWebDriver GetSeleniumDriver()
{
    return new ChromeDriver(ChromeDriverService.CreateDefaultService(), new ChromeOptions());
}

...

var listDrivers = new List<IWebDriver>();
listDrivers.Add(GetSeleniumDriver());
// you can create as many as you want

You should keep links on each instance and just switch between them.

Parallel.ForEach(listDrivers, driver =>
{
   // do what you want
});

I believe the idea should be the same on Python.

Upvotes: 2

Related Questions