CODEWITHSUNDEEP
pythongoogle-chromefirefoxselenium
David542
David542

Reputation: 110432

Benchmarking memory consumption of Firefox vs Chrome in webdriver

I am trying the maximize the number of browsers that I can run a script on on my computer. How would I figure out how much memory consumption each Chrome or Firefox process is taking when I launch webdriver. For example:

from selenium import webdriver
driver = webdriver.Chrome()
driver.get('www.yahoo.com')

# - or - #

from selenium import webdriver
driver = webdriver.Firefox()
driver.get('www.yahoo.com')

Upvotes: 3

Views: 1890

Answers (1)

unutbu
unutbu

Reputation: 880547

Instantiating webdriver.Chrome or webdriver.Firefox spawns at least one other process, so to get an estimate of total memory consumption it might be easiest to measure total system memory available before and after launching the processes. How to find the total memory used depends on your OS; the psutils module supports Linux, Windows, OSX, FreeBSD and Sun Solaris.

import os
import multiprocessing as mp
import contextlib
import time
import selenium.webdriver as webdriver
import selenium.webdriver.support.ui as UI
import selenium.webdriver.support.expected_conditions as EC
import psutil

def worker(browsername, args, N):
    start_mem = psutil.virtual_memory()
    Driver = getattr(webdriver, browsername)
    drivers = [Driver(*args) for i in range(N)]
    end_mem = psutil.virtual_memory()
    for driver in drivers:
        driver.quit()
    return end_mem.used - start_mem.used

if __name__ == '__main__':
    N = 10
    pool = mp.Pool(1)

    for driver, args in [('Firefox', ()),
                         ('Chrome', ()), 
                         ('PhantomJS', ('phantomjs',))]:
        used = pool.apply(worker, [driver, args, N])
        print('{:>10}: {}'.format(driver, used))
        time.sleep(5)

    pool.close()
    pool.join()

yields

   Firefox: 1072779264
    Chrome: 1124683776
 PhantomJS: 102670336

So it appears PhantomJS uses about 10x less memory.

Technical note: There is no reliable way to force a Python process to relinquish memory it has used back to the OS short of terminating the process. So I used multiprocessing to spawn each test in a separate process so when it terminates the memory used is freed. The pool.apply() blocks until worker ends.

Upvotes: 5

Related Questions