CODEWITHSUNDEEP
pythonpython-2.7selenium-webdriver
Parzival
Parzival

Reputation: 2064

multiple download folders using Selenium Webdriver?

I'm using the Selenium webdriver (in Python) to automate the donwloading of thousands of files. I want the files to be saved in different folders. The following code works, but it requires quitting and re-starting the webdriver multiple times, which slows down the process:

some_list = ["item1", "item2", "item3"] # over 300 items on the actual code
for item in some_list:
    download_folder = "/Users/myusername/Desktop/" + item
    os.makedirs(download_folder)
    fp = webdriver.FirefoxProfile()
    fp.set_preference("browser.download.folderList", 2)
    fp.set_preference("browser.download.manager.showWhenStarting", False)
    fp.set_preference("browser.download.dir", download_folder)
    fp.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/plain")
    browser = webdriver.Firefox(firefox_profile = fp)

    # a bunch of code that accesses the site and downloads the files

    browser.close()
    browser.quit()

So, at every iteration I have to quit the webdriver and re-start it, which is pretty innefficient. Is there a better way to do this? Apparently we can't change the Firefox profile after the webdriver is instantiated (see this and this previous questions), but perhaps there is some alternative I'm missing?

(Mac OS X 10.6.8, Python 2.7.5, Selenium 2.2.0)

Upvotes: 0

Views: 1595

Answers (1)

Yi Zeng
Yi Zeng

Reputation: 32895

No, I don't think you can do it.

Option one: specify different default directories for one FirefoxProfile

You can't. In my opinion, this is the issue with Firefox, not Selenium. However, this Firefox limitation looks like the correct design to me. browser.download.dir is the default download destination, if it allows multiple directories, then that's not "default" anymore.

Option two: switch multiple FirefoxProfile for one driver instance

If not doing it in Firefox, can FirefoxProfile be switched for same driver instance? As far as I know, the answer is no. (You have already done some research on this)

Option three: use normal non-Selenium way to do the downloading

If you want to avoid using this auto-downloading approach and do it the normal way (like Auto-it etc.), then it falls in the category of "How To Download Files With Selenium And Why You Shouldn’t". But in this case, your code can be simplified.

some_list = ["item1", "item2", "item3"] # over 300 items on the actual code
for item in some_list:
    download_folder = "/Users/myusername/Desktop/" + item
    some_way_magically_do_the_downloading(download_folder)

Upvotes: 1

Related Questions