CODEWITHSUNDEEP
pythonseleniumpython-2.7
Arup Rakshit
Arup Rakshit

Reputation: 118271

can python be useful to open multiple tabs in a browser in one shot?

I am looking for a faster way to do my task. i have 40000 file downloadable urls. I would like to download them in local desktop is.now the thought is currently what I am doing is placing the link on the browser and then download them via a script.now what I am looking for is to place 10 urls in a chunk to the address bar and get the 10 files to be downloaded at the same time.If it possible hope overall time will be decreased.

Sorry I was late to give the code,here it is :

def _download_file(url, filename):
    """
    Given a URL and a filename, this method will save a file locally to the»
    destination_directory path.
    """
    if not os.path.exists(destination_directory):
        print 'Directory [%s] does not exist, Creating directory...' % destination_directory
        os.makedirs(destination_directory)
    try:
        urllib.urlretrieve(url, os.path.join(destination_directory, filename))
        print 'Downloading File [%s]' % (filename)
    except:
        print 'Error Downloading File [%s]' % (filename)


def _download_all(main_url):
    """
    Given a URL list, this method will download each file in the destination
    directory.
    """

    url_list = _create_url_list(main_url)
    for url in url_list:
        _download_file(url, _get_file_name(url))

Thanks,

Upvotes: 1

Views: 479

Answers (1)

dr jimbob
dr jimbob

Reputation: 17731

Why use a browser? This seems like an XY problem.

To download files, I'd use a library like requests (or make a system call to wget).

Something like this:

import requests

def download_file_from_url(url, file_save_path):
    r = requests.get(url)
    if r.ok: # checks if the download succeeded
        with file(file_save_path, 'w') as f: 
           f.write(r.content)
        return True
    else:
        return r.status_code

download_file_from_url('http://imgs.xkcd.com/comics/tech_support_cheat_sheet.png', 'new_image.png')
# will download image and save to current directory as 'new_image.png'

You first have to install requests using whatever python package manager you prefer e.g., pip install requests. You can also get fancier; e.g.,

Upvotes: 2

Related Questions