CODEWITHSUNDEEP
pythonunit-testing
Ginger
Ginger

Reputation: 8630

How to unit test a downloading function?

I have a python function that downloads a few files.

e.g.

def downloader():
    file_list=['fileone.htm','filetwo.htm','filethree.htm']
    for f in file_list:
        (filename,headers) = urllib.urlretrieve(f,'c:\\temp\\'+f)

What is the correct way to unit test the function? Whether or not it works is dependent on how the urlretrieve function behaves, which is dependent on external factors.

Upvotes: 5

Views: 3344

Answers (2)

Philip
Philip

Reputation: 308

If all you want to test is that the function calls urlretrieve on all elements in file_list you can modify the function to take the retrieve-function as a parameter:

def downloder(urlretrieve):
  file_list=['fileone.htm','filetwo.htm','filethree.htm']
  for f in file_list:
    (filename,headers) = urlretrieve(f,'c:\\temp\\'+f)

And then in your unit test you can create a custom function and check that is called the correct amount of times and with the correct parameters.

calls = []
def retrieve(url, local) :
  calls.append([url,local])

assert(len(calls) == 3)
assert(calls[0][0] == 'fileone.html')
assert(calls[0][2] == 'c:\\temp\\fileone.html')
...

You can use the library Mock to simplify the part of creating your own retrieve function for the unit test.

Upvotes: 4

Ludo
Ludo

Reputation: 842

Basically, if you need to test how your logic responds to the behaviour of the urlretrieve function, you need to inject a simulated behaviour into your program flow. One way of doing this is to wrap the urllib functionality in a module or class that generates this behaviour when the unit test is run. For example, if your download functionality is in its own module, you can do something like this (simplified pseudo-code):

file: dowloader.py

class UrllibWrapper:
    # thin wrapper for urllib

class Downloader:
    def __init__(self, urllib_class=None):
        if urllib_class is None:
            self.ul = UrllibWrapper()
        else:
            self.ul = urllib_class

    def download(self, f, dest):
        self.ul.urlretrieve(f, dest)

if __name__ == '__main__':
    class FakeUrllibWrapper:
        # implement desired behaviour of urrlib

    ul = FakeUrllibWrapper()
    dl = Downloader(ul)
    # do unit tests

In your progam you would do:

# ...
from downloader import Downloader
dl = Downloader()
for f in files:
    dl.download(f, dest)

Upvotes: 0

Related Questions