CODEWITHSUNDEEP
pythonunit-testingcsvstreamstringio
David258
David258

Reputation: 757

Unittesting - replacing filepaths with StringIO objects

I am trying to unittest parsing functions which take a filepath and return some of the file contents. I would like to be able to pass these functions strings of data for testing purposes.

I understand that I can pas csv.reader() either StringIO or a file_handle (e.g. csv.reader(StringIO("my,data") or csv.reader(open(file))), but I can't see a way that I can pass a StringIO object in place of a filepath since open(StringIO("my, data")) fails. Equally I want to have the file open/close logic in these parsing methods rather than in the main bulk of my code as that would clutter my main code and also mean I have to re-write all the file IO interfaces!

It seems my choices are:

  1. Rewrite all the existing code so that it passes file handles to the parsing functions - this is a real pain!
  2. Use mock.patch() to replace the open() method - this should work, but seems more complex than this task should require!
  3. Do something which I haven't yet thought of, but am convinced must exist!

    import csv
    def parse_file(input):
        with open(input, 'r') as f:
            reader = csv.reader(f)
            output = []
            for row in reader:
                #Do something complicated
                output.append(row)
            return output


import unittest  
class TestImport(unittest.TestCase):
    def test_read_string(self):
        string_input = u"a,b\nc,d\n"
        output = read_file(string_input)
        self.assertEqual([['a', 'b'], ['c', 'd']], output)
    def test_read_file(self):
        filename = "sample_data.csv"
        output = read_file(filename)
        self.assertEqual([['a', 'b'],['c', 'd']], output)

Upvotes: 1

Views: 6124

Answers (3)

David258
David258

Reputation: 757

For others looking for this in future I was able to use Mock to do this quite effectively.

---- module: import_data.py -----

import csv

def read_file(input):
    with open(input, 'r') as f:
        reader = csv.reader(f)
        output = []
        for row in reader:
            #Do something complicated
            output.append(row)
        return output

---- Unittests ----

import unittest
from io import StringIO
from mock import patch
from import_data import read_file

class TestImport(unittest.TestCase):

    @patch('import_data.open')
    def test_read_string(self, mock_file):
        mock_file.return_value = StringIO(u"a,b\nc,d")
        output = read_file(None)
        self.assertEqual([['a', 'b'], ['c', 'd']], output)


    def test_read_file(self):
        filename = "sample_data.csv"
        output = read_file(filename)
        self.assertEqual([['a', 'b', 'c'],['d', 'e', 'f']], output)

Upvotes: 0

Don Kirkby
Don Kirkby

Reputation: 56710

If you don't want to change the interface to accept open file objects like StringIO, look at the testfixtures module. I have used it to manage files and directories for unit tests, although I usually prefer to pass in StringIO objects.

If you don't like that, then patching open() sounds like a reasonable strategy. I haven't tried it, myself.

Upvotes: 1

kmaork
kmaork

Reputation: 6012

You can use temporary files.

If you really prefer not to use the hard disk, you can use StringIO to replace your files, and redefine the builtin open function, like so:

import StringIO
import csv

#this function is all you need to make your code work with StringIO objects
def replaceOpen():
    #the next line redefines the open function
    oldopen, __builtins__.open = __builtins__.open, lambda *args, **kwargs: args[0] if isinstance(args[0], StringIO.StringIO) else oldopen(*args, **kwargs)

    #these methods below have to be added to the StringIO class
    #in order for the with statement to work
    StringIO.StringIO.__enter__ = lambda self: self
    StringIO.StringIO.__exit__ = lambda self, a, b, c: None

replaceOpen()

#after the re-definition of open, it still works with normal paths
with open(__file__, 'rb') as f:
    print f.read(16)

#and it also works with StringIO objects
sio = StringIO.StringIO('1,2\n3,4')
with open(sio, 'rb') as f:
    reader = csv.reader(f)
    output = []
    for row in reader:
        output.append(row)
    print output

This outputs:

import StringIO
[['1', '2'], ['3', '4']]

Upvotes: 3

Related Questions