Python Nose tests, SQLAlchemy, and "convenience functions"

Question

I have a question about how to design good Nose unit tests (using per test transactions and rollbacks) not just around SQLAlchemy models, but also around convenience functions I've written which surround the creation of SQLAlchemy models.

I have a decent understanding of how to write a basic unit test class, which includes the necessary setup and teardown fixtures to wrap all of the tests in transactions, and roll them back when the test is complete. However, all of these tests so far involve directly creating models. For example, testing a User model like so (BaseTestCase contains setup/teardown fixtures):

from Business.Models import User

class test_User(BaseTestCase):

    def test_user_stuff(self):
        user = User(username, first_name, last_name, ....)
        self.test_session.add(user)
        self.test_session.commit()
        # do various test stuff here, and then the
        # transaction is rolled back after the test ends

However, I've also written a convenience function which wraps around the creation of a User object. It handles various things like confirming password matches verification password, then creating a salt, hashing password + salt, etc, and then putting those values into the relevant columns/fields of the User table/object. It looks something like this:

def create_user(username, ..., password, password_match):

    if password != password_match:
        raise PasswordMatchError()

    try:
        salt = generate_salt()
        hashed_password = hash_password(password, salt)

        user = User(username, ..., salt, hashed_password)
        db_session.add(user)
        db_session.commit()

    except IntegrityError:
        db_session.rollback()
        raise UsernameAlreadyExistsError()

    return user

Now, I'd like to unit test this function as well, but I'm not sure the correct way to wrap this in unit test cases which implement using a test database, rolling back transactions after every test, etc.

from Business.Models.User import create_user

class test_User(BaseTestCase):

    def test_create_user_stuff(self):
        user = create_user(username, first_name, last_name, ....)
        # do various test stuff here
        # Now how do I finangle things so the transaction
        # executed by create_user is rolled back after the test?

Thanks in advance for the help, and pointing me in the right direction.

Answer 1

before we talk about appropriate unittest pattern for your app, may I recommend few things (this will significantly simplify your app flow and tests):

1. do not session.commit() from create_user() or, in fact, from any convenience function or method you create, i.e replace session.commit() with session.flush() to persist data w/o committing transaction.

2. modify your validation exceptions (e.g. UsernameAlreadyExistsError) to make sure it does session.rollback() on error. This will guarantee you're not committing the whole thing if some validation(s) fails.

3. session.commit() once at the very end, when the entire request is processed and all methods and validation passed. This can be achieved by, for example, calling commit() from your controller, e.g. on request termination. Alternatively (this is what I do), you can call commit() from object destructor. You can take it further and introduce commit=1|0 flag which must be explicitly passed by a caller (commit by default if this flag is now passed).

if that approach works for you, suddenly your unittest implementation becomes much cleaner. First, implement base unittest class which will be overwritten by every other unittest, e.g.

from Business.Models.User import create_user

class BaseTestCase(unittest2.TestCase):

  @classmethod
  def setUpClass(cls):
    cls.session = ... # get your already init'ed session, init it otherwise (normally it should be done once)
    # other common statements
    ... 

  def setUp(self):
    self.session.rollback()
    ...

  def tearDown(self):
    self.session.rollback()
    ...

class UserTest(BaseTestCase):

  @classmethod
  def setUpClass(cls):
    super(UserTest, cls).setUpClass()
    ...

  def setUp(self):
    super(UserTest, self).setUp()
    ...

  def tearDown(self):
    super(UserTest, self).tearDown()

  def test_create_user_success(self):
    user = create_user('john_smith', 'John', 'Smith')
    self.session.commit()
    user_was_created = ... # verify

  def test_create_user_error(self):
    user = create_user('john_smith', 'John', 'Smith')
    self.session.commit()
    with self.assertRaisesRegexp(UsernameAlreadyExistsError, "john_smith already exists"):
      user = create_user('john_smith', 'John', 'Smith')

Answer 2

Here are two possible approaches:

Recreate the DB between tests

In your setUp() code, start from a blank database, creating the tables you need for the objects you're testing. In tearDown(), clean up after yourself.

You can create a base test class like:

class SqlaTestCase(unittest.TestCase):
    db_url = 'sqlite:///:memory:'
    auto_create_tables = True

    def setUp(self):
        self.engine = create_engine(self.db_url)
        self.connection = self.engine.connect()

        if self.auto_create_tables:
            self.create_tables()

        Session = sessionmaker(bind=self.connection)
        self.session = Session()

    def tearDown(self):
        self.session.close_all()
        if self.auto_create_tables:
            self.drop_tables()
        self.connection.close()
        self.engine.dispose()

    def create_tables(self):
        self.Base.metadata.create_all(self.engine)

    def drop_tables(self):
        self.Base.metadata.drop_all(self.engine)

Then each test doesn't have to worry about leaving committed data behind.

Don't retest the same functionality at different levels

If you have good test coverage, the underlying functionality this helper calls is already tested. So you can focus on making sure you have the correct logic at this level.

Mock out the session (and even the password hashing functionality) and test to make sure the appropriate calls are made (e.g. assert_called_with()).