Flask Testing in Python -- Building one API in a repo with many to unit test it via import_module

Question

We have an ETL data API repo. We do all etl processing inside of it, then spit the data out in API's. These API's are run one at a time from a single command passing the resource class through to the server to build an API. the resource class is in a web directory in an __init__.py.

This is a wonderful convention and quite simple to use, but the problem I am having is coming from trying to get one of the 3 API's available spun up for testing. Our directory stucture is like this (calling the project 'tomato')

tomato

- category_api
    - web
    - etl
    - test
        - here is where we are writing some tests (test_category_api.py)
    - data
- article_api
    - web
    - etl
    - test
    - data
- recommendation_api
    - web
    - etl
    - test
    - data
- common
    - common shit

Inside this test, I have the following test class. On the seventh line up from the bottom, you will see a comment on where it breaks. It is the import_module method.

    import unittest
    import sys
    import os
    import sys
    import json

    from importlib import import_module
    from flask import Flask
    from flask_restful import Api, abort, wraps
    from flask_restful.utils import cors
    from flask.ext.testing import TestCase

    #dir_above_top_level = os.path.join(os.path.abspath(__file__), '../../..    /')
    #sys.path.append(os.path.abspath(dir_above_top_level))

    _CATEGORY_ENDPOINT = '/v1/category/'
    _PACKAGE_NAME = os.environ['RECO_API_PACKAGE_NAME']
    _CORS = cors.crossdomain(origin='*',
                             headers=['Origin', 'X-Requested-With',
                                      'Content-Type', 'Accept'],
                             methods=['GET', 'HEAD', 'OPTIONS'],
                             max_age=3600)

    class CategoryTests(TestCase):
        def __init__(self):
            self.app = Flask(__name__)
            self._configure_app()
            for resource in self.resource_classes:
                self.api.add_resource(self.resource,
                                      self.resource.URI_TEMPLATE)
        def test_status_code(self):
            self.response = self.client.post(_CATEGORY_ENDPOINT,
                                         data=json.dumps(
                                         {'title': 'Enjoy this delicious food'}),
                                         headers=json.dumps(
                                         {'content-type':'application/json'}))
            self.assertEqual(self.response.status_code, 200)

        def test_version(self):
            self.response = self.client.post(_CATEGORY_ENDPOINT,
                                             data=json.dumps(
                                             {"title": "eat some delicious stuff"}),
                                             headers=json.dumps(
                                             {'content-type':'application/json'}))
            self.assertEqual(json.dumps(self.response['version']), '1')

        def _configure_app(self):
            self.app = Flask(__name__)
            self.app.config['TESTING'] = True
            self.app.debug = True
            self.decorators = [_CORS]
            self.app.Threaded = True
            self.web_package = 'tomato.category.web'
            self.package = import_module('.__init__', self.web_package) # WE BREAK HERE
            self.resources = package.RESOURCE_NAMES
            self.resource_classes = [ getattr(package, resource) for resource in resources ]
            self.api = Api(self.app, catch_all_404s=True, decorators=self.decorators)

    if __name__ == '__main__':
        unittest.main()

we are given an exception when running these tests:

ImportError: No module named tomato.category.web.__init__

yet cd into the main top dir, and ls tomato/category/web gets us __init__.py and its right there with the resource class.

How do I import this class so that I can instantiate the API to run the tests in this class? Or if I'm completely on the wrong track what should I be doing instead?

Answer 1

The problem here lies in the directory structure. In the current path, I am not at the top level. It is a module. So What was needed was to uncomment the line two lines at the top, and change the structure to append the path like this.

dir_above_top_level = os.path.join(os.path.abspath(__file__), '../../../..')
sys.path.append(os.path.abspath(dir_above_top_level))

and now, I can import it using

self.web_package = 'tomato.category.web'
self.package = import_module('.__init__', self.web_package)

and now it will import fine and I can grab the resource class to set up the testing API

Answer 2

You don't need to import __init__, just like you probably wouldn't do from tomato.category.web import __init__. You should be able to import the web package directly.

self.web_package = 'tomato.category.web'
self.package = import_module(self.web_package)