Setting up Pyramid 1.5 testing the right way

Question

While Pyramid documentation is usually quite good, I can't find how to set up integration tests for a Pyramid 1.5 application (note essential code fragment missing!). That is, where to put particular configurations.

In the __init__ I have main() function, which includes some other modules, among which there is one, which requires SQLAlchemy. Another complication is that auth policy is dependent on settings, so all that lives in main (otherwise I'd made includeme, not sure if it is relevant to this question).

At first I tried the following (after gathering what I could into includeme function):

class ViewTests(unittest.TestCase):
    def setUp(self):
        self.config = testing.setUp()
        self.config.include('myproject.includeme')

Which of course fails somewhere in the sqlalchemy engine_from_config. But then I've found, that I can probably use main() and provide test.ini for configuration. Still I do like the all-included approach of the gist:

class IntegrationTestBase(BaseTestCase):
    @classmethod
    def setUpClass(cls):
        cls.app = main({}, **settings)
        super(IntegrationTestBase, cls).setUpClass()

    def setUp(self):
        self.app = webtest.TestApp(self.app)
        self.config = testing.setUp()
        super(IntegrationTestBase, self).setUp()

Part of the problem with the code above is that settings are not ending up in self.config. If I enhance it with self.config = testing.setUp(settings=settings), the test still fails:

AttributeError: 'DummyRequest' object has no attribute 'include'

pyramid_bowerstatic does not have its chance to modify request, etc. So, instead of having integration tests out-of-the-box (the app itself runs without complains!) with given config and concentrate on writing tests, I need to care for all third-party modules whims.

Thus, I hope there is some better way to do integration tests, which are deal with several included packages, custom auth policies, events, and other "magic".

Py.test is to be used, if that matters. pytest_pyramid seems to be relevant, but docs do not have any examples.

Not exactly, but similar problem: How to Make view_config decorator work with a Pyramid Unit Test?, especially in the comments to the answer.

I have found ad hoc solution (which works for me for now) in Japanese: http://qiita.com/podhmo/items/2c6d8cb78c1f84569f0a

but the question how to reliably setup integration tests, equivalent to application in all aspects, without looking into third-party modules, except for specific ini-file?

Update: this is one of the calls in question:

View:

components = pyramid_bowerstatic.create_components('sbadmin',
             os.path.join(os.path.dirname(__file__), 'bower_components'))
class MainViews(Layouts):
    @view_config(route_name='home', renderer='templates/mytemplate.pt')
    def my_view(self):
        self.request.include(components, 'jquery')
        return {'project': 'Myproject'}

In tests:

class ViewTests(IntegrationTestBase):
    def test_my_view(self):
        from myproject.views.main import MainViews
        request = self._make_request(self.config)  # before was: DummyRequest
        info = MainViews(request).my_view()
        self.assertEqual(info['project'], 'Myproject')

Due to nicely decoupled architecture of Pyramid, it is probably enough to have all-inclusive request. That is, the question can be reformulated as: What is generic _make_request function, which gives a result, identical to the running application request (with extensions, tweens, added request attributes, including those from 3rd party modules, etc)? Is there some ready factory? IMHO, it is NOT integration testing if a developer needs to mock it's own "integration" to test against vs. take what application really is. I am not sure if these are the only things, but I guess the factory should at least provide a request with all hooks like mentioned here: http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/hooks.html regardless of which included package added them.

Update 2: It came to my mind that there is quite clear first approximation indicator for discrepancies between test and run-time environments. I have compared requests, for a normal call and test call to the view. First, here is my _make_request, which is sufficient for adding request methods, but much more is missing (below):

def _make_request(self, config, path="/"):
    from pyramid.request import Request
    from pyramid.interfaces import IRequestExtensions
    request = Request.blank(path)
    extensions = config.registry.getUtility(IRequestExtensions)
    request.registry = config.registry
    request._set_extensions(extensions)
    return request

Normal call has the following attributes (request.__dict__.keys()):

['traversed',
 'virtual_root',
 'virtual_root_path',
 'subpath',
 'request_iface',
 '__view__',
 'view_name',
 'tm',
 'environ',
 'registry',
 'context',
 'matched_route',
 'get_bowerstatic_path',
 'include',
 'root',
 'matchdict',
 'invoke_subrequest']

And test call only these:

['environ',
'registry',
'get_bowerstatic_path',
'include']

This clearly shows, that my approach is not sufficient yet, and I will encounter problems with views, which use other request capabilities. With the _make_request above, only minimal view goes through.

---

In other words, how to get the same request, which is in use in functional tests e.g. in webtest's TestApp, but instead of doing testapp.get(...), call the view with that request and make assertions on callable's returned result (not rendered as HTML)?

Answer 1

That essential code fragment is missing because of a temporary bug on RTD. An Issue has been logged.

Here's a link to the code that is not getting included. https://github.com/Pylons/pyramid/blob/master/docs/narr/MyProject/myproject/tests.py#L19-L43

[Edited]

Following are two examples of tests for Pyramid's alchemy scaffold. Each has its own style. Hopefully you find one of them useful to your scenario.

1. If you have Pyramid installed, you can create a sample project from the alchemy scaffold.

   $ $VENV/bin/pcreate -s alchemy tutorial
   

   The SQLAlchemy + URL Dispatch Wiki Tutorial goes into more detail and pre-requisites if that command does not work for you.

   The scaffold includes these tests.

2. We are working on a new branch that updates this scaffold, and is targeted for merge in Pyramid 1.7. Here is its tests.

   If you git checkout that branch, and run through the tutorial steps, you can see the full context.

[Edited]

Here's one more place to look: Pyramid's own integration tests.

As an aside, I'm having a hard time giving you an appropriate answer. For starters, I don't see where DummyRequest is getting called in your test's code. Can you give a complete reproducible example in your question?