CODEWITHSUNDEEP
pythondjangodjango-modelsdjango-rest-frameworkdjango-orm
Seenu S
Seenu S

Reputation: 3481

Multi db and unmanged models - Test case are failing

I have mutlidb setup with unmanaged (read-only) models. There's no migrations for these models. I was trying to test the functionality of the view.py. In sqlite3 database these schema of the test table is not creating a causing an issue in failing the test case. In the view.py I have imported the unamanaged (read-only) model is failing.

I have followed the link for test Testing against unmanaged models

Mentioned the Multi-db setup

test_runner.py

from django.test.runner import DiscoverRunner


class DisableMigrations(object):
    def __contains__(self, item):
        return True

    def __getitem__(self, item):
        return None


class UnManagedModelTestRunner(DiscoverRunner):

    def setup_test_environment(self, *args, **kwargs):
        from django.apps import apps
        # app_name = apps.get_app_config('core_esp')
        self.unmanaged_models = [m for m in apps.get_models() if not m._meta.managed and m._meta.app_label is 'core_esp']
        for m in self.unmanaged_models:
            m._meta.managed = True
        super(UnManagedModelTestRunner, self).setup_test_environment(*args, **kwargs)

    def teardown_test_environment(self, *args, **kwargs):
        super(UnManagedModelTestRunner, self).teardown_test_environment(*args, **kwargs)
        # reset un managed models
        for m in self.unmanaged_models:
            m._meta.managed = False

settings.py

 if 'test' in sys.argv or 'test_coverage' in sys.argv: 
    DATABASES = {                                                       
        'default': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': 'db.sqlite3',
        },
        'test_db': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': 'db.sqlite3',
        }
    }

INSTALLED_APPS = ['test']
# Skip the migrations by setting "MIGRATION_MODULES"
# to the DisableMigrations class defined above
#
MIGRATION_MODULES = DisableMigrations()

# Set Django's test runner to the custom class defined above
TEST_RUNNER = 'apps.settings.test_runner.UnManagedModelTestRunner'
DATABASE_ROUTERS = ('apps.tests.test_dbrouter.TestApiRouter', )

models.py

from django.db import models

class TestModelA(models.Model):
    testid = models.CharField(max_length=200)
    class Meta:
        managed = False
        db_table = 'TestD'

class TestModelB(models.Model):
    testid = models.CharField(max_length=200)
    class Meta:
        managed = False
        db_table = 'test_model_b'
        app_label = 'application_b'

test.py

class MyTestCase(TestCase):

    def test_my_function(self):
       # view is called
       # serializer called with read only model
       pass

Error

Traceback (most recent call last):                                                                                 
File "C:\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
return self.cursor.execute(sql, params)                                                                                                                                   
File "C:\lib\site-packages\django\db\backends\sqlite3\base.py", line 383, in execute return Database.Cursor.execute(self, query, params)                                                                                                            
sqlite3.OperationalError: no such table: TestD 
The above exception was the direct cause of the following exception:                                                                                                                                              Traceback (most recent call last): 
File "manage.py", line 21, in <module> main() 
File "manage.py", line 17, in main execute_from_command_line(sys.argv)
File "C:\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line utility.execute()
File "C:\lib\site-packages\django\core\management\__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\lib\site-packages\django\core\management\commands\test.py", line 23, in run_from_argv
super().run_from_argv(argv)
File "C:\lib\site-packages\django\core\management\base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\lib\site-packages\django\core\management\base.py", line execute
output = self.handle(*args, **options)
File "C:\lib\site-packages\django\core\management\commands\test.py", line 53, in handle failures = test_runner.run_tests(test_labels)
File "C:\lib\site-packages\django\test\runner.py", line 627, in run_tests  
suite = self.build_suite(test_labels, extra_tests)
File "C:\lib\site-packages\django\test\runner.py", line 488, in build_suites   
tests = self.test_loader.loadTestsFromName(label)                                                                                                    
File "c:\program files\python37\Lib\unittest\loader.py", line 154, in loadTestsFromName 
module = __import__(module_name)                                                                                                                                           
File "C:\tests\tests.py", line 5, in <module>       
from .views import (                                                                                                                             
File "C:\views.py", line 6, in <module>    
from .serializers import (                                                                                                                       
File "C:\serializers.py", line 9, in <module>   
class TestSerializer(serializers.Serializer):                                                                                                                           
File "C:\serializers.py", line 13, in TestSerializer     
required=False, style={'base_template': 'select_multiple.html'}                                                                                                           
File "C:\lib\site-packages\rest_framework\fields.py", line 1476, in __init__                                                                      
super(MultipleChoiceField, self).__init__(*args, **kwargs)                                                                                                              
File "C:\lib\site-packages\rest_framework\fields.py", line 1417, in __init__                                                                         
self.choices = choices                                                                                                                        
File "C:\lib\site-packages\rest_framework\fields.py", line 1453, in _set_choices                                                                                                                                                                                                                         
for row in compiler.results_iter(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size):  
File "C:\lib\site-packages\django\db\models\sql\compiler.py", line 1052, in results_iter                                          
results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)                                       
File "C:\lib\site-packages\django\db\models\sql\compiler.py", line 1100, in execute_sql                                                             
cursor.execute(sql, params)                                                                                
File "C:\lib\site-packages\django\db\backends\utils.py", line 67, in execute 
return self._execute_with_wrappers(sql, params, many=False, 
executor=self._execute)
File "C:\lib\site-packages\django\db\backends\utils.py", line 76, in 
_execute_with_wrappers                                               
return executor(sql, params, many, context)                                                                                                                               
File "C:\lib\site-packages\django\db\backends\utils.py", line 84, in _execute                                                   
return self.cursor.execute(sql, params)                                                                                                                                   
File "C:\lib\site-packages\django\db\utils.py", line 89, in __exit__  
raise dj_exc_value.with_traceback(traceback) from exc_value           
File "C:\lib\site-packages\django\db\backends\utils.py", line 84, in _execute                                                                  
return self.cursor.execute(sql, params)                                                                                                                                   
File "C:\lib\site-packages\django\db\backends\sqlite3\base.py", line 383, in execute                                                                
return Database.Cursor.execute(self, query, params)                                                                                                                     
django.db.utils.OperationalError: no such table: TESTD

Upvotes: 2

Views: 759

Answers (2)

Michel Samia
Michel Samia

Reputation: 4477

I found a good solution working in current django 2.2. Works perfectly with pytest-django, but will probably work even without pytest.

To make it work, you need to:

  1. split your project into two django apps. One will contain only the unmanaged models and the other will contain the rest. Don't specify managed = False in any of these two apps

  2. in settings.py in DATABASES you will have two databases, one is default and one is your external DB

  3. In the same folder where you have settings, create routers.py and implement a router that will route based on app label to given DB (cca 45 lines including checks that you will never ever write to these external DBs). Then add this router to settings.py's ROUTERS list.

When running tests with pytest, it really creates all tables for apps that don't have migrations folder

Upvotes: 1

opalczynski
opalczynski

Reputation: 1647

I can show you how I am solving this issue;

So I've created a signal, which more or less looks like this:

def create_test_models(**kwargs):
    if "test" in sys.argv:
        Organization = apps.get_model("organizations.Organization")
        ...  # other models here too
        # as we do not manage it - we need to create it manually;
        with connection.schema_editor() as schema_editor:
            sid = transaction.savepoint()
            try:
                schema_editor.create_model(Organization)
                ... # different models here if needed
                transaction.savepoint_commit(sid)
            except ProgrammingError:  # tables already exists;
                transaction.savepoint_rollback(sid)

This signal is connected in the config as pre_migrate signal:

class OrganizationsConfig(AppConfig):
    name = "engine.organizations"

    def ready(self):
        # run after each migration; so each deploy, but this method can handle the
        # incremental updates.
        pre_migrate.connect(create_test_models, sender=self)

It is maybe not super solution - but it is working and you have models created during tests and you can play with them or create test data etc.

Hope that this will help you to move forward.

Upvotes: 1

Related Questions