CODEWITHSUNDEEP
pythondjangodjango-querysetpython-multithreadingdjango-testing
Gaurav Toshniwal
Gaurav Toshniwal

Reputation: 3722

Django unittesting: model.all() queryset is empty in a method running in separate thread

I have a method running in a separate thread, that does some contacts matching.

I'm writing tests to check if the contacts have been synced. The test case goes something like this:

class ContactSyncTestCase(TestCase):
    fixtures = ['fix.json']

    def setUp(self):
        # get a few contacts that exist in the database to be sent for matching
        self.few_contacts = CompanyContact.objects.all().order_by('?')[:5].values_list('contact_number',flat=True)

    def test_submit_contacts(self):

        # Pick up a random user
        user = User.objects.all().order_by('?')[0]

        # Get API Key for that user
        key = ApiKey.objects.get(user=user).key

        # The url that submits contacts for matching, returns a matching key immediately and starts a separate thread to sync contacts
        sync_request_url = '/sync_contacts/?username=%s&api_key=%s'%(user.username,key)
        sync_request_response = self.client.post(path=sync_request_url,
                                    data=json.dumps({"contacts":','.join(self.few_contacts)}),
                                    content_type="application/json")

        # Key that is used to fetch the status of contacts syncing and returns a json if contacts are matched
        key = sync_request_response.content

        # At this point, the other thread is doing the task of syncing inside the method mentioned next

        # Hence I put the test on pause so that it does not fail and exit
        try:
            while True:
                time.sleep(100)
        except KeyboardInterrupt:
            pass

The async method that matches the numbers starts something like this:

def match_numbers(key, contacts, user):
    # Get all contacts stored in the system
    """

    :param key:
    :param contacts:
    :param user:
    """
    import pdb;pdb.set_trace()
    system_contacts = CompanyContact.objects.all().values_list('contact_number', flat=True)

Now the weird issue here is that:

CompanyContact.objects.all().values_list('contact_number', flat=True)

Returns an empty queryset while testing. However, during runtime it works fine.

For that matter, any query (including User model) returns an empty queryset.

Any ideas why?

EDIT:

Turns out that inheriting from TrasactionTestCase solves this issue. I still have my doubts and I dug up more for the same.

My database's default transaction level is Repeatable Read.

Reading from this post,

REPEATABLE READ (default) : ensure that is a transaction issues the same SELECT twice, it gets the same result both times, regardless of committed or uncommitted changes made by other transactions. In other words, it gets a consistent result from different executions of the same query. In some database systems, REPEATABLE READ isolation level allows phantoms, such that if another transaction inserts new rows,in the interval between the SELECT statements, the second SELECT will see them. This is not true for InnoDB; phantoms do not occur for the REPEATABLE READ level.

Summary of this: I should still have got the existing records, which I didn't.

Upvotes: 2

Views: 1389

Answers (1)

erthalion
erthalion

Reputation: 3244

You can find a good explanation in the django LiveServerTestCase

class LiveServerTestCase(TransactionTestCase):
    """
    ...
    Note that it inherits from TransactionTestCase instead of TestCase because
    the threads do not share the same transactions (unless if using in-memory
    sqlite) and each thread needs to commit all their transactions so that the
    other thread can see the changes.
    """

Upvotes: 3

Related Questions