CODEWITHSUNDEEP
pythondjango
vanbastelaer
vanbastelaer

Reputation: 398

When Exactly does Django Run the Query?

While I understand the high-level ideas of Django's QuerySet and lazy execution, I don't see in Django (3.1.2) source code how query execution is triggered (i.e., database is being hit). For example, according to Django documentation, get() query seems to hit the database immediately. But looking at the source code (django.db.models.query):

    def get(self, *args, **kwargs):
        """
        Perform the query and return a single object matching the given
        keyword arguments.
        """
        clone = self._chain() if self.query.combinator else self.filter(*args, **kwargs)
        if self.query.can_filter() and not self.query.distinct_fields:
            clone = clone.order_by()
        limit = None
        if not clone.query.select_for_update or connections[clone.db].features.supports_select_for_update_with_limit:
            limit = MAX_GET_RESULTS
            clone.query.set_limits(high=limit)
        num = len(clone)
        if num == 1:
            return clone._result_cache[0]
        if not num:
            raise self.model.DoesNotExist(
                "%s matching query does not exist." %
                self.model._meta.object_name
            )
        raise self.model.MultipleObjectsReturned(
            'get() returned more than one %s -- it returned %s!' % (
                self.model._meta.object_name,
                num if not limit or num < limit else 'more than %s' % (limit - 1),
            )
        )

I don't see where clone (which is a QuerySet object) sends to the database engine its query (i.e., self._query that contains the actual Query object being parsed into SQL) and somehow caches the result in the _result_cache. In fact, when I tried to step through the code, at some point, self._result_cache "magically" got populated with query results. I never stepped into any SQL-related code, like results_iter() or execute_sql() in django.db.models.sql.compiler, which I assume must be called to interact with the database backend? My guess is that Django is running a different process/thread and PyCharm only steps through the main thread? If this is the case, can someone point out some relevant code? I can't seem to find any help from Google myself. Thanks!

EDIT: I know from looking at the code that _fetch_all() can trigger database hits because it connects to ModelIterable, which executes SQL commands and processes the results into Python objects, but how come I never stepped into this method ever? I guess the better question for this is, when is _fetch_all() ever called?

Upvotes: 2

Views: 2357

Answers (1)

Abdul Aziz Barkat
Abdul Aziz Barkat

Reputation: 21807

From the code you have added, see this line:

num = len(clone)

One can see that we are calling len on a queryset. According to When QuerySets are evaluated - Django docs:

len(). A QuerySet is evaluated when you call len() on it. This, as you might expect, returns the length of the result list.

Looking further in the source code - GitHub len would call _fetch_all() as you had correctly assumed is where the results are coming from:

def __len__(self):
    self._fetch_all()
    return len(self._result_cache)

The above referred documentation also notes when exactly a queryset is evaluated.

Upvotes: 3

Related Questions