CODEWITHSUNDEEP
pythondjangooptimizationpagination
Eduardo
Eduardo

Reputation: 258

Django pagination. What if I have 1 million of rows?

The official Django Documentation gives us something like this:

from django.core.paginator import Paginator
my_list = MyModel.objects.all()
p = Paginator(my_list, 10)

But. What if I have to paginate 1 million of rows? It's not so efficient to load the 1 million rows with MyModel.objects.all() every time I want to view a single paginated page.

Is there a more efficient way to do this without the need of call objects.all() to make a simple pagination?

Upvotes: 1

Views: 1915

Answers (1)

Ian Clelland
Ian Clelland

Reputation: 44112

MyModel.objects.all() doesn't actually load all of the objects. It could potentially load all of them, but until you actually perform an action that requires it to be evaluated, it won't do anything.

The Paginator will almost certainly add some limits on that query set. For example, using array-slicing notation, it can create a new object, like this

my_list = MyModel.objects.all()
smaller_list = my_list[100:200]

That will create a different query set, which will only request 100 items from the database. Or calling .count() on the original query set, which will just instruct the database to return the number of rows in the table.

You would have to do something that requires all of the objects to be retrieved, like calling

list(my_list)

to get 1000000 rows to be transferred from the database to Python.

Upvotes: 5

Related Questions