CODEWITHSUNDEEP
django
JasonGenX
JasonGenX

Reputation: 5434

Django ORM Performance Issues

I am looping over 13,000 in-memory city names and generating queries to filter for something. I've encountered something I cannot explain...

When the loop has a single line:

cities = City.objects.filter(name__iexact=city)

performance is almost 800 items/second

When the loop measures the length of the returned set...

cities = City.objects.filter(name__iexact=city)
num_citles = len(cities)

performance drops to 8 items/second

I can't explain where the performance degradation occurrs. Obviously, I'm missing something... Why would counting the number of items on an in-memory array that is always between 0 and 3 items reducing performance by a factor of x100?

Upvotes: 1

Views: 233

Answers (1)

heemayl
heemayl

Reputation: 41987

Django querysets are lazy so QuerySet.filter does not actually evaluate the queryset i.e. run the queries in the database. When you run len function on it, it is evaluated and it will get all the items from database after running the filter only to get the count. Hence, the count is very slower.

You'll get a far better performance if you run COUNT on the database level:

num_cities = City.objects.filter(name__iexact=city).count()

Upvotes: 1

Related Questions