CODEWITHSUNDEEP
django
xpanta
xpanta

Reputation: 8418

django: Should I try to decrease queries and increase time in ms?

in my site, the main page (where users spend most of their time) has ~140 queries (70 of them come from django-avatar module -- will have to see if I can find an alternative some day). However it needs 5 seconds to load (metrics come from django-toolbar).

I can increase number of queries to 170 and decrease time it takes to load down to 2.5s (!).

For that I have to simplify a complex query that uses annotate() and create a dictionary by making an iteration of smaller queries. I noticed that although number of queries increases, django likes it more as I take away the load from that complex query by removing the annotation.

What should I do?

PS: apart from MySql I also use memcached

Upvotes: 0

Views: 141

Answers (1)

Chris Pratt
Chris Pratt

Reputation: 239430

First, django-debug-toolbar provides a subjective measure of performance for quick checking yourself. If you see that your queries spike or the request suddenly takes much longer than it did before, then that let's you know you need to dig in and see if you can optimize it some. However, it is not suited for generalized "how responsive is my website" type tests; django-debug-toolbar, itself introduces a number of inefficiencies to give you the data it does. Your code will inevitable perform much better with the toolbar disabled than it will enabled. If you truly want to gauge your site, you need to employ a profiling backend to hit your site hard under optimal conditions and give you stats about where bottlenecks occur.

That said, the time stats you're getting aren't just query dependent; they also depends on file I/O, template rendering, etc. If you're employing memcached in development, there's inevitably some caching taking place between requests, even if it's just the template being cached. Not having to re-parse the template and create a response object takes a lot of time off the request. So, it may seem like the 170 queries is faster, but that may be just because there's already some caching. If you turned the cache off, the time might go up significantly. If you're in development and optimizing query load, you should disable caching entirely by setting it to DummyCache.

Finally, in development, the actual time to transfer the request is less of an issue because it's going local-to-local, which is nearly instantaneous. In real world scenarios, it took take hundreds of milliseconds for the client to send a request to your server. If you're database server is not on the same server as your webserver, it takes time to send requests back and forth from the web app to the database, and finally it takes time to send the response back to the client. The actual transport time, usually ends up creating more of a bottleneck than the query time in practice, so generally fewer, more complex queries is always preferable to more, small queries.

Basically, there's just too many variables testing this stuff in development. Something like django-debug-toolbar is a good guide, but it's not comprehensive and it's not fool-proof. Put your code out on a production-like machine and hit it hard -- that's the only way to know for sure.

Upvotes: 2

Related Questions