CODEWITHSUNDEEP
djangoprofiling
sharjeel
sharjeel

Reputation: 6035

Is there a way to profile my Django application?

My Django application has become painfully slow on the production. Probably it is due to some complex or unindexed queries.

Is there any django-ish way to profile my application?

Upvotes: 119

Views: 61610

Answers (9)

Tony S Yu
Tony S Yu

Reputation: 3183

I needed to profile a Django app recently and tried many of these suggestions. I ended up using pyinstrument instead, which can be added to a Django app using a single update to the middleware list and provides a stack-based view of the timings.

Quick summary of my experience with some other tools:

  • Django Debug Toolbar is great if you the issue is due to SQL queries and works well in combination with pyinstrument
  • django-silk works well, but requires adding a context manager or decorator to each part of the stack where you want sub-request timings. It also provides an easy way to access cProfile timings and automatically displays ajax timings, both of which can be really helpful.
  • djdt-flamegraph looked promising, but the page never actually rendered on my system.

Compared to the other tools I tried, pyinstrument was dramatically easier to install and to use.

Upvotes: 13

ThePhi
ThePhi

Reputation: 2641

You can use line_profiler.

It allows to display a line-by-line analysis of your code with the time alongside of each line (When a line is hit several times, the time is summed up also).

enter image description here

It's used with not-Django python code but there's a little trick to use it on Django in fact: https://stackoverflow.com/a/68163807/1937033

Upvotes: 2

Waqas Ali
Waqas Ali

Reputation: 1529

I am using silk for live profiling and inspection of Django application. This is a great tool. You can have a look on it.

https://github.com/jazzband/django-silk

Upvotes: 1

Michael
Michael

Reputation: 2298

Shameless plug here, but I recently made https://github.com/django-silk/silk for this purpose. It's somewhat similar to django toolbar but with history, code profiling and more fine grained control over everything.

Upvotes: 18

Tomasz Zieliński
Tomasz Zieliński

Reputation: 16377

Just type "django-profiling" on google, you'll get these links (and more):

http://code.djangoproject.com/wiki/ProfilingDjango

http://code.google.com/p/django-profiling/

http://www.rkblog.rk.edu.pl/w/p/django-profiling-hotshot-and-kcachegrind/

Personally I'm using the middleware approach - i.e. each user can toggle a "profiling" flag stored in a session, and if my profiling middleware notices that a flag has been set, it uses Python's hotshot module like this:

def process_view(self, request, view_func, view_args, view_kwargs):

     # setup things here, along with: settings.DEBUG=True 
     # to get a SQL dump in connection.queries

     profiler = hotshot.Profile(fname)
     response = profiler.runcall(view_func, request, *view_args, **view_kwargs)
     profiler.close()

     # process results

     return response

EDIT: For profiling SQL queries http://github.com/robhudson/django-debug-toolbar mentioned by Konstantin is a nice thing - but if your queries are really slow (probably because there are hundreds or thousands of them), then you'll be waiting insane amount of time until it gets loaded into a browser - and then it'll be hard to browse due to slowness. Also, django-debug-toolbar is by design unable to give useful insight into the internals of AJAX requests.

EDIT2: django-extensions has a great profiling command built in:

https://github.com/django-extensions/django-extensions/blob/master/docs/runprofileserver.rst

Just do this and voila:

$ mkdir /tmp/my-profile-data
$ ./manage.py runprofileserver --kcachegrind --prof-path=/tmp/my-profile-data

Upvotes: 31

Silver Light
Silver Light

Reputation: 46002

Try the Django Debug Toolbar. It will show you what queries are executed on each page and how much time they take. It's a really useful, powerful and easy to use tool.

Also, read recommendations about Django performance in Database access optimization from the documentation.

And Django performance tips by Jacob Kaplan-Moss.

Upvotes: 99

Joshua Pokotilow
Joshua Pokotilow

Reputation: 1233

For all you KCacheGrind fans, I find it's very easy to use the shell in tandem with Django's fantastic test Client for generating profile logs on-the-fly, especially in production. I've used this technique now on several occasions because it has a light touch — no pesky middleware or third-party Django applications are required!

For example, to profile a particular view that seems to be running slow, you could crack open the shell and type this code:

from django.test import Client
import hotshot

c = Client()
profiler = hotshot.Profile("yourprofile.prof")  # saves a logfile to your pwd
profiler.runcall(c.get, "/pattern/matching/your/view/")
profiler.close()

To visualize the resulting log, I've used hotshot2cachegrind:

But there are other options as well:

Upvotes: 6

atereshkin
atereshkin

Reputation: 706

For profiling data access (which is where the bottleneck is most of the time) check out django-live-profiler. Unlike Django Debug Toolbar it collects data across all requests simultaneously and you can run it in production without too much performance overhead or exposing your app internals.

Check out this screenshot

Upvotes: 16

Evgeny Zislis
Evgeny Zislis

Reputation: 6967

When the views are not HTML, for example JSON, use simple middleware methods for profiling.

Here are a couple examples:

https://gist.github.com/1229685 - capture all sql calls went into the view

https://gist.github.com/1229681 - profile all method calls used to create the view

Upvotes: 4

Related Questions