CODEWITHSUNDEEP
pythonsqldjangosqlite
Peksio
Peksio

Reputation: 585

Is Django supposed to get so slow with around 250 000 model's objects?

I am doing some small project but with around 250 000 model's objects. I have my car model and I am importing data from external json. I talked with some people and they told me that 250 000 objects in django is not a big amount and it won't have much impact on performance.

Loading up to models this data from json is takin around 5 - 10 min and I don't mind that. But, right now If I make any changes to the website (locally) while running runserver I have to wait around 1 minute (on macbook pro 2020 (not M1)). And it is pretty annoying.

I ran command python -v manage.py check and apparently django.db.models.sql.compiler takes this whole time.

I am wondering, if my approach with models is wrong, or it is normal with SQL Light?

Or maybe I could safely turn off this check? Or I got wrong informations and workflow with this many model's object is totally wrong?

cProfile sorted by tottime:

236646 function calls (229052 primitive calls) in 50.709 seconds
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        9   50.280    5.587   50.280    5.587 {built-in method posix.waitpid}
    6    0.067    0.011    0.067    0.011 {built-in method time.sleep}
  466    0.041    0.000    0.041    0.000 {built-in method marshal.loads}
   32    0.031    0.001    0.032    0.001 {built-in method _imp.create_dynamic}

1713/1614 0.026 0.000 0.099 0.000 {built-in method builtins.build_class} 2235 0.022 0.000 0.022 0.000 {built-in method posix.stat} 466 0.020 0.000 0.020 0.000 {built-in method io.open_code}

cProfile sorted by cumtime:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
495/1    0.003    0.000   63.643   63.643 {built-in method builtins.exec}
    1    0.000    0.000   63.643   63.643 manage.py:2(<module>)
    1    0.000    0.000   63.643   63.643 manage.py:7(main)
    1    0.000    0.000   63.549   63.549 __init__.py:398(execute_from_command_line)
    1    0.000    0.000   63.549   63.549 __init__.py:321(execute)
    2    0.000    0.000   63.304   31.652 subprocess.py:448(run)
    1    0.000    0.000   63.302   63.302 base.py:313(run_from_argv)
    1    0.000    0.000   63.300   63.300 runserver.py:55(execute)
    1    0.000    0.000   63.300   63.300 base.py:349(execute)
    1    0.000    0.000   63.300   63.300 runserver.py:67(handle)
    1    0.000    0.000   63.300   63.300 runserver.py:98(run)
    1    0.000    0.000   63.300   63.300 autoreload.py:608(run_with_reloader)
    1    0.000    0.000   63.300   63.300 autoreload.py:240(restart_with_reloader)
    2    0.000    0.000   63.300   31.650 subprocess.py:980(communicate)
    3    0.000    0.000   63.297   21.099 subprocess.py:1074(wait)
    4    0.000    0.000   63.297   15.824 subprocess.py:1772(_wait)
    9    0.000    0.000   63.227    7.025 subprocess.py:1759(_try_wait)
    9   63.227    7.025   63.227    7.025 {built-in method posix.waitpid}
   65    0.001    0.000    0.678    0.010 __init__.py:1(<module>)

Upvotes: 3

Views: 1086

Answers (1)

Le Minaw
Le Minaw

Reputation: 885

A conveniant solution for profiling is django-debug-toolbar. It shows DB queries, and it can even profile your Python code.

The profiler is disabled by default though, you have to enable it -see docs.

Upvotes: 1

Related Questions