CODEWITHSUNDEEP
python-3.xmemory-profiling
derek
derek

Reputation: 10217

python: will memory_profiler affect runtime?

I am evaluating the tools that profile my python program. One of the interesting tools here is memory_profiler. Before moving forward, just want to know whethermemory_profiler affects runtime. The reason I am asking this question is that memory_profiler will output a lot of memory usages. So I am suspecting it might affect runtime.

Thanks

Derek

Upvotes: 0

Views: 1307

Answers (1)

Fabian Pedregosa
Fabian Pedregosa

Reputation: 6539

It depends how you are using memory_profiler. This can be used in two different ways:

  • To get memory usage line-by-line (run with python -m memory_profiler my_script.py). This needs to get memory information (from the OS) for every line executed within the profiled function. How this affects run-time depends on the amount of lines in the function: if it has a lot of lines with fast execution times, it might suppose a significant overhead. On the other hand, if the function to profile has few lines, and each lines has a significant computing time, then the overhead will be negligible.

  • To get memory as a function of time (run with mprof run my_script.py and plot with mprof plot). In this case the function that collects the memory usage is in a different process as the one that runs your script, hence the overhead is minimal (unless you are using all CPUs).

Upvotes: 1

Related Questions