CODEWITHSUNDEEP
pythoncelerypytestmemory-profiling
Shravan
Shravan

Reputation: 33

How can we profile memory usage of each Celery task?

I run tests using pytest which in turn runs a couple of celery tasks. But, when I use 

mprof run --include-children py.test test.py

I am not able to track the memory consumed by each celery task that is running inside a celery worker. All I could see 50 MB of RAM being consumed by the py.test process (which I am able see in the mprof plot) whereas the celery worker consumes around 600-700 MB of RAM which does not get reflected in the mprof plot. How can I plot the memory consumption of each Celery task triggered by the py.test?

P.S I use memory profiler package to profile the memory consumption of a python script.

Upvotes: 2

Views: 2258

Answers (1)

Forethinker
Forethinker

Reputation: 3748

What about --multiprocess flag? https://github.com/pythonprofilers/memory_profiler#tracking-forked-child-processes

Upvotes: 0

Related Questions