CODEWITHSUNDEEP
pythonmemoryout-of-memory
Henrik Leijon
Henrik Leijon

Reputation: 1787

psutil gives other result than resource

I want to check memory consumption of my python code and have therefore added the following rows in the code:

import resource
print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)

As an alternative I have also tried this:

import psutil
process = psutil.Process(os.getpid())
print(process.memory_info().rss)  # in bytes

However, I get different results, as for example 866 480 from resource and 730 689 536 from psutil. Of course as you can see, in the first case it is kilobytes and in the second case bytes, but it is a difference also in addition to that.

Reading the documentation, I still don't understand causes the difference, so input would be valuable.

Upvotes: 3

Views: 803

Answers (1)

Arigion
Arigion

Reputation: 3548

TLDR: resource.getrusage sometimes misses that Python already removed objects from memory

There was a bug in memory profiler (which was using resource.getrusage at that time). In this blog post the different methods for memory measurements are described. I cite:

"this approach [resource.getrusage] is several times faster than the one based in psutil [...] The problem with this approach is that it seems to report results that are slightly different in some cases. Notably it seems to differ when objects have been recently liberated from the python interpreter. In the following example, orphaned arrays are liberated by the python interpreter, which is correctly seen by psutil but not by resource..."

Upvotes: 2

Related Questions