CODEWITHSUNDEEP
phpcurlmemorycron
Ryan
Ryan

Reputation: 162

memory_get_peak_usage(true) PHP and Virtual Memory Size resource usage are very different

I have a PHP script which runs many http requests via curl - I use a variation on Rolling Curl (curl_multi) so that the requests can be run simultaneously. The script runs every few minutes with cron.

It is on a VPS and I received some 'Excessive resource usage' warnings from lfd (ConfigServer Security & Firewall) because the resource usage of the script went over the threshold of 512MB.

An example notification is:

Resource: Virtual Memory Size

Exceeded: 582 > 512 (MB)

Executable: /path/to/php-cgi

Command Line: /path/to/myscript.php

So I upped the threshold to 800MB and recorded the memory usage of the script using memory_get_peak_usage(true) every time the script runs.

However, the results from memory_get_peak_usage(true) are consistently 2MB... which is nowhere near the Virtual Memory usage as seen in the warning.

Note - only one instance of the script runs as multiple instances are prevented using flock.

So what am I missing?

Update - virtual memory usage also greater than php.ini memory_limit

After upping the threshold to 800MB I still get occasional notifications from lfd. I also checked the php.ini settings and memory_limit is set to 256MB - in theory the script wouldn't run if it was using more than this. From this I am guessing that either:

a) It is not PHP that is using the memory (could it be MySQL or CURL - is the memory used by these included in the value frommemory_get_peak_usage(true)?)

b) I'm not getting an accurate figure from lfd

Second Update - memory used by MySQL is not included in memory_get_peak_usage(true)

I suspect this is the issue - however I'm not sure what exactly CSF includes in resource figure. I will look into making the MySQL requests more efficient and see how that effects things.

Upvotes: 0

Views: 738

Answers (1)

IMSoP
IMSoP

Reputation: 97783

PHP's memory_get_usage family of functions tracks the state of PHP's internal memory manager, which is responsible for all memory directly used by things like PHP variables. This is also what is limited by the memory_limit setting - after the limit is reached, the internal memory manager will refuse to claim more RAM from the OS.

However, not all the RAM used by your process is allocated via that memory manager. For instance, third party libraries like CURL or the MySQL connection library will have memory allocation completely independent of PHP, because they are effectively separate programs being executed by the PHP engine. Another example is that opening a file might cause it to be mapped into memory by the OS kernel, even if you never read its contents into a PHP variable.

This talk by Julien Pauli at PHP UK a few years ago goes into more details on the different types of memory usage, and how to measure them.

Upvotes: 1

Related Questions