CODEWITHSUNDEEP
pythonmemory
bdhar
bdhar

Reputation: 22983

Limit Python process memory usage

I have a system with 16GB of memory. I run a python script for some data mining application and the process takes up the entire 16GB. I want to limit the python process to take up only a limited amount of memory.

Is it possible to do this? If yes, how?

Update:

The application retrieves hotel review data from a huge database and tries to build graphs among hotels and users for some analysis. The data structure to hold the data goes beyond 16GB.

Upvotes: 4

Views: 3294

Answers (1)

Jay Sullivan
Jay Sullivan

Reputation: 18279

You have an application which has a data structure that appears to require 16GB+ of RAM, right? If so, then limiting its RAM will cause the application to fail, will it not?

Is there a reason all of this data needs to be in RAM all at once? Likely, the application could be redesigned to take up less than 16GB of RAM at a time. For example, searching all file contents of all files on a computer does not require having all files open in RAM all at once, but only one at a time.

My point is that "limiting the RAM usage" isn't likely to be solvable any way other than redesigning the application.

Upvotes: 2

Related Questions