CODEWITHSUNDEEP
memorymemory-managementlinux-kernelubuntu-18.04
Amir
Amir

Reputation: 541

Limit available memory to a program

I am doing performance analysis on a multi-threaded program that uses all the memory that is available in the system. My OS is Ubuntu 18.04. I'm trying to limit the available memory to e.g 32GB even though my server may have 128GB of memory available. Haven't been able to find a reliable solution. Seems like ulimit is not exactly doing what I'm looking for. I can also clog up memory by another process (e.g a controllable process that will consume 64GB of RAM). But even for that purpose I'm not sure how to reliably clog up the memory.

Would appreciate your thoughts.

Upvotes: 4

Views: 2179

Answers (2)

osgx
osgx

Reputation: 94205

cgroups is the feature or modern linux kernels which allow you to limit resources like memory for group of processes (or for single process with threads). More about cgroups: https://en.wikipedia.org/wiki/Cgroups https://man7.org/linux/man-pages/man7/cgroups.7.html

The cgroups feature should be already enabled in your ubuntu 18.04 kernels. There are some descriptions how to use cgroups to limit memory:

# Create a group for memory named “limited_group_1”
cgcreate -g "memory:limited_group_1" -t USERNAME:GROUPNAME

# Specify memory limit to 1G for this group
cgset -r memory.limit_in_bytes=1G "limited_group_1"

# Launch the application in this group
cgexec -g "memory:limited_group_1" ./YOUR_APPLICATION

# If needed, we can remove the group
cgdelete "memory:limited_group_1"

https://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process/279175#279175 was also mentioned in https://dev.to/vga/how-to-see-and-limit-memory-consumption-of-an-application-5bfl

PS: Default memory allocators in older glibc versions (malloc, new) has awful behavior for freed regions: they are not returned back without periodic malloc_trim() library calls. You should try to link your application with libjemalloc or libtcmalloc which will replace malloc implementation of glibc with some code better in memory returning.

Upvotes: 2

lunix
lunix

Reputation: 323

ulimit will limit process memory, not system memory.

Add mem=32G to the kernel command line at boot if you want to accurately simulate a smaller machine.

Upvotes: -1

Related Questions