CODEWITHSUNDEEP
linuxmemory-managementmemory-leaksheap-memory
LuisABOL
LuisABOL

Reputation: 2991

What is dirty private memory?

I'm developing an application on a 64-bit Linux system. As I could see, my app is eating too much dirty heap memory. Talking about the heap memory, what does "dirty" mean? What makes it arise and what can be done to prevent it to arise?

EDIT

I'd better explain what operations my application performs.

My application runs in two threads: the first thread sends jobs to a queue which are then executed in another thread. So, the first thread allocates pages to be queued and the second thread dequeues them, executes their jobs and free them. All these operations perform in a thread-safe fashion.

So I took a test on this thing, making it queue 100000000 jobs and execute them all. Until a particular instant, the memory usage grows. Then, when the queuing process finishes and only the dequeuing one remains, memory usage inexplicably doesn't decrease. Finally, when all the jobs are dequeued and executed, all that memory is freed. So, the memory leak seems to be happening in the dequeuing process, since when it finishes all the memory is freed, but I found nothing wrong in its code.

I know it would be better if I posted my code here, but it is too large. But, from what I added, does anybody have a guess about what might be causing this?

Upvotes: 7

Views: 8367

Answers (2)

user2491313
user2491313

Reputation:

Talking about the memory non-decrease even after freeing some chunks, you'd better use mmap in anonymous mode like this:

mmap(NULL, chunck_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

which maps no file descriptor and returns a pointer to a memory chunk which is immediately returned to the OS when you unmap it. However, mmap requires a system call, which is slower than malloc. So, you should use mmap for allocating large pages.

Upvotes: 3

maazza
maazza

Reputation: 7193

I found this

Inact_dirty: Dirty means "might need writing to disk or swap." Takes more work to free. Examples might be files that have not been written to yet. They aren't written to memory too soon in order to keep the I/O down. For instance, if you're writing logs, it might be better to wait until you have a complete log ready before sending it to disk.

Taken from here : http://www.redhat.com/advice/tips/meminfo.html

I guess it is quite like the dirty bit on I/O buffers ? By this I mean a bit that indicates that this buffer should be written on disk because it has been modified (on linux).

Here you have a similar question : https://unix.stackexchange.com/questions/33381/getting-information-about-a-process-memory-usage-from-proc-pid-smaps

Upvotes: 3

Related Questions