CODEWITHSUNDEEP
c++clinux
HAL9000
HAL9000

Reputation: 3751

Why isn't malloc filling up memory?

I have the following code:

#include <iostream>
#include <stdlib.h>
#include <stdio.h>

int main() {
    int data = 0;
    char *byte = (char *)malloc(sizeof(char)*1000000000);
byte[999999999] = 'a';
printf("%c",byte[999999999]);
    scanf("%d",&data);
    return 0;
}

Looking at memory before the program starts and before the scanf I would expect the memory to increase of 1 GB. Why isn't this happening?

Edit: I added

byte[999999999] = 'a';
printf("%c",byte[999999999]);

The program outputs a.

Upvotes: 0

Views: 411

Answers (3)

Mike Seymour
Mike Seymour

Reputation: 254431

By default, Linux allocates physical memory lazily, the first time it's accessed. Your call to malloc will allocate a large block of virtual memory, but there are not yet any pages of physical memory mapped to it. The first access to an unmapped page will cause a fault, which the kernel will handle by allocating and mapping one or more pages of physical memory.

To allocate all the physical memory, you'll have to access at least one byte on every page; or you could bypass malloc and go straight to the operating system with something like

mmap(0, 1000000000, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE, -1, 0);

Note the use of MAP_POPULATE to populate the page tables, i.e. to allocate physical memory, immediately. According to the manpage, this only works on fairly recent versions of Linux.

Upvotes: 8

Juri Robl
Juri Robl

Reputation: 5736

Most (all?) Linux systems overcommit memory. Which means a malloc never fails, even if you reserve ridiculous amounts of memory if you use a specific commit strategy. It can fail on some occasions if you reserve way too much memory, for example more than the virtual memory.

You don't get real memory adresses, only a pointer which is promised to point to enough memory to use for you. The system allocates the memory for you if you try to use it, so you have to write to it to signal the system that you really want it.

That system is used because many programs don't need the memory they reserve or need it at another time.

After your Edit you access only one page, so you only get one page reserved. You need to access every page to get all the memory.

Upvotes: 5

Sunil Bojanapally
Sunil Bojanapally

Reputation: 12658

malloc didn't gave fetch you all the 1000000000 when you ask. Its just 1st page you get until you start access it (read/write), you'll get assigned rest of it.

Linux, malloc requests will expand virtual address space immediately but doesn't assign physical memory pages until you access it.

Upvotes: 0

Related Questions