A technical question about dynamic memory allocation in C

Question

I'm studying dynamic memory allocation in C, and I want to ask a question - let us suppose we have a program that receives a text input from the user. We don't know how long that text will be, it could be short, it could also be extremely long, so we know that we have to allocate memory to store the text in a buffer. In cases in which we receive a very long text, is there a way to find out whether we have enough memory space to allocate more memory to the text? Is there a way to have an indication that there is no memory space left?

Answer 1

I made a test on linux with 8GB RAM. The overcommit has three main modes 0, 1 and 2 which are default, unlimited, and never:

Default:

$ echo 0 > /proc/sys/vm/overcommit_memory 
$ ./a.out
After loop: Cannot allocate memory
size 17179869184
size 400000000
log2(size)34.000000

This means 8.5 GB were successfuly allocated, just about the amount of physical RAM. I tried to tweak it, but without changing swap, which is only 4 GB.

Unlimited:

$ echo 1 > /proc/sys/vm/overcommit_memory 
$ ./a.out
After loop: Cannot allocate memory
size 140737488355328
size 800000000000
log2(size)47.000000

48 bits is virtual address size. 140 TB. Physical is only 39 bits (500 GB).

No overcommmit:

$ echo 2 > /proc/sys/vm/overcommit_memory 
$ ./a.out
After loop: Cannot allocate memory
size 2147483648
size 80000000
log2(size)31.000000

2 GB is just what free command declares as free. Available are 4.6 GB.

---

malloc() fails in the same way if the process's resources are restricted - so this ENOMEM does not really specify much. "Cannot allocate memory" (aka ENOMEM aka 12) just says "malloc failed, guess why" or rather "malloc failed, NO more MEMory for you now.".

---

Well here is a.out which allocates doubling sizes until error.

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <math.h>

int main() {
    size_t sz = 4096;
    void *p;
    while (!errno) {
        p = malloc(sz *= 2);
        free(p);
    }
    perror("After loop");
    printf("size %ld\n", sz);
    printf("size %lx\n", sz);
    printf("log2(size)%f\n", log2((double)sz));
}

---

But I don't think this kind of probing is very useful/

Buffer

we have to allocate memory to store the text in a buffer

But not the whole text at once. With a real buffer (not just an allocated memory as destination) you could read portions of the input and store them away (out of memory onto disk).

Only disadvantage is: if I cannot use a partial input, then all the buffered copying and saving is wasted.

I really wonder what happens if I type and type fast for a couple of billion years -- without a newline.

---

We can allocate much more than we have as RAM, but we only need a fraction of that RAM: the buffer. But Lundin's answer shows it is much easier (typical) to rely on newlines and maximum length.

getline(3)

This gnu/posix function has the malloc/realloc built in. The paramters are a bit complicated, because a new pointer and size can be returned by reference. And return value of -1 can also mean ENOMEM, not end-of-file.

fgets() is the line-truncating version.

fread() is newline independant, with fixed size. (But you asked about text input - long lines or long overall text, or both?)

Good Q, good As, good comments about "live input":

getline how to limit amount of input as you can with fgets

Answer 2

There is no standard library function that tells you how much memory is available for use.

The best you can do within the bounds of the standard library is to attempt the allocation using malloc, calloc, or realloc and check the return value - it it’s NULL, then the allocation operation failed.

There may be system-specific routines that can provide that information, but I don’t know of any off the top of my head.

Answer 3

We don't know how long that text will be

Sure we do, we always set a maximum limit. Because all user input needs to be sanitised anyway - so we always require a maximum limit on every single user input. If you don't do this, it likely means that your program is broken since it's vulnerable to buffer overruns.

Typically you'll read each line of user input into a local array allocated on the stack. Then you can check if it is valid (are strings null terminated etc) before allocating dynamic memory and then copy it over there.

By checking the return value of malloc etc you'll see if there was enough memory left or not.

Answer 4

In C we typically use malloc, calloc and realloc for allocation of dynamic memory. As it has been pointed out in both answers and comments these functions return a NULL pointer in case of failure. So typical C code would be:

SomeType *p = malloc(size_required);
if (p == NULL)
{
    // ups... malloc failed... add error handling
}
else
{
    // great... p now points to allocated memory that we can use
}

I like to add that on (at least) Linux systems, the return value from malloc (and friends) is not really an out-of-memory indicator.

If the return value is NULL, we know the call failed and that we didn't get any memory that we can use.

But even if the return value is non-NULL, there is no guarantee that the memory really is available.

From https://man7.org/linux/man-pages/man3/free.3.html :

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.

Answer 5

ANSI C has no standard functions to get the size of available free RAM. You may use platform-specific solutions.

C - Check currently available free RAM?

Answer 6

You can use malloc() function if it returned NULL that means there no enough mem space but if it returned address of the mem it means there are mem space available example:

void* loc = malloc(sizeof(string));