"invalid next size" exception while reallocing memory

Question

P.S.: I have almost all questions for "invalid next size" but they are not helping me because I do not have another piece of code which malloc or realloc, so that is ruled out. Also, I am not assigning beyond the limits of memory space so I am fine on that front as well.

I have below code which is giving me - *** Error in./server_issue': realloc(): invalid next size: 0x08249170 ***`.

I am not able to identify the problem, I am allocating just 64 bytes at a time so I am not out of memory address space and there is other memory allocation which could have possibly corrupted my heap memory.

Error says, invalid next size but as my last log tells Reallocating 64 bytes, Aborted (core dumped) so it means I am still trying to allocate 64 bytes. So, why error?

For HTML file, have any file which is more than 256 bytes.

Code: (minimum code to reproduce the issue)

#include<stdio.h>
#include<stdlib.h>
#include <stdbool.h>
#include <string.h>

typedef char BYTE;
bool load(FILE*, BYTE**, size_t*);

int main(void)
{
    FILE* file = fopen("/home/university/psets/pset6/pset6_working/public/hello.html", "r");
    BYTE* content;
    size_t length;
    load(file, &content, &length);
}

bool load(FILE* file, BYTE** content, size_t* length)
{
    int totalLength = 0;
    int readBytes = 0;
    *content = NULL;

    BYTE* fileContentTemp[64]; // working with 222222
    while ((readBytes = fread(fileContentTemp, 1, 64, file)) > 0)
    {
        printf("Reallocating %d bytes, ", readBytes);
        *content = realloc(*content, readBytes);
        printf("%p\n", *content);
        if(totalLength != 0)
        {
            memcpy(*content + totalLength + 1, fileContentTemp, readBytes);        
        } else{
            memcpy(*content + totalLength, fileContentTemp, readBytes);        
        }
        totalLength = totalLength + readBytes;
    }

    *length = totalLength;

    printf("CC image: %s\n", *content);
    printf("length is %d\n", *length);
    printf("fileContent %p\n", *content);

    return true;
}

Output:

Reallocating 64 bytes, 0x8249170
Reallocating 64 bytes, 0x8249170
*** Error in `./server_issue': realloc(): invalid next size: 0x08249170 ***
Reallocating 64 bytes, Aborted (core dumped)

---

Update: Even if I allocate 64 bytes instead of using readBytes while realloc then also I get the same error. I am getting error because of this line - *content = realloc(*content, readBytes);

Answer 1

Also, I am not assigning beyond the limits of memory space so I am fine on that front as well.

Being overly sure of your code is a great way to make debugging more difficult. Suspect everything is wrong.

---

You're not allocating enough memory, because you're using realloc incorrectly.

p = realloc(p, n) doesn't add n bytes to the allocation, it changes the total allocation size to the size you tell it.

Each time through the loop you're passing readBytes which is the number of additional bytes you want to copy. So your buffer never grows. But then you proceed to write past the end of it:

memcpy(*content + totalLength, ...

as totalLength keeps growing throughout the loop.

---

Calling realloc in a loop like this is typically a bad idea performance-wise. You should allocate enough total space up-front, and then read into it in the loop.

To get the total file size:

size_t get_file_size(FILE *f)
{
    long oldpos;
    size_t result;

    // Save the original file position
    oldpos = ftell(f);

    // Seek to the first byte past the end of the file
    fseek(f, 0, SEEK_END);

    // Get the file position - the number of bytes in the file
    result = ftell(f);

    // Return the file position to where it was
    fseek(f, oldpos, SEEK_SET);

    return result;
}

---

Here's a test, to demostrate what I am saying about realloc. Note, this uses malloc_usable_size which is bad practice, in general.

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>

int main(void)
{
    int i;
    char *p = NULL;

    for (i = 0; i < 10; i++) {
        p = realloc(p, 16);
        printf("[%d] size: %zd\n", i, malloc_usable_size(p));
    }

    return 0;
}

Output:

$ gcc -Wall -Werror -o realloc realloc.c 
$ ./realloc 
[0] size: 24
[1] size: 24
[2] size: 24
[3] size: 24
[4] size: 24
[5] size: 24
[6] size: 24
[7] size: 24
[8] size: 24
[9] size: 24

Note that malloc_usable_size() returns the same value, even after multiple calls to realloc which pass the same size. Also, the fact that it returns 24, even though we allocated 16 bytes is expected, and is a result of implementation details of the allocator.

From the man page for malloc_usable_size:

NOTES

The value returned by malloc_usable_size() may be greater than the requested size of the allocation because of alignment and minimum size constraints. Although the excess bytes can be overwritten by the application without ill effects, this is not good programming practice: the number of excess bytes in an allocation depends on the underlying implementation.

The main use of this function is for debugging and introspection.