GCC: Segmentation fault and debugging program that only crashes when optimized

Question

This is a follow up to thread: C: Segmentation fault and maybe GDB is lying to me

I have a program that compiles fine with -O0 but segfaults with -O1, -O2, -O3 and Ofast. It seems the stack somehow gets corrupted at some point but I can't figure out why or where.

First, here's part of the struct that I'm using. It's in a header file:

typedef struct {
    GLuint nsHandle;
} OGL_STATE_T;

Here's the relevant watered down part of the main file:

void init(OGL_STATE_T *state) {

    printf("init: %p\n", state);    // Print pointer address make sure it's the same.

    compileShaders(state);

}

int main(argc, char *argv[]) {

    static OGL_STATE_T _state, *state=&_state;

    printf("main: %p\n", state);    // Print pointer address

    init(state);

    return 0;
}

Then here's the compileShaders function. This is where the pointer address corruption happens:

void compileShaders(OGL_STATE_T *state) {

    printf("compileShaders entry: %p\n", state);    // Print pointer address make sure it's good

    GLuint nsVertex = compileShader("nsVertex", GL_VERTEX_SHADER, state);
    GLuint nsFragment = compileShader("nsFragment", GL_FRAGMENT_SHADER, state);

    printf("compileShaders return: %p\n", state);    // Print pointer when returning.

    state->nsHandle = glCreateProgram();    // Segmentation fault here.

    /* ... */
}

As will be illustrated in the output here shortly, the second printf statement returns the wrong address.

Finally, the compileShader function (Notice the lack of 's' at the end in the name here). Originally the function didn't take the state pointer but I added it so I could track where in the execution the corruption happens.

GLuint compileShader{char * shaderName, GLenum shaderType, OGL_STATE_T *state} {

    printf("compileShader 1: %p\n", state);    // Print state address at function entry

    FILE *shaderFile;
    char fileName[sizeof shaderName + 8];
    long lSize;
    char *buffer;

    strcpy(fileName, "./");
    strcpy(fileName, shaderName);
    strcpy(fileName, ".glsl");

    shaderFile = fopen(fileName, "rb");

    fseek(shaderFile, 0L, SEEK_END);
    lSize = ftell(shaderFile);
    rewind(shaderFile);

    buffer = calloc(1, lSize + 1);

    GLuint shaderHandle = glCreateShader(shaderType);

    printf("compileShader 2: %p\n", state);    // Print state address at function middle

    const GLchar *shaderString = buffer;
    glShaderSource(shaderHandle, 1, &shaderString, 0);

    glCompileShader(shaderHandle);

    GLint compileSuccess;
    glGetShaderiv(shaderHandle, GL_COMPILE_STATUS, &compileSuccess);

    fclose(shaderFile);
    shaderString = NULL;
    free(buffer);

    printf("compileShader 3: %p\n\n", state);    // Print state address before returning

    return shaderHandle;
}

Now for the kicker here's the output:

main: 0x1387c
init: 0x1387c

compileShaders entry: 0x1387c

compileShader 1: 0x1387c
compileShader 2: 0x1387c
compileShader 3: 0x1387c

compileShader 1: 0x1387c
compileShader 2: 0x1387c
compileShader 3: 0x1387c

compileShaders return: 0x1006c
Segmentation fault. (Core dumped)

So this is telling me that when the compileShader() function exits the address is still good and after it returns into compileShaders() (Please do not get confused one has a 's' and the other one doesn't), the address gets corrupted?

I'm a bit flabbergasted at this point. It's hard to debug this because if I don't optimize the code I do not get any errors. However if I do optimize the code (Doesn't matter if it's O1, O2, O3 or Ofast) I get the segmentation fault.

The printf statements are my only friend right now and they're not telling me anything at this point. Is this the part where I submit a bug report to GCC?

Thanks to anyone who takes the time to read this thread. I know it's a bit on the longer side of things.

Answer 1

Problem is with the fileName definition in the compileShaders()

char fileName[sizeof shaderName + 8]; . 

That is not correct and not allocate enough bytes for fileName.

You need to allocate with strlen(shaderName)+8, not sizeof(shaderName).