CODEWITHSUNDEEP
cgccundefined-behavior
user4668606
user4668606

Reputation:

Why is a returned stack-pointer replaced by a null-pointer by gcc?

I've created the following function in c as a demonstration/small riddle about how the stack works in c:

#include "stdio.h"

int* func(int i)
{
    int j = 3;
    j += i;
    return &j;
}

int main()
{
    int *tmp = func(4);
    printf("%d\n", *tmp);
    func(5);
    printf("%d\n", *tmp);
}

It's obviously undefined behavior and the compiler also produces a warning about that. However unfortunately the compilation didn't quite work out. For some reason gcc replaces the returned pointer by NULL (see line 6d6).

00000000000006aa <func>:
 6aa:   55                      push   %rbp
 6ab:   48 89 e5                mov    %rsp,%rbp
 6ae:   48 83 ec 20             sub    $0x20,%rsp
 6b2:   89 7d ec                mov    %edi,-0x14(%rbp)
 6b5:   64 48 8b 04 25 28 00    mov    %fs:0x28,%rax
 6bc:   00 00 
 6be:   48 89 45 f8             mov    %rax,-0x8(%rbp)
 6c2:   31 c0                   xor    %eax,%eax
 6c4:   c7 45 f4 03 00 00 00    movl   $0x3,-0xc(%rbp)
 6cb:   8b 55 f4                mov    -0xc(%rbp),%edx
 6ce:   8b 45 ec                mov    -0x14(%rbp),%eax
 6d1:   01 d0                   add    %edx,%eax
 6d3:   89 45 f4                mov    %eax,-0xc(%rbp)
 6d6:   b8 00 00 00 00          mov    $0x0,%eax
 6db:   48 8b 4d f8             mov    -0x8(%rbp),%rcx
 6df:   64 48 33 0c 25 28 00    xor    %fs:0x28,%rcx
 6e6:   00 00 
 6e8:   74 05                   je     6ef <func+0x45>
 6ea:   e8 81 fe ff ff          callq  570 <__stack_chk_fail@plt>
 6ef:   c9                      leaveq 
 6f0:   c3                      retq

This is the disassembly of the binary compiled with gcc version 7.5.0 and the -O0-flag; no other flags were used. This behavior makes the entire code pointless, since it's supposed to show how the stack behaves across function-calls. Is there any way to achieve a more literal compilation of this code with a at least somewhat up-to-date version of gcc?

And just for the sake of curiosity: what's the point of changing the behavior of the code like this in the first place?

Upvotes: 0

Views: 178

Answers (1)

KamilCuk
KamilCuk

Reputation: 141698

Putting the return value in a pointer variable seems to change the behavior of the compiler and it generates the assembly code that returns a pointer to stack:

int* func(int i) {
     int j = 3;
     j += i;
     int *p = &j;
     return p;
}

Upvotes: 1

Related Questions