CODEWITHSUNDEEP
cgccprintfcompiler-optimization
Noor Yag
Noor Yag

Reputation: 43

Why does printf print 0s when I wrongly returned pointer to value on the stack only when optimizing with gcc compiler?

I am trying to understand the behavior of printf in this example. of course the main issue here is that we are returning a pointer to a value on the stack that was popped after the function Boo returned.

I compiled with gcc. In test1: I got 7 printed twice which was expected. And a garbage value on the second printf in test2. but when I compiled with gcc -O3 I got 0 printed on both cases and a compiler warning about returning address of local variable.

test.c: In function ‘Foo’: test.c:8:12: warning: function returns address of local variable [-Wreturn-local-addr] return t; ^ test.c:5:9: note: declared here int j;

Can someone help me explain how does the behavior of printf that causes this behavior?

int *Boo(int i, int *p)
{
    int j;
    int *t = &j;
    *t = i + *p;
    return t;
}

void Foo(int x)
{
    if (x == 0) { return;}
    Foo(x - 1);
}

//test1
int main(void)
{
    int x = 5;
    int *t = Boo(2, &x);
    printf("%d", *t);
    printf("%d", *t);

    return 0;
}

//test2
int main(void)
{
    int x = 5;
    int *t = Boo(2, &x);
    
    printf("%d", *t);
    Foo(8);
    printf("%d", *t);

    return 0;
}

Upvotes: 0

Views: 154

Answers (1)

Nate Eldredge
Nate Eldredge

Reputation: 58663

GCC sees that Boo returns a pointer to a local variable, and that therefore any attempt to use this pointer is undefined behavior. That means, according to the C standard, that the compiler can do whatever it wants, and in such cases GCC will often generate code that's "efficient" even if it is totally unrelated to what the programmer may have intended. It inlines the call to Boo, which has no visible effects and therefore is optimized away, and it picks an "efficient" way to provide an argument for printf, which happens to be the constant 0.

See on godbolt. The integer argument to printf goes in esi, which comes from ebp which is zeroed. I guess there's a missed optimization in that it saves the same 0 in ebp across the call to printf to reload into esi, instead of just re-zeroing esi afterward. But the whole analysis is kind of pointless since, again, the behavior is undefined.

Boo itself is optimized into a function that just returns a NULL pointer and does nothing else, which again is legal because of undefined behavior, but that version of Boo is not called by the program. Foo is also optimized into a function which returns immediately without recursing; the compiler can tell that the recursive calls have no effect. (And if a negative argument were passed to Foo, you'd have undefined behavior due to signed integer overflow, so the compiler need not handle that case.)

Upvotes: 0

Related Questions