Which optimization compiler switches make debugging very hard?

Question

I was involved in a debugging situation where I had no PDBs at all (unfortunately this still happens). In the particular case I was researching a stack corruption and I tried to do a manual stack walk. However, there was a strong mismatch between the ESP and the EBP registers, so I think the code was compiled with the /Oy optimization (frame pointer omission) turned on. In this case, I had to give up.

My question now is: from the Visual Studio 2015 C++ compiler switches for optmization, which ones will make debugging hard? A short explanation why it becomes hard would be great.

To limit the scope of the question, answers should consider x86 (32 bit) only.

The available options can be found on MSDN, as there are:

• O1 - creates small code
• Os - favors small code
• O2 - creates fast code
• Ot - favors fast code
• Ob - controls inline expansion
• Oi - generate intrinsic functions

The following ones do not need to be considered:

• Od - disables optimization. This will obviously cause the least trouble
• Og - is deprecated
• Ox - is just a combination of others. This will obviously cause the sum of troubles of the individual switches
• Oy - omit frame pointers. I already know about it. It makes stack walking really difficult. It much about guessing.

Answer 1

Wow, there are a lot of different types of code optimisation, way more than I have a detailed knowledge of, but I will try to detail different optimisations that significantly affect the debugging experience, knowledge of an optimisation can help guide which compiler switches will enable it.

• Reordering instructions to prevent the cpu idling, generally produces more code, the debugger will appear to jump around the code and not execute linearly.
• Reducing code to a compile time constant, smaller code and faster code, this code will be skipped over when debugging.
• Omitting frame pointers, produces smaller code but makes it more difficult to walk the stack.
• Efficient register usage, this will cause variables to be unreadable or wrong before they go out of scope because the compiler has decided the variable can be safely retired early and it's register freed up for usage elsewhere, instead of needlessly pushing it's value to the stack. This produces smaller and faster code.
• Inlining, inlining produces fatter code which is faster, the inlined functions will not appear in the call stack.

All the optimisation flags are frowned upon these days, Profile Guided Optimisation is by far the preferable way to use optimisations in release and if you want to debug release-ish code you should use the -Zo flag which will produce better pdb's which can get you more information about inlined functions and variables in registers.

GCC & clang have a per optimisation set of flags the -O flags are just amalgamations of those flags and looking at how GCC distributes the optimisations and details about what each optimisation is will help further understanding about all the different optimisations compilers, in general, do.

EDIT: Also if you want to turn on individual compiler flags and see what it does to the compiled code for GCC and clang god bolt is really useful https://gcc.godbolt.org

Answer 2

As far as I know, debugging problems may be mostly caused by information omission in the compiled binary.
Some of the causes are:

• frame pointers omission: /Oy, as you already found out
• inlining: /Ob2 is in effect when /O1, /O2 or /Ox is used
• code reduction: hard to know how this maps exactly to /O VC++ compiler options
• better register usage
• (to some degree) intrinsic functions: /Oi

My personal opinion is that a binary without debugging information (.PDB file) is usually "hard enough" to be debugged, at least not worth my time :-)