[Bug libgcc/112635] New: stack smash protection does not work when code is compiled with -O

[Bug libgcc/112635] New: stack smash protection does not work when code is compiled with -O

[ago at gentoo dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112635

            Bug ID: 112635
           Summary: stack smash protection does not work when code is
                    compiled with -O
           Product: gcc
           Version: 13.2.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libgcc
          Assignee: unassigned at gcc dot gnu.org
          Reporter: ago at gentoo dot org
  Target Milestone: ---

If this is a known thing, please forgive me.

I was playing with the stack-smash protection and this is what I have found.

Please consider the following block of code:

#include <stdio.h>

int main () {
        char *buf[1];
        buf[1] = "A (stack-based) buffer overflow";
        printf("%s \n", buf[1]);
        return 0;
}

If compiled with -fsanitize=address I get:
==24180==ERROR: AddressSanitizer: stack-buffer-overflow on address
0x7f54c7600028 at pc 0x000000401210 bp 0x7fff54f73180 sp 0x7fff54f73178
WRITE of size 8 at 0x7f54c7600028 thread T0

If compiled with -fstack-protector-strong I get:

A (stack-based) buffer overflow 
*** stack smashing detected ***: terminated
Aborted

But while compiled with -fstack-protector-strong and -O1 (and also -O2/-O3/-Os)
I just get it working without any issue (it prints "A (stack-based) buffer
overflow")

So in the practice it does not work when -O is used.

I'm running gcc 13 on Gentoo, but I have reproduced this issue on ubuntu-22
(gcc-11.4.0), ubuntu-20 (gcc-9.4.0), and also centos-7 (gcc-4.8.5)

A side note:
I used checksec (https://github.com/slimm609/checksec.sh) basically a `readelf
-W -s a.out | grep "stack_chk_fail"` in this case to understand what happens
and I can see:
- when '-fstack-protector-strong' reports canary found (as expected)
- when '-fstack-protector-strong -O2' reports NO canary found
- when '-fstack-protector-all -O2' reports canary found but no 'stack smashing
detected' at runtime

[Bug tree-optimization/112635] stack smash protection does not work when code is compiled with -O

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112635

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libgcc                      |tree-optimization

--- Comment #1 from Richard Biener <rguenth at gcc dot gnu.org> ---
Yep, GCC fixes the code for you by eliding 'buf'.  That's perfectly OK though
since the code invokes undefined behavior.

I'm inclined to close this as INVALID/WONTFIX, it isn't really a bug
(but also not a feature you can rely on of course).

[Bug tree-optimization/112635] stack smash protection does not work when code is compiled with -O

[ago at gentoo dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112635

--- Comment #2 from Agostino Sarubbo <ago at gentoo dot org> ---
(In reply to Richard Biener from comment #1)
> Yep, GCC fixes the code for you by eliding 'buf'.  That's perfectly OK though
> since the code invokes undefined behavior.
> 
> I'm inclined to close this as INVALID/WONTFIX, it isn't really a bug
> (but also not a feature you can rely on of course).

Thanks for the response.

So the question is: to have a reliable ssp, do I have to use -O0 ?

[Bug tree-optimization/112635] stack smash protection does not work when code is compiled with -O

[ago at gentoo dot org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112635

Agostino Sarubbo <ago at gentoo dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |INVALID

--- Comment #3 from Agostino Sarubbo <ago at gentoo dot org> ---
(In reply to Richard Biener from comment #1)
> Yep, GCC fixes the code for you by eliding 'buf'

With a tool like https://godbolt.org/ I now get what you mean, so I'm closing
this as invalid by myself

[Bug tree-optimization/112635] stack smash protection does not work when code is compiled with -O

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112635

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |xry111 at gcc dot gnu.org

--- Comment #4 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Agostino Sarubbo from comment #2)
> (In reply to Richard Biener from comment #1)
> > Yep, GCC fixes the code for you by eliding 'buf'.  That's perfectly OK though
> > since the code invokes undefined behavior.
> > 
> > I'm inclined to close this as INVALID/WONTFIX, it isn't really a bug
> > (but also not a feature you can rely on of course).
> 
> Thanks for the response.
> 
> So the question is: to have a reliable ssp, do I have to use -O0 ?

The SSP is reliable in this case because there is no stack smashing, as the
stack access has been optimized away.  SSP is a hardening feature, not a debug
feature, so it's not designed for finding bugs in the program.

OTOH sanitizers are debugging features, and both ubsan and asan can detect this
bug.