[Bug c/112442] New: Segfault from casting a ptr when using -O2

[Bug c/112442] New: Segfault from casting a ptr when using -O2

[adam.andersson at elisapolystar dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

            Bug ID: 112442
           Summary: Segfault from casting a ptr when using -O2
           Product: gcc
           Version: 13.2.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: adam.andersson at elisapolystar dot com
  Target Milestone: ---

Created attachment 56532
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=56532&action=edit
Small testprogram that reproduces the issue

I have attached a simple program that segfaults when compiling with GCC 13 and
-O2

Compiled with: gcc -v -save-temps -O2 gcc-segfault.c

How to reproduce:
$ gcc -O2 a-gcc-segfault.i -o test && ./test
Segmentation fault (core dumped)


This does not happen in GCC 12 or earlier, or if I use -O1 or no optimization.
Also doesnt happen if I remove the cast to unsigned char*, or if I inline the
test-function. 


My system:
Linux adam1 6.5.9-arch2-1 #1 SMP PREEMPT_DYNAMIC Thu, 26 Oct 2023 00:52:20
+0000 x86_64 GNU/Linux
gcc version 13.2.1 20230801 (GCC)

[Bug c/112442] Segfault from casting a ptr when using -O2

[adam.andersson at elisapolystar dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #1 from Adam Andersson <adam.andersson at elisapolystar dot com> ---
Disregard my comment about it working GCC 12. In gcc version 12.3.0 (GCC) it
does not work either.

[Bug c/112442] Segfault from casting a ptr when using -O2

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #2 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Looks like it doesn't always segfault, but the contents of the tmp buffer are
incorrect (which might segfault, or might fail to print "test!").

[Bug c/112442] Segfault from casting a ptr when using -O2

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I am not 100% sure but there seems like some kind of aliasing issue going on.

Basically you have a pointer to an `unsigned char` but writing it via a pointer
to `char`.
Yes writing to a type via `char` would be valid and well defined but you are
writing to a pointer of char.

[Bug c/112442] Segfault from casting a ptr when using -O2

[adam.andersson at elisapolystar dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #4 from Adam Andersson <adam.andersson at elisapolystar dot com> ---
(In reply to Andrew Pinski from comment #3)
> I am not 100% sure but there seems like some kind of aliasing issue going on.
> 
> Basically you have a pointer to an `unsigned char` but writing it via a
> pointer to `char`.
> Yes writing to a type via `char` would be valid and well defined but you are
> writing to a pointer of char.

Something weird is going on when casting a char pointer to an unsigned char
pointer. If you replace the unsigned char pointer with a void pointer it works
fine.

[Bug c/112442] Segfault from casting a ptr when using -O2

[schwab@linux-m68k.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #5 from Andreas Schwab <schwab@linux-m68k.org> ---
warning: dereferencing type-punned pointer will break strict-aliasing rules
[-Wstrict-aliasing]
   15 |     test((char **)&ptr, "test!");

[Bug c/112442] Segfault from casting a ptr when using -O2

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

Xi Ruoyao <xry111 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|UNCONFIRMED                 |RESOLVED
                 CC|                            |xry111 at gcc dot gnu.org

--- Comment #6 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
It's definitely an aliasing rule violation.  And it's still wrong even if you
use a void pointer.  The void pointer "workaround" just happens to work by
luck.

[Bug c/112442] Segfault from casting a ptr when using -O2

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #7 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
Note that in the "new bug" page, there is a red banner saying:

Before reporting that GCC compiles your code incorrectly, compile it with gcc
-Wall -Wextra and see whether this shows anything wrong with your code.
Similarly, if compiling with -fno-strict-aliasing -fwrapv makes a difference,
your code probably is not correct.

In this case -fno-strict-aliasing makes a difference.  And the code is indeed
incorrect.

[Bug c/112442] Segfault from casting a ptr when using -O2

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
The aliasing doesn't happen when writing to the array, it's when reading a
char* value from an object of type unsigned char*.

If you just passed the unsigned char* to memcpy instead of *(char**)&ptr it
would be OK.

memcpy(*&ptr, ...) would also be OK.

[Bug c/112442] Segfault from casting a ptr when using -O2

[adam.andersson at elisapolystar dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #9 from Adam Andersson <adam.andersson at elisapolystar dot com> ---
I was sure I had tried -fno-strict-aliasing without any difference, but I
guessed I messed up somehow. Sorry about that.

Still, is it not strange that -Wall doesn't generate a warning about this then?

[Bug c/112442] Segfault from casting a ptr when using -O2

[xry111 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112442

--- Comment #10 from Xi Ruoyao <xry111 at gcc dot gnu.org> ---
(In reply to Adam Andersson from comment #9)
> I was sure I had tried -fno-strict-aliasing without any difference, but I
> guessed I messed up somehow. Sorry about that.
> 
> Still, is it not strange that -Wall doesn't generate a warning about this
> then?

-Wall only enables -Wstrict-aliasing=3 which may have false negatives.

-Wstrict-aliasing=1 or -Wstrict-aliasing=2 warns about this, but generally they
can produce many false positives (as they are documented).

Generally it's impossible to make a reliable way to detect aliasing violation
at compile time.  For runtime checking LLVM folks were developing a Type
Sanitizer
(https://llvm.org/devmtg/2017-10/slides/Finkel-The%20Type%20Sanitizer.pdf) but
the development seems stalled now.

Thus we document "try -fno-strict-aliasing" in the "new bug" page as a "not so
bad" way to rule out aliasing issues (it's only "not so bad", not "very good"
because it may still hide real bugs).