[Bug bootstrap/111505] New: Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[Bug bootstrap/111505] New: Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[jamborm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

            Bug ID: 111505
           Summary: Asan (address-sanitizer) bootstrap fails since
                    r14-4003-geaa8e8541349df
           Product: gcc
           Version: 14.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: bootstrap
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jamborm at gcc dot gnu.org
                CC: dmalcolm at gcc dot gnu.org, fkastl at suse dot cz
            Blocks: 86656
  Target Milestone: ---
              Host: x86_64-linux-gnu
            Target: x86_64-linux-gnu

Bootstrapping with active address sanitizer fails at the beginning of
stage 3 since r14-4003-geaa8e8541349df (ggc, jit: forcibly clear GTY
roots in jit).

To reproduce, use --with-build-config=bootstrap-asan at configure
time, for example:

../src/configure --prefix=/home/user/install/prefix --enable-languages=c,c++
--enable-checking=release --enable-host-shared --disable-multilib
--with-build-config=bootstrap-asan

and run make (and wait).

At least one failure happens during configure script run of libiberty,
which fails with "C compiler cannot create executables" and the
corresponding config.log contains the following ASAN errors:

configure:3470:  /home/mjambor/gcc/mine/b-obj/./prev-gcc/xgcc
-B/home/mjambor/gcc/mine/b-obj/./prev-gcc/
-B/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/bin/
-B/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/bin/
-B/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/lib/ -isystem
/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/include -isystem
/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/sys-include   -fchecking=1 -o
conftest -g -O2 -fchecking=1 -fsanitize=address  -static-libstdc++
-static-libgcc -fsanitize=address -static-libasan
-B/home/mjambor/gcc/mine/b-obj/prev-x86_64-pc-linux-gnu/libsanitizer/
-B/home/mjambor/gcc/mine/b-obj/prev-x86_64-pc-linux-gnu/libsanitizer/asan/
-B/home/mjambor/gcc/mine/b-obj/prev-x86_64-pc-linux-gnu/libsanitizer/asan/.libs
 conftest.c  >&5
=================================================================
==2683==ERROR: AddressSanitizer: global-buffer-overflow on address
0x00000718d4d0 at pc 0x0000007cd234 bp 0x7ffdc15756e0 sp 0x7ffdc1574ea0
WRITE of size 16 at 0x00000718d4d0 thread T0
    #0 0x7cd233 in __interceptor_memset
/home/mjambor/gcc/mine/src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:847
    #1 0x12151ab in ggc_common_finalize()
/home/mjambor/gcc/mine/src/gcc/ggc-common.cc:1311
    #2 0x1dad8ef in toplev::finalize()
/home/mjambor/gcc/mine/src/gcc/toplev.cc:2354
    #3 0x796732 in main /home/mjambor/gcc/mine/src/gcc/main.cc:42
    #4 0x7f74182281af in __libc_start_call_main (/lib64/libc.so.6+0x281af)
(BuildId: 7729cbd8376d2b42276cc2cc10693449ff810847)
    #5 0x7f7418228278 in __libc_start_main@@GLIBC_2.34
(/lib64/libc.so.6+0x28278) (BuildId: 7729cbd8376d2b42276cc2cc10693449ff810847)
    #6 0x797e84 in _start ../sysdeps/x86_64/start.S:115

0x00000718d4d0 is located 48 bytes before global variable 'int_n_enabled_p'
defined in '/home/mjambor/gcc/mine/src/gcc/tree.cc:234:6' (0x718d500) of size 1
0x00000718d4d0 is located 0 bytes after global variable 'int_n_trees' defined
in '/home/mjambor/gcc/mine/src/gcc/tree.cc:235:22' (0x718d4c0) of size 16
SUMMARY: AddressSanitizer: global-buffer-overflow
/home/mjambor/gcc/mine/src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:847
in __interceptor_memset
Shadow bytes around the buggy address:
  0x00000718d200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x00000718d480: 00 00 00 00 f9 f9 f9 f9 00 00[f9]f9 f9 f9 f9 f9
  0x00000718d500: 01 f9 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00
  0x00000718d580: 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9 f9
  0x00000718d600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2683==ABORTING

[...]

configure:3708:  /home/mjambor/gcc/mine/b-obj/./prev-gcc/xgcc
-B/home/mjambor/gcc/mine/b-obj/./prev-gcc/
-B/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/bin/
-B/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/bin/
-B/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/lib/ -isystem
/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/include -isystem
/home/mjambor/gcc/mine/inst/x86_64-pc-linux-gnu/sys-include   -fchecking=1 -c
-g -O2 -fchecking=1 -fsanitize=address  conftest.c >&5
=================================================================
==2789==ERROR: AddressSanitizer: global-buffer-overflow on address
0x00000718d4d0 at pc 0x0000007cd234 bp 0x7ffd9f6fc140 sp 0x7ffd9f6fb900
WRITE of size 16 at 0x00000718d4d0 thread T0
    #0 0x7cd233 in __interceptor_memset
/home/mjambor/gcc/mine/src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:847
    #1 0x12151ab in ggc_common_finalize()
/home/mjambor/gcc/mine/src/gcc/ggc-common.cc:1311
    #2 0x1dad8ef in toplev::finalize()
/home/mjambor/gcc/mine/src/gcc/toplev.cc:2354
    #3 0x796732 in main /home/mjambor/gcc/mine/src/gcc/main.cc:42
    #4 0x7fbdc4e281af in __libc_start_call_main (/lib64/libc.so.6+0x281af)
(BuildId: 7729cbd8376d2b42276cc2cc10693449ff810847)
    #5 0x7fbdc4e28278 in __libc_start_main@@GLIBC_2.34
(/lib64/libc.so.6+0x28278) (BuildId: 7729cbd8376d2b42276cc2cc10693449ff810847)
    #6 0x797e84 in _start ../sysdeps/x86_64/start.S:115

0x00000718d4d0 is located 48 bytes before global variable 'int_n_enabled_p'
defined in '/home/mjambor/gcc/mine/src/gcc/tree.cc:234:6' (0x718d500) of size 1
0x00000718d4d0 is located 0 bytes after global variable 'int_n_trees' defined
in '/home/mjambor/gcc/mine/src/gcc/tree.cc:235:22' (0x718d4c0) of size 16
SUMMARY: AddressSanitizer: global-buffer-overflow
/home/mjambor/gcc/mine/src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:847
in __interceptor_memset
Shadow bytes around the buggy address:
  0x00000718d200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x00000718d480: 00 00 00 00 f9 f9 f9 f9 00 00[f9]f9 f9 f9 f9 f9
  0x00000718d500: 01 f9 f9 f9 f9 f9 f9 f9 00 00 00 00 00 00 00 00
  0x00000718d580: 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9 f9
  0x00000718d600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x00000718d700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2789==ABORTING


Referenced Bugs:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86656
[Bug 86656] [meta-bug] Issues found with -fsanitize=address

[Bug middle-end/111505] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |14.0
           Keywords|                            |GC
          Component|bootstrap                   |middle-end

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Hmm:
from gtype-desc.cc:
  {
    &int_n_trees[0].signed_type,
    1 * (NUM_INT_N_ENTS),
    sizeof (int_n_trees[0]),
    &gt_ggc_mx_tree_node,
    &gt_pch_nx_tree_node
  },
  {
    &int_n_trees[0].unsigned_type,
    1 * (NUM_INT_N_ENTS),
    sizeof (int_n_trees[0]),
    &gt_ggc_mx_tree_node,
    &gt_pch_nx_tree_node
  },

That looks wrong ...

[Bug middle-end/111505] [14 Regression] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Priority|P3                          |P1
            Summary|Asan (address-sanitizer)    |[14 Regression] Asan
                   |bootstrap fails since       |(address-sanitizer)
                   |r14-4003-geaa8e8541349df    |bootstrap fails since
                   |                            |r14-4003-geaa8e8541349df

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Quite odd indeed.

[Bug middle-end/111505] [14 Regression] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |slyfox at gcc dot gnu.org

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
*** Bug 111629 has been marked as a duplicate of this bug. ***

[Bug middle-end/111505] [14 Regression] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

--- Comment #4 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
In https://gcc.gnu.org/PR111629#c0 profiled bootstrap fales for a similar
reason.

There ggc_common_finalize() memset()s unexpected memory location

#1  0x0000000001933651 in ggc_common_finalize () at
/home/slyfox/dev/git/gcc/gcc/ggc-common.cc:1312
1312          memset (rti->base, 0, rti->stride * rti->nelt);
(gdb) list
1307        for (rti = *rt; rti->base != NULL; rti++)
1308          memset (rti->base, 0, rti->stride * rti->nelt);
1309
1310      for (rt = gt_ggc_rtab; *rt; rt++)
1311        for (rti = *rt; rti->base != NULL; rti++)
1312          memset (rti->base, 0, rti->stride * rti->nelt);


for this global:

struct GTY(()) ovl_op_info_t {
  /* The IDENTIFIER_NODE for the operator.  */
  tree identifier;
  /* The name of the operator.  */
  const char *name;
  /* The mangled name of the operator.  */
  const char *mangled_name;
  /* The (regular) tree code.  */
  enum tree_code tree_code : 16;
  /* The (compressed) operator code.  */
  enum ovl_op_code ovl_op_code : 8;
  /* The ovl_op_flags of the operator */
  unsigned flags : 8;
};

/* Overloaded operator info indexed by ass_op_p & ovl_op_code.  */
extern GTY(()) ovl_op_info_t ovl_op_info[2][OVL_OP_MAX];

Generated tables:

/* Structures for the easy way to mark roots.
   In an array, terminated by having base == NULL.  */
struct ggc_root_tab {
  void *base;
  size_t nelt;
  size_t stride;
  gt_pointer_walker cb;
  gt_pointer_walker pchw;
};

EXPORTED_CONST struct ggc_root_tab gt_ggc_r_gt_cp_tree_h[] = {
...
  { // 4
    &ovl_op_info[0][0].identifier,
    1 * (2) * (OVL_OP_MAX),
    sizeof (ovl_op_info[0][0]),
    &gt_ggc_mx_tree_node,
    &gt_pch_nx_tree_node
  },
  { // 5
    &ovl_op_info[0][0].name,
    1 * (2) * (OVL_OP_MAX),
    sizeof (ovl_op_info[0][0]),
    (gt_pointer_walker) &gt_ggc_m_S,
    (gt_pointer_walker) &gt_pch_n_S
  },
  { //6
    &ovl_op_info[0][0].mangled_name,
    1 * (2) * (OVL_OP_MAX),
    sizeof (ovl_op_info[0][0]),
    (gt_pointer_walker) &gt_ggc_m_S,
    (gt_pointer_walker) &gt_pch_n_S
  },

Is it a ggc_common_finalize() bug in assuming that `base` does not point to the
bbeginning of a struct?
Or a `gt_ggc_r_gt_cp_tree_h` bug that it does not point to the beginning of the
struct?

Also while at it: what should `ggc_common_finalize ()` ideally do to
`ovl_op_info`? memset() all of it? Or only pointers? I think it has information
only to do latter but it does it in a very strange way.

[Bug middle-end/111505] [14 Regression] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

--- Comment #5 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
(In reply to Sergei Trofimovich from comment #4)
> Is it a ggc_common_finalize() bug in assuming that `base` does not point to
> the bbeginning of a struct?
> Or a `gt_ggc_r_gt_cp_tree_h` bug that it does not point to the beginning of
> the struct?
> 
> Also while at it: what should `ggc_common_finalize ()` ideally do to
> `ovl_op_info`? memset() all of it? Or only pointers? I think it has
> information only to do latter but it does it in a very strange way.

Proposed fix as
https://gcc.gnu.org/pipermail/gcc-patches/2023-September/631634.html . That
fixes `profiledbootstrap` for me.

Having stared a bit more at gcc/ggc-common.cc the answer is: it's complicated.

There are 3 roots:

   gt_ggc_rtab
   gt_ggc_deletable_rtab
   gt_pch_scalar_rtab

Last two are simple: these are single-element complete values without
complexities. But gt_ggc_rtab is different: it encodes pointer locations within
structs (of non-pointers). GCH code carefully extracts and restores those
pointers. An example:

  for (rt = gt_ggc_rtab; *rt; rt++)
    for (rti = *rt; rti->base != NULL; rti++)
      for (i = 0; i < rti->nelt; i++)
        (*rti->pchw)(*(void **)((char *)rti->base + rti->stride * i));

finalize() should do the same.

[Bug middle-end/111505] [14 Regression] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

--- Comment #6 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Sergei Trofimovich <slyfox@gcc.gnu.org>:

https://gcc.gnu.org/g:7525707c5f3edb46958c4fdfbe30de5ddfa8923a

commit r14-4327-g7525707c5f3edb46958c4fdfbe30de5ddfa8923a
Author: Sergei Trofimovich <siarheit@google.com>
Date:   Thu Sep 28 20:20:31 2023 +0100

    ggc: do not wipe out unrelated data via gt_ggc_rtab [PR111505]

    There are 3 GC root tables:

       gt_ggc_rtab
       gt_ggc_deletable_rtab
       gt_pch_scalar_rtab

    `deletable` and `scalar` tables are both simple: each element always
    contains a pointer to the beginning of the object and it's size is the
    full object.

    `rtab` is different: it's `base` is a pointer in the middle of the
    struct and `stride` points to the next GC pointer in the array.

    Before the change there were 2 problems:

    1. We memset()ed not just pointers but data around them.
    2. We wen out of bounds of the last object described by gt_ggc_rtab
       and triggered bootstrap failures in profile and asan bootstraps.

    After the change we handle only pointers themselves like the rest of
    ggc-common.cc code.

    gcc/
            PR middle-end/111505
            * ggc-common.cc (ggc_zero_out_root_pointers, ggc_common_finalize):
            Add new helper. Use helper instead of memset() to wipe out
pointers.

[Bug middle-end/111505] [14 Regression] Asan (address-sanitizer) bootstrap fails since r14-4003-geaa8e8541349df

[slyfox at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111505

Sergei Trofimovich <slyfox at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
           Assignee|unassigned at gcc dot gnu.org      |slyfox at gcc dot gnu.org
         Resolution|---                         |FIXED

--- Comment #7 from Sergei Trofimovich <slyfox at gcc dot gnu.org> ---
I can confirm --with-build-config=bootstrap-asan fails the same way before the
fix and passes successfully after the fix.

Let's declare it FIXED.