[Bug other/101949] New: git miscompiled with -flto -fipa-pta

[Bug other/101949] New: git miscompiled with -flto -fipa-pta

[alex_y_xu at yahoo dot ca]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

            Bug ID: 101949
           Summary: git miscompiled with -flto -fipa-pta
           Product: gcc
           Version: 11.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: other
          Assignee: unassigned at gcc dot gnu.org
          Reporter: alex_y_xu at yahoo dot ca
  Target Milestone: ---

Created attachment 51313
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51313&action=edit
step 2 patch

steps to reproduce:

1. download and unpack git-2.33.0 (other versions may also work)
2. apply the attached patch to work around binutils bugs with duplicate
function names
3. make CFLAGS="-O2 -flto -fipa-pta" git git-remote-https
4. git clone https://github.com/Hello71/empty-project.git
5. PATH=$PWD:$PATH git -C empty-project fetch
6. objdump --disassemble=iterate_ref_map2 git

expected behavior:

step 5 succeeds. in step 6, function references %rsi at some point

actual behavior:

step 5 segfaults. in step 6, function never references %rsi (i.e. returns 0
without populating *oid)

additional behavior:

it works ok without -flto or without -fipa-pta, thus the non-reduced test case.

note that the patch is only required for step 6; step 5 fails even if the patch
is not applied.

[Bug other/101949] git miscompiled with -flto -fipa-pta

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2021-08-17
                 CC|                            |marxin at gcc dot gnu.org
     Ever confirmed|0                           |1

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
Confirmed. Isolating and bisecting right now..

[Bug ipa/101949] git miscompiled with -flto -fipa-pta

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
I don't think all of the bugs with -fipa-pta have been fixed which is why it is
not turned on by default.

[Bug ipa/101949] git miscompiled with -flto -fipa-pta

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #3 from Martin Liška <marxin at gcc dot gnu.org> ---
Created attachment 51315
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51315&action=edit
tarball with pre-processed source files

[Bug ipa/101949] git miscompiled with -flto -fipa-pta

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hubicka at gcc dot gnu.org,
                   |                            |rguenth at gcc dot gnu.org

--- Comment #4 from Martin Liška <marxin at gcc dot gnu.org> ---
Ok, so it started with ipa modref change, I was able to reduce it 5
pre-processed source files. Unfortunately, one needs at least 2 LTRANS.
What happens is that:

builtin/fetch.c:
...
static int iterate_ref_map2(void *cb_data, struct object_id *oid)
{
        struct ref **rm = cb_data;
        struct ref *ref = *rm;

        while (ref && ref->status == REF_STATUS_REJECT_SHALLOW)
                ref = ref->next;
        if (!ref)
                return -1; /* end of the list */
        *rm = ref->next;
        oidcpy(oid, &ref->old_oid);
        return 0;
}

the following 2 statements are optimized out (I think in PRE in ltrans1):

        *rm = ref->next;
        oidcpy(oid, &ref->old_oid);

Bad run:
gcc -flto=auto -o xxx -fdump-tree-optimized -shared -fPIC -fipa-pta *.i -O2

Good one:
gcc -flto=auto -o xxx -fdump-tree-optimized -shared -fPIC *.i -O2

$ diff -u xxx.ltrans1.ltrans.248t.optimized.good
xxx.ltrans1.ltrans.248t.optimized.bad

-;; Function iterate_ref_map2 (iterate_ref_map2, funcdef_no=7, decl_uid=4567,
cgraph_uid=27, symbol_order=7008)
+;; Function iterate_ref_map2 (iterate_ref_map2, funcdef_no=65, decl_uid=4567,
cgraph_uid=27, symbol_order=7008)

+Removing basic block 6
 Removing basic block 7
 Removing basic block 8
 Removing basic block 9
@@ -264,51 +265,39 @@
 {
   struct ref * ref;
   <unnamed type> _1;
-  struct ref * pretmp_4;
   int _5;
-  const unsigned char[32] * _13;
-  unsigned char[32] * _14;
-  int _15;

   <bb 2> [local count: 114863530]:
   ref_9 = MEM[(struct ref * *)cb_data_7(D)];
   if (ref_9 != 0B)
     goto <bb 4>; [94.50%]
   else
-    goto <bb 6>; [5.50%]
+    goto <bb 5>; [5.50%]

   <bb 3> [local count: 958878296]:
-  if (pretmp_4 != 0B)
+  ref_10 = ref_3->next;
+  if (ref_10 != 0B)
     goto <bb 4>; [94.50%]
   else
-    goto <bb 6>; [5.50%]
+    goto <bb 5>; [5.50%]

   <bb 4> [local count: 1014686026]:
-  # ref_21 = PHI <pretmp_4(3), ref_9(2)>
-  _1 = ref_21->status;
-  pretmp_4 = ref_21->next;
+  # ref_3 = PHI <ref_10(3), ref_9(2)>
+  _1 = ref_3->status;
   if (_1 == 8)
     goto <bb 3>; [94.50%]
   else
     goto <bb 5>; [5.50%]

-  <bb 5> [local count: 55807731]:
-  MEM[(struct ref * *)cb_data_7(D)] = pretmp_4;
-  _13 = &MEM[(const struct object_id *)ref_21 + 8B].hash;
-  _14 = &oid_12(D)->hash;
-  memcpy (_14, _13, 32);
-  _15 = MEM[(const struct object_id *)ref_21 + 8B].algo;
-  oid_12(D)->algo = _15;
-
-  <bb 6> [local count: 114863531]:
-  # _5 = PHI <-1(3), 0(5), -1(2)>
+  <bb 5> [local count: 114863531]:
+  # _5 = PHI <-1(3), 0(4), -1(2)>
   return _5;

 }

I'm trying reducing that even more..

[Bug ipa/101949] git miscompiled with -flto -fipa-pta

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #5 from Martin Liška <marxin at gcc dot gnu.org> ---
One sees it on the current master, gcc-11 branch make a different
partitioning..

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|git miscompiled with -flto  |[11/12 Regression] git
                   |-fipa-pta                   |miscompiled with -flto
                   |                            |-fipa-pta since
                   |                            |r11-5061-g85ebbabd85e03bdc

--- Comment #6 from Martin Liška <marxin at gcc dot gnu.org> ---
Started with r11-5061-g85ebbabd85e03bdc.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #7 from Martin Liška <marxin at gcc dot gnu.org> ---
Created attachment 51317
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51317&action=edit
Reduced test-case

Now I have 4 files, ~250kB in total. Won't reduce it more..

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |11.3

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[boris2.9 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Boris Carvajal <boris2.9 at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |boris2.9 at gmail dot com

--- Comment #8 from Boris Carvajal <boris2.9 at gmail dot com> ---
I just want to add that running 'git ls-tree HEAD' inside a git repo is also
bugged, there is no crash here but the file names in the output are corrupted.

A Gentoo user experiencing something similar reported that adding
'-ffat-lto-objects' resolves the problem.
(https://github.com/InBetweenNames/gentooLTO/pull/776)

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #9 from Martin Liška <marxin at gcc dot gnu.org> ---
All right, so I investigated more, with a new dbg counter in
gimple_call_arg_flags the following flags are updated:

combine: flags=0, modref_flags=0
# .MEM_817 = VDEF <.MEM_816>
_35 = check_connected (iterate_ref_map2, &rm, &opt);

combine: flags=0, modref_flags=0
# .MEM_68 = VDEF <.MEM_67>
_29 = check_connected (iterate_ref_map2, &rm, &opt);

where arg == 0, so IPA MOD ref claims that iterate_ref_map2 is '(EAF_NOCLOBBER
| EAF_NOESCAPE)' which is likely the problem.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #10 from Martin Liška <marxin at gcc dot gnu.org> ---
Created attachment 51328
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51328&action=edit
LTRANS IPA PTA dump - good

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #11 from Martin Liška <marxin at gcc dot gnu.org> ---
Created attachment 51329
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=51329&action=edit
LTRANS IPA PTA dump - bad

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #12 from Richard Biener <rguenth at gcc dot gnu.org> ---
> cat gcc/testsuite/gcc.dg/lto/pr101949_0.c
/* { dg-lto-do run } */
/* { dg-lto-options { "-O2 -fipa-pta -flto -flto-partition=1to1" } } */

extern int bar (int (*)(int *), int *);

static int x;

static int __attribute__ ((noinline)) foo (int *p)
{
  *p = 1;
  x = 0;
  return *p;
}

int main ()
{
  if (bar (foo, &x) != 0)
    __builtin_abort ();
  return 0;
}
> cat gcc/testsuite/gcc.dg/lto/pr101949_1.c
int __attribute__((noinline,noclone)) bar (int (*fn)(int *), int *p)
{
  return fn (p);
}

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #13 from Richard Biener <rguenth at gcc dot gnu.org> ---
When modref claims that bar doesn't write to 'fn' that confuses IPA PTA since
internally it treats calling a function as "writing" to the argument space of
all functions the function pointer points to.

I'm not sure if it makes much sense to maintain any EAF_* flags when the
argument is called, maybe we can drop to varying here in modref?  For PTA
it's really difficult since there may be data and functions in points-to
sets so we can't easily "ignore" EAF_* flags for some arguments but not
others.  Also consider function pointers passed through a void * argument.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|unassigned at gcc dot gnu.org      |marxin at gcc dot gnu.org
             Status|NEW                         |ASSIGNED

--- Comment #14 from Martin Liška <marxin at gcc dot gnu.org> ---
Let me write a patch for it.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #15 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jan Hubicka <hubicka@gcc.gnu.org>:

https://gcc.gnu.org/g:9b08f7764cecd16cba84944f2a8b67a7f73a7ce7

commit r12-3059-g9b08f7764cecd16cba84944f2a8b67a7f73a7ce7
Author: Jan Hubicka <jh@suse.cz>
Date:   Sun Aug 22 20:57:19 2021 +0200

    Clear EAF_NOCLOBBER for indirect calls

    gcc/ChangeLog:

    2021-08-22  Jan Hubicka  <hubicka@ucw.cz>
                Martin Liska  <mliska@suse.cz>

            PR middle-end/101949
            * ipa-modref.c (analyze_ssa_name_flags): Indirect call implies
            ~EAF_NOCLOBBER.

    gcc/testsuite/ChangeLog:

    2021-08-22  Jan Hubicka  <hubicka@ucw.cz>
                Martin Liska  <mliska@suse.cz>

            * gcc.dg/lto/pr101949_0.c: New test.
            * gcc.dg/lto/pr101949_1.c: New test.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hjl.tools at gmail dot com

--- Comment #16 from H.J. Lu <hjl.tools at gmail dot com> ---
On Linux/x86-64 with -m32, r12-3059 gave

FAIL: gcc.dg/lto/pr101949 c_lto_pr101949_0.o-c_lto_pr101949_1.o execute -O2
-fipa-pta -flto -flto-partition=1to1

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[hjl.tools at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #17 from H.J. Lu <hjl.tools at gmail dot com> ---
(In reply to H.J. Lu from comment #16)
> On Linux/x86-64 with -m32, r12-3059 gave
> 
> FAIL: gcc.dg/lto/pr101949 c_lto_pr101949_0.o-c_lto_pr101949_1.o execute -O2
> -fipa-pta -flto -flto-partition=1to1

It also failed with -m64.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #18 from Martin Liška <marxin at gcc dot gnu.org> ---
I can confirm that failure. The test-case still crashes after Honza's patch.

[Bug ipa/101949] [11/12 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #19 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Jan Hubicka <hubicka@gcc.gnu.org>:

https://gcc.gnu.org/g:5bd4ab91660c8f5534c46979d4846a4f1a0972b0

commit r12-3083-g5bd4ab91660c8f5534c46979d4846a4f1a0972b0
Author: Jan Hubicka <hubicka@ucw.cz>
Date:   Mon Aug 23 16:16:25 2021 +0200

    Fix previous ipa-modref patch

    gcc/ChangeLog:

            PR middle-end/101949
            * ipa-modref.c (analyze_ssa_name_flags): Fix merging of
            EAF_NOCLOBBER

[Bug ipa/101949] [11 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[11/12 Regression] git      |[11 Regression] git
                   |miscompiled with -flto      |miscompiled with -flto
                   |-fipa-pta since             |-fipa-pta since
                   |r11-5061-g85ebbabd85e03bdc  |r11-5061-g85ebbabd85e03bdc
      Known to work|                            |12.0

--- Comment #20 from Martin Liška <marxin at gcc dot gnu.org> ---
Fixed on master, I'm going to do a backport to gcc-11 branch.

[Bug ipa/101949] [11 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #21 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Martin Liska
<marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:d0adc557a26963b9ad0e07829872c968e67c5ffc

commit r11-8898-gd0adc557a26963b9ad0e07829872c968e67c5ffc
Author: Jan Hubicka <jh@suse.cz>
Date:   Sun Aug 22 20:57:19 2021 +0200

    Clear EAF_NOCLOBBER for indirect calls

    gcc/ChangeLog:

    2021-08-22  Jan Hubicka  <hubicka@ucw.cz>
                Martin Liska  <mliska@suse.cz>

            PR middle-end/101949
            * ipa-modref.c (analyze_ssa_name_flags): Indirect call implies
            ~EAF_NOCLOBBER.

    gcc/testsuite/ChangeLog:

    2021-08-22  Jan Hubicka  <hubicka@ucw.cz>
                Martin Liska  <mliska@suse.cz>

            * gcc.dg/lto/pr101949_0.c: New test.
            * gcc.dg/lto/pr101949_1.c: New test.

    (cherry picked from commit 9b08f7764cecd16cba84944f2a8b67a7f73a7ce7)

[Bug ipa/101949] [11 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #22 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-11 branch has been updated by Martin Liska
<marxin@gcc.gnu.org>:

https://gcc.gnu.org/g:e527b92b183de8c887213ccf6b024ddb4e363865

commit r11-8899-ge527b92b183de8c887213ccf6b024ddb4e363865
Author: Jan Hubicka <hubicka@ucw.cz>
Date:   Mon Aug 23 16:16:25 2021 +0200

    Fix previous ipa-modref patch

    gcc/ChangeLog:

            PR middle-end/101949
            * ipa-modref.c (analyze_ssa_name_flags): Fix merging of
            EAF_NOCLOBBER

    (cherry picked from commit 5bd4ab91660c8f5534c46979d4846a4f1a0972b0)

[Bug ipa/101949] [11 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #23 from Martin Liška <marxin at gcc dot gnu.org> ---
Fixed now.

[Bug ipa/101949] [11 Regression] git miscompiled with -flto -fipa-pta since r11-5061-g85ebbabd85e03bdc

[alex_y_xu at yahoo dot ca]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101949

--- Comment #24 from alex_y_xu at yahoo dot ca ---
Thanks! :D