[Bug bootstrap/49786] New: [4.7 Regression] bootstrap failed with bootstrap-profiled

[Bug bootstrap/49786] New: [4.7 Regression] bootstrap failed with bootstrap-profiled

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

           Summary: [4.7 Regression] bootstrap failed with
                    bootstrap-profiled
           Product: gcc
           Version: 4.7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: bootstrap
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: hjl.tools@gmail.com


On Linux/x86-64, revision 176488 gave:

../src-trunk/configure \
        --enable-clocale=gnu --with-system-zlib --enable-shared
--with-demangler-in-ld --enable-cloog-backend=isl
--with-ppl-include=/opt/gnu/include --with-ppl-lib=/opt/gnu/lib64
--with-cloog-include=/opt/gnu/include --with-cloog-lib=/opt/gnu/lib64
--with-build-config=bootstrap-lto --with-fpmath=sse
--enable-languages=c,c++,fortran,java,lto,objc
...
build/print-rtl.o build/read-md.o build/errors.o .././libiberty/libiberty.a
lto1: error: caller edge count is negative
VEC_vect_el_t_base_quick_push.3721.constprop.26/232 @0x7f648067bd80 (asm:
VEC_vect_el_t_base_quick_push.3721.constprop.26) (clone of
VEC_vect_el_t_base_quick_push/319) availability:local analyzed executed
5157220072x reachable local finalized
 called by: VEC_vect_el_t_heap_safe_push/324 (1.00 per call)
output_reserved_units_table/352 create_state_ainsn_table/359 (2163498541x)
(100.00 per call) output_translate_vect/371 (255723456x) (100.00 per call)
output_min_issue_delay_table/374 (2163498541x) (100.00 per call)
output_min_issue_delay_table/374 (1591314722x) (100.00 per call)
VEC_vect_el_t_heap_safe_push.3754.constprop.21/177 (-1016815188x) (1.00 per
call)
 calls: VEC_vect_el_t_base_quick_push.part.58.3716.constprop.27/233
 References:
 Refering this function:
lto1: internal compiler error: verify_cgraph_node failed
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
lto-wrapper: /export/gnu/import/svn/gcc-test-profile/bld/./prev-gcc/xgcc
returned 1 exit status
/usr/local/x86_64-unknown-linux-gnu/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[6]: *** [build/genautomata] Error 1
make[6]: *** Waiting for unfinished jobs....
profiling:/export/gnu/import/svn/gcc-test-profile/bld/gcc/pretty-print.gcda:Invocation
mismatch - some data files may have been
removedprofiling:/export/gnu/import/svn/gcc-test-profile/bld/gcc/insn-emit.gcda:Invocation
mismatch - some data files may have been removedrm gcj-dbtool.pod jcf-dump.pod
jv-convert.pod grmic.pod gcov.pod gcj.pod gc-analyze.pod gij.pod gcc.pod
gfortran.pod cpp.pod fsf-funding.pod gfdl.pod
make[6]: Leaving directory `/export/gnu/import/svn/gcc-test-profile/bld/gcc'
make[5]: *** [all-stagefeedback-gcc] Error 2
make[5]: Leaving directory `/export/gnu/import/svn/gcc-test-profile/bld'
make[4]: *** [stagefeedback-bubble] Error 2
make[4]: Leaving directory `/export/gnu/import/svn/gcc-test-profile/bld'
make[3]: *** [profiledbootstrap] Error 2

Revision 176460 is OK.
make[3]: Leaving directory `/export/gnu/import/svn/gcc-test-profile/bld'

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

H.J. Lu <hjl.tools at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ian at airs dot com
   Target Milestone|---                         |4.7.0

--- Comment #1 from H.J. Lu <hjl.tools at gmail dot com> 2011-07-20 00:45:50 UTC ---
It is caused by revision 176480:

http://gcc.gnu.org/ml/gcc-cvs/2011-07/msg00747.html

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #2 from Ian Lance Taylor <ian at airs dot com> 2011-07-20 13:25:32 UTC ---
I'm using an x86_64 Ubuntu Lucid system.  I'm using unmodified revision 176479
of mainline.  I used ppl-0.11 and cloog-0.16.2 from
ftp://gcc.gnu.org/pub/gcc/infrastructure.  I configured like this:

../trunk/configure --enable-clocale=gnu --with-system-zlib --enable-shared
--with-demangler-in-ld --enable-cloog-backend=isl
--with-ppl=/home/iant/gnu/ppl-0.11-install
--with-cloog=/home/iant/gnu/cloog-0.16.2-install
--with-build-config=bootstrap-lto --with-fpmath=sse
--enable-languages=c,c++,fortran,java,lto,objc

I ran "make profiledbootstrap".  The bootstrap completed successfully.

How can I recreate the problem?

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #3 from H.J. Lu <hjl.tools at gmail dot com> 2011-07-20 13:31:09 UTC ---
(In reply to comment #2)
> I'm using an x86_64 Ubuntu Lucid system.  I'm using unmodified revision 176479
> of mainline.  I used ppl-0.11 and cloog-0.16.2 from
> ftp://gcc.gnu.org/pub/gcc/infrastructure.  I configured like this:
> 
> ../trunk/configure --enable-clocale=gnu --with-system-zlib --enable-shared
> --with-demangler-in-ld --enable-cloog-backend=isl
> --with-ppl=/home/iant/gnu/ppl-0.11-install
> --with-cloog=/home/iant/gnu/cloog-0.16.2-install
> --with-build-config=bootstrap-lto --with-fpmath=sse
> --enable-languages=c,c++,fortran,java,lto,objc
> 
> I ran "make profiledbootstrap".  The bootstrap completed successfully.
> 
> How can I recreate the problem?

I saw it on a 8-core machine with "make -j8 profiledbootstrap".

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #4 from Ian Lance Taylor <ian at airs dot com> 2011-07-20 14:11:55 UTC ---
Is it repeatable for you?  I don't know how to investigate this if I can't
repeat it myself.  I also don't see how this could be related to the change to
building with C++, though of course anything is possible.  I was using "make
-j6" but I think it's only a dual-core system.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #5 from H.J. Lu <hjl.tools at gmail dot com> 2011-07-20 14:16:47 UTC ---
(In reply to comment #4)
> Is it repeatable for you?  I don't know how to investigate this if I can't
> repeat it myself.  I also don't see how this could be related to the change to
> building with C++, though of course anything is possible.  I was using "make
> -j6" but I think it's only a dual-core system.

Please see "Gcc [trunk revision XXX] failed to profiledbootstrap on x86_64!":

http://gcc.gnu.org/ml/gcc-regression/2011-07/

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #6 from Ian Lance Taylor <ian at airs dot com> 2011-07-20 15:29:58 UTC ---
I'm sorry, I can't see any useful information in that link.  It seems to simply
repeat the information that is already in this bug report.  Am I missing
something?

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #7 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-07-20 15:34:59 UTC ---
I think you need to make sure that a / the linker plugin works.  IIRC HJ
uses his own binutils branch and GNU ld.

So in case you are using gold (and configury correctly detects that and
enables linker plugin use by default) try using GNU ld instead.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

Richard Guenther <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jamborm at gcc dot gnu.org

--- Comment #8 from Richard Guenther <rguenth at gcc dot gnu.org> 2011-07-20 15:36:56 UTC ---
Btw, I think we're running into a bug in the new IPA-CP code (see the SPEC 2k6
LTO build fails HJ also reported).  Thus, CCing martin.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[jamborm at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #9 from Martin Jambor <jamborm at gcc dot gnu.org> 2011-07-20 15:47:54 UTC ---
(In reply to comment #8)
> Btw, I think we're running into a bug in the new IPA-CP code (see the SPEC 2k6
> LTO build fails HJ also reported).  Thus, CCing martin.

I understand this is LTO profiled bootstrap.  I did not try that
before committing (as opposed to one without LTO).  I will try doing
that with and without the patch.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[markus at trippelsdorf dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

Markus Trippelsdorf <markus at trippelsdorf dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |markus at trippelsdorf dot
                   |                            |de

--- Comment #10 from Markus Trippelsdorf <markus at trippelsdorf dot de> 2011-07-20 16:38:38 UTC ---
Also happens with gold:

/var/tmp/gcc_build_dir/./prev-gcc/xgcc -B/var/tmp/gcc_build_dir/./prev-gcc/
-B/usr/x86_64-pc-linux-gnu/bin/ -B/usr/x86_64-pc-linux-gnu/bin/
-B/usr/x86_64-pc-linux-gnu/lib/ -isystem /usr/x86_64-pc-linux-gnu/include
-isystem /usr/x86_64-pc-linux-gnu/sys-include      -march=native -O2 -pipe
-flto=jobserver -frandom-seed=1 -fprofile-use -DIN_GCC   -W -Wall
-Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes
-Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros
-Wno-overlength-strings -Wold-style-definition -Wc++-compat -fno-common 
-DHAVE_CONFIG_H -DGENERATOR_FILE
-Wl,-O1,--hash-style=gnu,--as-needed,--gc-sections,--icf=all,--icf-iterations=3
 -o build/genautomata \
    build/genautomata.o build/rtl.o build/read-rtl.o build/ggc-none.o
build/vec.o build/min-insn-modes.o build/gensupport.o build/print-rtl.o
build/read-md.o build/errors.o .././libiberty/libiberty.a -lm
lto1: error: caller edge count is negative
vec_heap_p_reserve.constprop.79/580 @0x7fc29b280480 (asm:
vec_heap_p_reserve.constprop.79) (clone of vec_heap_p_reserve/537)
availability:local analyzed executed 16253965x reachable local finalized
  called by: VEC_alt_state_t_heap_reserve/146 VEC_ainsn_t_heap_reserve/140
VEC_state_t_heap_reserve/133 (13343217x) (0.00 per call)
VEC_decl_t_heap_reserve/124 (0.01 per call) VEC_reserv_sets_t_heap_reserve/118
(1.00 per call) VEC_unit_usage_t_heap_reserve/110 (0.05 per call)
VEC_alt_state_t_heap_reserve.3046.constprop.49/451
VEC_ainsn_t_heap_reserve.3003.constprop.55/523
VEC_state_t_heap_reserve.2975.constprop.59/534 (-6021287x) (0.00 per call)
VEC_decl_t_heap_reserve.2922.constprop.65/545 (179900x) (0.01 per call)
VEC_reserv_sets_t_heap_reserve.2884.constprop.70/550 (1.00 per call)
VEC_unit_usage_t_heap_reserve.2828.constprop.78/578 (8752135x) (0.05 per call)
  calls: vec_heap_o_reserve_1.4977.constprop.80/581 (850115x) (1.00 per call)
  References:
  Refering this function:
lto1: internal compiler error: verify_cgraph_node failed

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[markus at trippelsdorf dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #11 from Markus Trippelsdorf <markus at trippelsdorf dot de> 2011-07-20 17:48:34 UTC ---
And it's indeed caused by the new IPA-CP code (Revision 176424,
commit 821d0e0f73d79232eb827c3988c34d5a1fbeb422).
Reverting the commit "solves" the issue.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

Ian Lance Taylor <ian at airs dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2011.07.20 22:03:04
         AssignedTo|unassigned at gcc dot       |jamborm at gcc dot gnu.org
                   |gnu.org                     |
     Ever Confirmed|0                           |1

--- Comment #12 from Ian Lance Taylor <ian at airs dot com> 2011-07-20 22:03:04 UTC ---
I was able to recreate the problem.  The count is going negative at line 1926
of ipa-cp.c, the first line setting cs->count here:

  for (cs = new_node->callees; cs ; cs = cs->next_callee)
    if (cs->frequency)
      cs->count = cs->count * new_sum / orig_node_count;
    else
      cs->count = 0;

In the test case, cs->count, new_sum, and orig_node_count all == 3870557758. 
Computing 3870557758 * 3870557758 overflows to a negative number.  Dividing by
3870557758 leaves the number still negative.  This then leads to the failure.

For reference, profile_info->sum_max == 2619109064700.

I don't know if these numbers are plausible.

I think that this code probably needs to be written along the lines of
scale_bbs_frequencies_gcov_type.  I will leave this for Martin.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[jamborm at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #13 from Martin Jambor <jamborm at gcc dot gnu.org> 2011-07-21 13:00:31 UTC ---
The problem in the summary is a call graph verification error, which
is most likely a duplicate of PR 49796 (an infrastructure/verifier
problem really) while the problem reported in comment #10 is "caller
edge count is negative" which is most certainly a different error.

I ran a profiled LTO bootstrap overnight but unfortunately could not
reproduce neither.  Nevertheless, I will try to follow Ian's
suggestion.

Ian, was there anything specific that you did in order to reproduce
the issue?  Thanks.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[markus at trippelsdorf dot de]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #14 from Markus Trippelsdorf <markus at trippelsdorf dot de> 2011-07-21 13:10:36 UTC ---
H.J. also reported:
lto1: error: caller edge count is negative
in the description. So it's likely the same issue
as in comment #10.

BTW the following already fixes the problem for me:
-      cs->count = cs->count * new_sum / orig_node_count;
+      cs->count = cs->count * (new_sum / orig_node_count);

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #15 from Ian Lance Taylor <ian at airs dot com> 2011-07-21 19:56:28 UTC ---
Martin: I was able to reproduce it by configuring using --with-ld to point to a
newly built version of gold configured with --enable-plugins.

Markus: That patch will fix the problem but will do the wrong thing in many
cases, because the division will truncate.

My exact gcc configure line:

--enable-clocale=gnu' '--with-system-zlib' '--enable-shared'
'--with-demangler-in-ld' '--enable-cloog-backend=isl'
'--with-ppl=/home/iant/gnu/ppl-0.11-install'
'--with-cloog=/home/iant/gnu/cloog-0.16.2-install'
'--with-build-config=bootstrap-lto' '--with-fpmath=sse'
'--with-ld=/usr/local/google/iant/gold/gold-objdir/gold/ld-new'
'--enable-languages=c,c++,fortran,java,lto,objc

The ld-new mentioned in the --with-ld option was mainline binutils configured
with:

--enable-gold --enable-plugins

The ppl and cloog mentioned in the gcc configure line were downloaded from
ftp://gcc.gnu.org/pub/gcc/infrastructure.

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[ian at airs dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #16 from Ian Lance Taylor <ian at airs dot com> 2011-07-21 19:56:59 UTC ---
Forgot to mention that I ran "make -j6 profiledbootstrap".

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #17 from H.J. Lu <hjl.tools at gmail dot com> 2011-07-22 17:39:59 UTC ---
It isn't fixed by 176630:

http://gcc.gnu.org/ml/gcc-cvs/2011-07/msg00897.html

See:

http://gcc.gnu.org/ml/gcc-regression/2011-07/msg00403.html

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[jamborm at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #18 from Martin Jambor <jamborm at gcc dot gnu.org> 2011-07-26 12:26:32 UTC ---
Proposed fix posted to the mailing list:
http://gcc.gnu.org/ml/gcc-patches/2011-07/msg02247.html

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[jamborm at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

--- Comment #19 from Martin Jambor <jamborm at gcc dot gnu.org> 2011-07-26 12:27:01 UTC ---
Author: jamborm
Date: Tue Jul 26 12:26:58 2011
New Revision: 176789

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=176789
Log:
2011-07-26  Martin Jambor  <mjambor@suse.cz>

    PR bootstrap/49786
    * ipa-cp.c (update_profiling_info): Avoid overflow when updating
    counts.
    (update_specialized_profile): Likewise.


Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/ipa-cp.c

[Bug bootstrap/49786] [4.7 Regression] bootstrap failed with bootstrap-profiled

[jamborm at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49786

Martin Jambor <jamborm at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #20 from Martin Jambor <jamborm at gcc dot gnu.org> 2011-07-26 12:29:21 UTC ---
Fixed.