[Bug bootstrap/38788] New: gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[Bug bootstrap/38788] New: gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[rob1weld at aol dot com]

I'm building gcc 4.4.0 20090109 on i386-pc-solaris2.11 (OpenSolaris 2008.11)
using gcc version 4.4.0 20090104. I added "--enable-intermodule" to my list
of configure options and 


# prev-gcc/xgcc -v
Using built-in specs.
Target: i386-pc-solaris2.11
Configured with: ../gcc_trunk/configure
--enable-languages=ada,c,c++,fortran,java,objc,obj-c++
--enable-shared --disable-static --enable-decimal-float --with-long-double-128
--enable-nls --with-included-gettext --enable-gather-detailed-mem-stats
--with-stabs --enable-debug -enable-largefile --enable-symvers
--without-system-zlib --enable-gtk-cairo --enable-qt-peer --enable-xmlj
--enable-gconf-peer --enable-gjdoc --enable-java-awt=gtk,xlib,qt,x
--enable-gc-debug --enable-libgcj-multifile --enable-libgcj-debug
--enable-objc-gc --enable-libstdcxx-debug --enable-stage1-checking
--enable-checking=release --enable-intermodule --without-system-libunwind
--with-gnu-as --with-as=/usr/local/bin/as --with-gnu-ld
--with-ld=/usr/local/bin/ld
Thread model: posix
gcc version 4.4.0 20090109 (experimental) (GCC) 

# gmake
...
/bin/sh ../../gcc_trunk/gcc/../move-if-change tmp-mlib.h multilib.h
echo timestamp > s-mlib
lsf="../../gcc_trunk/gcc/ada/gcc-interface/lang-specs.h
../../gcc_trunk/gcc/cp/lang-specs.h ../../gcc_trunk/gcc/fortran/lang-specs.h
../../gcc_trunk/gcc/java/lang-specs.h ../../gcc_trunk/gcc/objc/lang-specs.h
../../gcc_trunk/gcc/objcp/lang-specs.h"; for f in $lsf; do \
            echo "#include \"$f\""; \
        done | sed 's|../../gcc_trunk/gcc/||' > tmp-specs.h
/bin/sh ../../gcc_trunk/gcc/../move-if-change tmp-specs.h specs.h
echo timestamp > s-specs
/usr/share/src/gcc_build/./prev-gcc/xgcc -B/usr/share/src/gcc_build/./prev-gcc/
-B/usr/local/i386-pc-solaris2.11/bin/ -c  -g -O2 -fprofile-generate -DIN_GCC  
-W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wcast-qual
-Wold-style-definition -Wc++-compat -Wmissing-format-attribute -pedantic
-Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror  
-DHAVE_CONFIG_H -I. -I. -I../../gcc_trunk/gcc -I../../gcc_trunk/gcc/.
-I../../gcc_trunk/gcc/../include -I./../intl
-I../../gcc_trunk/gcc/../libcpp/include  -I../../gcc_trunk/gcc/../libdecnumber
-I../../gcc_trunk/gcc/../libdecnumber/dpd -I../libdecnumber 
-DCLOOG_PPL_BACKEND  -fexceptions ../../gcc_trunk/gcc/c-pretty-print.c -o
c-pretty-print.o
/usr/share/src/gcc_build/./prev-gcc/xgcc -B/usr/share/src/gcc_build/./prev-gcc/
-B/usr/local/i386-pc-solaris2.11/bin/  -g -O2 -fprofile-generate -DIN_GCC   -W
-Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wcast-qual
-Wold-style-definition -Wc++-compat -Wmissing-format-attribute -pedantic
-Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror  
-DHAVE_CONFIG_H -I. -I. -I../../gcc_trunk/gcc -I../../gcc_trunk/gcc/.
-I../../gcc_trunk/gcc/../include -I./../intl
-I../../gcc_trunk/gcc/../libcpp/include  -I../../gcc_trunk/gcc/../libdecnumber
-I../../gcc_trunk/gcc/../libdecnumber/dpd -I../libdecnumber 
-DCLOOG_PPL_BACKEND  -fexceptions \
          -DTARGET_NAME=\"i386-pc-solaris2.11\" \
          -DLOCALEDIR=\"/usr/local/share/locale\" \
          -c insn-attrtab.c insn-automata.c (HUGE LIST) ...
../../gcc_trunk/gcc/config/i386/i386.c -o libbackend.o  \
          -DBASEVER="\"4.4.0\"" -DDATESTAMP="\" 20090109\"" \
          -DREVISION= \
          -DDEVPHASE="\" (experimental)\"" -DPKGVERSION="\"(GCC) \"" \
          -DBUGURL="\"<http://gcc.gnu.org/bugs.html>\"" -combine
cc1: warnings being treated as errors
../../gcc_trunk/gcc/bitmap.c: In function 'bitmap_obstack_alloc_stat':
../../gcc_trunk/gcc/bitmap.c:359: error: request for implicit conversion from
'void *' to 'struct bitmap_head_def *' not permitted in C++
../../gcc_trunk/gcc/bitmap.c: In function 'bitmap_obstack_free':
../../gcc_trunk/gcc/bitmap.c:394: error: request for implicit conversion from
'void *' to 'struct bitmap_element *' not permitted in C++
../../gcc_trunk/gcc/dfp.c: In function 'encode_decimal32':
../../gcc_trunk/gcc/dfp.c:142: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c: In function 'decode_decimal32':
../../gcc_trunk/gcc/dfp.c:158: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c: In function 'encode_decimal64':
../../gcc_trunk/gcc/dfp.c:182: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c:188: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c: In function 'decode_decimal64':
../../gcc_trunk/gcc/dfp.c:207: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c:212: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c: In function 'encode_decimal128':
../../gcc_trunk/gcc/dfp.c:238: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c:248: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c: In function 'decode_decimal128':
../../gcc_trunk/gcc/dfp.c:267: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dfp.c:274: error: dereferencing type-punned pointer will
break strict-aliasing rules
../../gcc_trunk/gcc/dominance.c: In function 'iterate_fix_dominators':
../../gcc_trunk/gcc/dominance.c:1324: error: request for implicit conversion
from 'void *' to 'bitmap' not permitted in C++

cc1: out of memory allocating 12 bytes after a total of 347045888 bytes
gmake[3]: *** [libbackend.o] Error 1
gmake[3]: Leaving directory `/usr/share/src/gcc_build/gcc'
gmake[2]: *** [all-stageprofile-gcc] Error 2
gmake[2]: Leaving directory `/usr/share/src/gcc_build'
gmake[1]: *** [stageprofile-bubble] Error 2
gmake[1]: Leaving directory `/usr/share/src/gcc_build'
gmake: *** [all] Error 2

It seems there is little chance of "--enable-intermodule" configure option
working with the "-Werror" gcc option unless _all_ the files pass the
"-Werror" tests. 

It may be that "--enable-intermodule" would exceed the memory of any
reasonable Operating System (and it's hardware) and thus the configure 
option "--enable-intermodule" is not even a usable option. 

In it's current implementation within gcc it can only crash the build,
eventually, for most people. I can see it failing if it gets as far as 
Java. Don't tell us "all you need is 128G of memory to compile it...".


One way to fix this would be to allow a parameter to the gcc/configure
"--enable-intermodule" option that would specify memory available. 

The program could make a simple tree of files, chop it in half and retry 
the two parts if memory usage exceeded the desired amount. This would also 
allow use of the feature without bringing the System to it's knees.


Another possibility would be to add an "--exclude-from-intermodule=" option.
This could take either a single file, an "@file" list, or wildcards. The
list would consist of file to be excluded from being added to the "intermodule
list" and would simply be compiled normally.


Another possibility would be to add an "--assign-to-intermodule-1=" option.
This could take either a single file, an "@file" list, or wildcards. The
list would consist of file to be added to the "intermodule list" number one.
The user could add other files to "--assign-to-intermodule-2=", etc., and
any files excountered without an "--assign-to-intermodule-x=" option would
simply be compiled normally.


As it stands the "--enable-intermodule" is too likely to run into an "-Werror"
_or_ run the user out of memory, _most_ of the time. This makes it of limited
use in this situation. We should make it more usable (bring it up to "usable").

Rob


-- 
           Summary: gcc 4.4.0 20090109 - Configure with "--enable-
                    intermodule" breaks build
           Product: gcc
           Version: 4.4.0
            Status: UNCONFIRMED
          Severity: major
          Priority: P3
         Component: bootstrap
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: rob1weld at aol dot com
 GCC build triplet: i386-pc-solaris2.11
  GCC host triplet: i386-pc-solaris2.11
GCC target triplet: i386-pc-solaris2.11


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[pinskia at gcc dot gnu dot org]

------- Comment #1 from pinskia at gcc dot gnu dot org  2009-01-09 21:11 -------
--enable-intermodule is not well tested at all.  And second --combine is way
broken and really should be removed along with --enable-intermodule.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |normal


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[rob1weld at aol dot com]

------- Comment #2 from rob1weld at aol dot com  2009-01-09 21:23 -------
Perhaps if the "-combine" command for gcc had a complementary "-split="
command that would attempt to make optimal splits of the tree at "split"
sized chunks.


-- 

rob1weld at aol dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |major


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[pinskia at gcc dot gnu dot org]

------- Comment #3 from pinskia at gcc dot gnu dot org  2009-01-09 21:28 -------
You told the compiler to compile everything at the same time what do you expect
to cause out of memory issues?


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|major                       |normal


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[rob1weld at aol dot com]

------- Comment #4 from rob1weld at aol dot com  2009-01-09 22:16 -------
(In reply to comment #3)
> You told the compiler to compile everything at the same time what do
> you expect to cause out of memory issues?

It could create temporary files (like qsort) and shrink the size of
the file (by optimizing) until it had done what it could. It could
then cut the tree in half (find two "unions of functions" connected
by a single point) and do each half with a single jump from one
section to the other.

If the _only_ thing it can do is "grab every file in the directory"
then it will fail in many of gcc's directories and is unlikely ever
to complete a build of all languanges, let alone just "C".

I did not expect that "-combine" would be little better than cat'ting the
source files together and compiling them (as we do that a fair bit already).

Rob


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[pinskia at gcc dot gnu dot org]

------- Comment #5 from pinskia at gcc dot gnu dot org  2009-01-09 22:39 -------
That is the whole idea of http://gcc.gnu.org/wiki/LinkTimeOptimization :).


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[rob1weld at aol dot com]

------- Comment #6 from rob1weld at aol dot com  2009-01-10 05:56 -------
(In reply to comment #1)
>> --enable-intermodule is not well tested at all.  And second --combine is way
>> broken and really should be removed along with --enable-intermodule.
> ("one of Rob's comments")
>> That is the whole idea of http://gcc.gnu.org/wiki/LinkTimeOptimization :).

Look what's coming ... LTO's brother is ...

Whole Program Optimizer
http://gcc.gnu.org/wiki/whopr


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[pinskia at gcc dot gnu dot org]

------- Comment #7 from pinskia at gcc dot gnu dot org  2009-01-13 00:31 -------
Nobody builds using --enable-intermodule, it uses too much memory in general
anyways so closing as won't fix.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |WONTFIX


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[rob1weld at aol dot com]

------- Comment #8 from rob1weld at aol dot com  2009-01-13 14:58 -------
(In reply to comment #7)
> Nobody builds using --enable-intermodule, it uses too much memory in general
> anyways so closing as won't fix.
> 

We can "fix" it by reopening this bug and removing --enable-intermodule .

It is simply not possible for it to work, it makes no sense to have it (unless
you want to backport it to gcc 2.x (were it might actually work)).


-- 

rob1weld at aol dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |UNCONFIRMED
         Resolution|WONTFIX                     |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[jakub at gcc dot gnu dot org]

------- Comment #9 from jakub at gcc dot gnu dot org  2009-01-13 15:09 -------
Reopening a bug doesn't "fix" anything.  Patches do.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[pinskia at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|                            |WONTFIX
   Target Milestone|---                         |4.7.0

--- Comment #11 from Andrew Pinski <pinskia at gcc dot gnu.org> 2012-02-03 23:12:31 UTC ---
-combine has been removed and there is now a way to do bootstrap with LTO so
closing as won't fix.

[Bug bootstrap/38788] gcc 4.4.0 20090109 - Configure with "--enable-intermodule" breaks build

[gseanmcg at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38788

Sean McGovern <gseanmcg at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gseanmcg at gmail dot com

--- Comment #10 from Sean McGovern <gseanmcg at gmail dot com> 2010-12-09 17:56:45 UTC ---
Still any interest in removing this option?