[Bug bootstrap/54304] New: linking stage picks up system mpfr instead of in-tree version

[Bug bootstrap/54304] New: linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

             Bug #: 54304
           Summary: linking stage picks up system mpfr instead of in-tree
                    version
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: bootstrap
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: tobi@gcc.gnu.org


This is an old bug, I found mention of it all over the internet, and also on
the gcc mailing list <http://gcc.gnu.org/ml/gcc-help/2011-04/msg00126.html>.

What happens is that with an in-tree gmp, mpfr, mpc (I used
contrib/download_prerequisites) the linking stage picks up the system libraries
instead of the in-tree versions.  This causes a problem when the system libmpfr
< 3.0.0 and the in-tree version is >= 3.0.0 and vice versa, because
mpfr_get_z_exp was renamed to mpfr_get_z_2exp between the two (hidden behind a
macro).  The error I'm seeing with today's tree is:

/Users/tobi/src/gcc/build/./prev-gcc/g++
-B/Users/tobi/src/gcc/build/./prev-gcc/
-B/usr/local/x86_64-apple-darwin11.4.0/bin/ -nostdinc++
-B/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/src/.libs
-B/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/libsupc++/.libs
-I/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/include/x86_64-apple-darwin11.4.0
-I/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/include
-I/Users/tobi/src/gcc/libstdc++-v3/libsupc++
-L/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/src/.libs
-L/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/libsupc++/.libs
  -g -O2 -mdynamic-no-pic -gtoggle -DIN_GCC   -fno-exceptions -fno-rtti -W
-Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute
-pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror
-fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc -Wl,-no_pie -o
f951 \
                fortran/arith.o fortran/array.o fortran/bbt.o fortran/check.o
fortran/class.o fortran/constructor.o fortran/cpp.o fortran/data.o
fortran/decl.o fortran/dump-parse-tree.o fortran/error.o fortran/expr.o
fortran/interface.o fortran/intrinsic.o fortran/io.o fortran/iresolve.o
fortran/match.o fortran/matchexp.o fortran/misc.o fortran/module.o
fortran/openmp.o fortran/options.o fortran/parse.o fortran/primary.o
fortran/resolve.o fortran/scanner.o fortran/simplify.o fortran/st.o
fortran/symbol.o fortran/target-memory.o darwin-f.o fortran/convert.o
fortran/dependency.o fortran/f95-lang.o fortran/trans.o fortran/trans-array.o
fortran/trans-common.o fortran/trans-const.o fortran/trans-decl.o
fortran/trans-expr.o fortran/trans-intrinsic.o fortran/trans-io.o
fortran/trans-openmp.o fortran/trans-stmt.o fortran/trans-types.o
fortran/frontend-passes.o libbackend.a main.o tree-browser.o libcommon-target.a
libcommon.a ../libcpp/libcpp.a ../libdecnumber/libdecnumber.a libcommon.a
../libcpp/libcpp.a -lintl -L/opt/local/lib -liconv  ../libiberty/libiberty.a
../libdecnumber/libdecnumber.a  attribs.o 
-L/Users/tobi/src/gcc/build/./gmp/.libs
-L/Users/tobi/src/gcc/build/./mpfr/.libs
-L/Users/tobi/src/gcc/build/./mpc/src/.libs -lmpc -lmpfr -lgmp   -L../zlib -lz
Undefined symbols for architecture x86_64:
  "_mpfr_get_z_exp", referenced from:
      gfc_mpfr_to_mpz(__mpz_struct*, __mpfr_struct*, locus*)     in arith.o
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[3]: *** [f951] Error 1
make[2]: *** [all-stage2-gcc] Error 2
make[1]: *** [stage2-bubble] Error 2
make: *** [all] Error 2

(I mentioned this in the aftermath to PR54292)

Cheers

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[pinskia at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #1 from pinskia at gmail dot com <pinskia at gmail dot com> 2012-08-17 19:13:50 UTC ---
This is a darwin only issue.
On Aug 17, 2012 12:07 PM, "tobi at gcc dot gnu.org" <
gcc-bugzilla@gcc.gnu.org> wrote:

> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304
>
>              Bug #: 54304
>            Summary: linking stage picks up system mpfr instead of in-tree
>                     version
>     Classification: Unclassified
>            Product: gcc
>            Version: 4.8.0
>             Status: UNCONFIRMED
>           Severity: normal
>           Priority: P3
>          Component: bootstrap
>         AssignedTo: unassigned@gcc.gnu.org
>         ReportedBy: tobi@gcc.gnu.org
>
>
> This is an old bug, I found mention of it all over the internet, and also
> on
> the gcc mailing list <http://gcc.gnu.org/ml/gcc-help/2011-04/msg00126.html
> >.
>
> What happens is that with an in-tree gmp, mpfr, mpc (I used
> contrib/download_prerequisites) the linking stage picks up the system
> libraries
> instead of the in-tree versions.  This causes a problem when the system
> libmpfr
> < 3.0.0 and the in-tree version is >= 3.0.0 and vice versa, because
> mpfr_get_z_exp was renamed to mpfr_get_z_2exp between the two (hidden
> behind a
> macro).  The error I'm seeing with today's tree is:
>
> /Users/tobi/src/gcc/build/./prev-gcc/g++
> -B/Users/tobi/src/gcc/build/./prev-gcc/
> -B/usr/local/x86_64-apple-darwin11.4.0/bin/ -nostdinc++
>
> -B/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/src/.libs
>
> -B/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/libsupc++/.libs
>
> -I/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/include/x86_64-apple-darwin11.4.0
>
> -I/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/include
> -I/Users/tobi/src/gcc/libstdc++-v3/libsupc++
>
> -L/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/src/.libs
>
> -L/Users/tobi/src/gcc/build/prev-x86_64-apple-darwin11.4.0/libstdc++-v3/libsupc++/.libs
>   -g -O2 -mdynamic-no-pic -gtoggle -DIN_GCC   -fno-exceptions -fno-rtti -W
> -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute
> -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings
> -Werror
> -fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc -Wl,-no_pie
> -o
> f951 \
>                 fortran/arith.o fortran/array.o fortran/bbt.o
> fortran/check.o
> fortran/class.o fortran/constructor.o fortran/cpp.o fortran/data.o
> fortran/decl.o fortran/dump-parse-tree.o fortran/error.o fortran/expr.o
> fortran/interface.o fortran/intrinsic.o fortran/io.o fortran/iresolve.o
> fortran/match.o fortran/matchexp.o fortran/misc.o fortran/module.o
> fortran/openmp.o fortran/options.o fortran/parse.o fortran/primary.o
> fortran/resolve.o fortran/scanner.o fortran/simplify.o fortran/st.o
> fortran/symbol.o fortran/target-memory.o darwin-f.o fortran/convert.o
> fortran/dependency.o fortran/f95-lang.o fortran/trans.o
> fortran/trans-array.o
> fortran/trans-common.o fortran/trans-const.o fortran/trans-decl.o
> fortran/trans-expr.o fortran/trans-intrinsic.o fortran/trans-io.o
> fortran/trans-openmp.o fortran/trans-stmt.o fortran/trans-types.o
> fortran/frontend-passes.o libbackend.a main.o tree-browser.o
> libcommon-target.a
> libcommon.a ../libcpp/libcpp.a ../libdecnumber/libdecnumber.a libcommon.a
> ../libcpp/libcpp.a -lintl -L/opt/local/lib -liconv
>  ../libiberty/libiberty.a
> ../libdecnumber/libdecnumber.a  attribs.o
> -L/Users/tobi/src/gcc/build/./gmp/.libs
> -L/Users/tobi/src/gcc/build/./mpfr/.libs
> -L/Users/tobi/src/gcc/build/./mpc/src/.libs -lmpc -lmpfr -lgmp   -L../zlib
> -lz
> Undefined symbols for architecture x86_64:
>   "_mpfr_get_z_exp", referenced from:
>       gfc_mpfr_to_mpz(__mpz_struct*, __mpfr_struct*, locus*)     in arith.o
> ld: symbol(s) not found for architecture x86_64
> collect2: error: ld returned 1 exit status
> make[3]: *** [f951] Error 1
> make[2]: *** [all-stage2-gcc] Error 2
> make[1]: *** [stage2-bubble] Error 2
> make: *** [all] Error 2
>
> (I mentioned this in the aftermath to PR54292)
>
> Cheers
>

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

Tobias Schlüter <tobi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
               Host|                            |darwin

--- Comment #2 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 19:24:36 UTC ---
Sorry, forgot to fill in the field.

One additional piece of information: note that this certainly applies to GMP
and MPC as well, but their ABIs didn't change, so the error happens silently.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[glisse at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #3 from Marc Glisse <glisse at gcc dot gnu.org> 2012-08-17 19:27:08 UTC ---
When you say: "the system libraries", where are they? /opt/local/lib?

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #4 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 19:29:56 UTC ---
BTW PR50914, though closed as invalid, indicates that similar mixups can also
happen on i686-linux.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #5 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 19:31:37 UTC ---
It seems I have them in /usr/lib and in /opt/local/lib.  How can I find out
which one it picks up?

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #6 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 19:33:44 UTC ---
Sorry, my mistake, it's ~/usr/lib not /usr/lib.  I'll have to find out if that
is included in any search path (I doubt it).  About the one in /opt/local/lib I
wasn't mistaken.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #7 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 19:35:30 UTC ---
Ok, the one in ~/usr/lib is 2.2.1, so this error wouldn't appear if it got
picked up.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #8 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 19:38:54 UTC ---
Sorry, didn't mean to hit "Save Changes" yet.

In the past I had to configure with
  ../configure --with-libiconv-prefix=/opt/local --enable-languages=fortran
because otherwise the build breaks with iconv issues (also something you find
all over google).  And now I see why you asked: this of course puts the system
gmp etc in the search path ahead of the ones built in-tree.  Not a desirable
state admittedly.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[glisse at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #9 from Marc Glisse <glisse at gcc dot gnu.org> 2012-08-17 19:40:19 UTC ---
Well, there is the usual problem that if you use external libraries A and B
that are both installed in both path1 and path2 and you want A from path1 and B
from path2, that's not possible.

The difference here is that we know in-tree locations are safe (no unwanted
library in there). So instead of just putting -I and -L flags to these
locations as they appear, we should put the safe ones in front. That
complicates things a bit...

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #10 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 20:56:40 UTC ---
You don't need the -L flags for the in-tree libraries at all as you could just
put, say, "path_to_mpfr/libmpfr.a" on the command line.  I understand that this
might need some major Makefile surgery.

Anyway, a build without --with-libiconv-prefix=/opt/local succeeded, I have
updated my system in the meantime, so I can't investigate this further.  (The
iconv issue was of the type described e.g. here
<https://bugs.php.net/bug.php?id=52611>).

Anyway, I think this can be closed as INVALID unless you find the suggestion to
do away with -L for in-tree stuff worthwhile.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[glisse at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #11 from Marc Glisse <glisse at gcc dot gnu.org> 2012-08-17 21:21:57 UTC ---
(In reply to comment #10)
> You don't need the -L flags for the in-tree libraries at all as you could just
> put, say, "path_to_mpfr/libmpfr.a" on the command line.  I understand that this
> might need some major Makefile surgery.

That wouldn't be so hard, but it doesn't help for -I. And it seems preferable
to handle -I and -L in the same way so they have less chances of disagreeing
(which they sadly do here).

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #12 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 21:35:50 UTC ---
Well, directories given by -I are searched before system directories, -L adds
directories to the end of the library search list, so their functions are not
equivalent.  Which is the root of my problem, and which will always lead to
conflicts if a library is installed both in a system dir and under some prefix
specified via a --with-lib-prefix= option.  Ugly.

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[glisse at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #13 from Marc Glisse <glisse at gcc dot gnu.org> 2012-08-17 22:06:07 UTC ---
(In reply to comment #12)
> Well, directories given by -I are searched before system directories,

Yes (system directories are those given by -isystem or hardcoded in the
compiler).

> -L adds directories to the end of the library search list,

No, there are also system directories afterwards.

Not that system directories are relevant here...

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[tobi at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #14 from Tobias Schlüter <tobi at gcc dot gnu.org> 2012-08-17 22:25:15 UTC ---
Ah, true, further down in the ld manpage this is stated clearly.  Thanks!

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[egall at gwmail dot gwu.edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

Eric Gallager <egall at gwmail dot gwu.edu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |egall at gwmail dot gwu.edu

--- Comment #15 from Eric Gallager <egall at gwmail dot gwu.edu> ---
Still happens with current gcc trunk (i.e. gcc 6.0)...

[Bug bootstrap/54304] linking stage picks up system mpfr instead of in-tree version

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54304

--- Comment #16 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Just to let everyone know this is how Darin's ld work. It searches for shared
libraries through all of the library patches and then if it does not find one,
it searches for a static library. So for Darwin, you need to supply the full
path to the static library or name the static library slightly different. 

This behavior is different from all other Unix ld's too. It might be too hard
to solve and really happens with projects too where it depends on standard Unix
ld behavior.