[Bug driver/35532] New: Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[Bug driver/35532] New: Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[gschafer at zip dot com dot au]

The cause is this patch which was designed to fix behaviour of relocated 
compilers:

  http://gcc.gnu.org/ml/gcc/2006-10/msg00280.html

Unfortunately, it has the side-effect of breaking existing build scripts for 
*NON* relocated compilers which have relied on this behaviour for years.

Contrived procedure to demonstrate the problem (i686-linux):

  1. build and install Glibc --prefix=/tmp/foo

  2. build and install GCC --prefix=/tmp/foo but prevent startfiles from being
     found on the host by overriding relevant standard macros eg:

echo '
/* Remove /lib and /usr/lib from startfiles search path.  */
#define STANDARD_STARTFILE_PREFIX_1 ""
#define STANDARD_STARTFILE_PREFIX_2 ""' >> gcc/config/i386/linux.h

Above GCC build works fine with GCC-4.2.3 but fails with GCC-4.3.0 when linking
libgcc_s.so:

/usr/bin/ld: crti.o: No such file: No such file or directory

Essentially, the problem is that xgcc thinks it is a relocated compiler when it 
runs from $objdir eg: when building libgcc* and other target libs, when in
actual fact, it is not really a true relocated compiler at all.

Attached patch restores the old behaviour, but I'm not proposing it be applied.
I suspect that GCC needs a way to distinguish when it is being run from $objdir
and when it is truly a relocated compiler. GCC could then behave like it always
has and not break the many build procedures that rely on the current behaviour, 
and truly relocated compilers could also do the right thing.

More background in the thread starting here:

  http://gcc.gnu.org/ml/gcc/2008-03/msg00095.html


-- 
           Summary: Native GCC no longer searches $prefix/lib for startfiles
                    when run from $objdir
           Product: gcc
           Version: 4.3.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: driver
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: gschafer at zip dot com dot au


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[gschafer at zip dot com dot au]

------- Comment #1 from gschafer at zip dot com dot au  2008-03-10 22:37 -------
Created an attachment (id=15292)
 --> (http://gcc.gnu.org/bugzilla/attachment.cgi?id=15292&action=view)
Patch that restores old behaviour (for demonstration only)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[pinskia at gcc dot gnu dot org]

------- Comment #2 from pinskia at gcc dot gnu dot org  2008-03-10 22:39 -------
It is better to use -B $prefix/lib while building.
And seriously I think you are doing something wrong when you edit the files as
you did.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[gschafer at zip dot com dot au]

------- Comment #3 from gschafer at zip dot com dot au  2008-03-10 22:52 -------
(In reply to comment #2)
> It is better to use -B $prefix/lib while building.

-B doesn't work on multilib ie: -B $prefix/lib doesn't find
$prefix/lib/../lib64

Not only that, I'm talking about GCC_FOR_TARGET, your suggestion doesn't help.

> And seriously I think you are doing something wrong when you edit the files as
> you did.

Then you misunderstand the problem. It is a contrived example for demonstrating
the real problem.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[carlos at codesourcery dot com]

------- Comment #4 from carlos at codesourcery dot com  2008-03-11 19:21 -------
Greg,

The example you describe looks an awful lot like a cross-compile. Is there
anything preventing you from configuring with --enable-build-sysroot=/tmp/foo?

Could you also describe your original build process in detail?

Please don't think of this as an issue between a relocated or un-relocated
compiler. GCC took the kitchen sink approach to search directories, and the
patch in question rooted out exactly which directories are needed and which are
not.

Now that we've cleaned up the search directories, you are going to have to
prove why other build processes are *not* usable in your particular scenario. I
won't accept "It used to build and now it doesn't." By building gcc you become
a gcc developer, not a user, and there is no guarantee that the gcc build
process will not change over time.

I look forward to the sordid details of your build problem :-)


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[gschafer at zip dot com dot au]

------- Comment #5 from gschafer at zip dot com dot au  2008-03-12 11:26 -------
(In reply to comment #4)
> 
> The example you describe looks an awful lot like a cross-compile.

No, it's definitely native. See below.

> Is there
> anything preventing you from configuring with --enable-build-sysroot=/tmp/foo?

Sysroot options are only for cross-compiles. GCC docs are quite clear on this.
I tried it anyway and no, it doesn't help.

> Could you also describe your original build process in detail?

The build process is the same old process that has been is use for years at
http://www.linuxfromscratch.org/ and my project at http://www.diy-linux.org/
It's a *native* build process for a reason. Carlos, what you must keep in mind
is that these build procedures are aimed at relatively mere mortals and not
toolchain gurus like yourself. The goal is to natively bootstrap a complete
Linux system from source and the basic theory of operation is summarized here:

http://www.diy-linux.org/reference-build/introduction.html#buildmethod

Because the first phase is installed into a non-standard prefix, say
/temptools, the --prefix plays a pivotal role in the process. Up until 4.3, GCC
has always (mostly) respected $prefix. I say "mostly" because a few tweaks are
required for the native toolchain to function correctly when installed in
/temptools, including some hacks to prevent the toolchain from searching the
host. With your patch, GCC no longer treats $prefix like it used to. But it's
only during the GCC_FOR_TARGET stage of the GCC build. Once GCC is installed
into /temptools, everything works fine! But your patch breaks the assumption
about $prefix that these procedures have relied upon for years. In particular,
building a 64-bit system from a 32-bit host breaks horribly (ie: build a
cross-toolchain, cross-compile Glibc, cross-compile a 64-bit kernel, reboot
into it, carry on as 64-bit native).

> Please don't think of this as an issue between a relocated or un-relocated
> compiler.

But it clearly *is*, I don't understand how you can say this. gcc_exec_prefix
is key AFAICT,

> By building gcc you become a gcc developer, not a user,

Sorry, but that's nonsense.

> and there is no guarantee that the gcc build
> process will not change over time.

Of course. But may I draw your attention to this snippet from the GCC
installation docs:

"GCC automatically searches for ordinary libraries using GCC_EXEC_PREFIX. Thus,
when the same installation prefix is used for both GCC and packages, GCC will
automatically search for both headers and libraries. This provides a
configuration that is easy to use. GCC behaves in a manner similar to that when
it is installed as a system compiler in /usr."

The above now seems at odds with GCC's current behaviour.

Anyhow, if you can help me work around the problem and/or come up with a better
build procedure for the "mere mortals" target audience, I'd appreciate it.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[carlos at codesourcery dot com]

------- Comment #6 from carlos at codesourcery dot com  2008-03-14 13:26 -------
Greg,

As a workaround can you try using all of the sysroot framework? Configure with
--with-sysroot=/mnt/foo and --with-build-sysroot=/mnt/foo. Build with
LDFLAGS_FOR_TARGET="--sysroot=/mnt/foo" and
CPPFLAGS_FOR_TARGET="--sysroot=/mnt/foo".

I'm reading through the DIY instructions right now to make sense of your
bootstrap process.


-- 

carlos at codesourcery dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carlos at codesourcery dot
                   |                            |com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[gschafer at zip dot com dot au]

------- Comment #7 from gschafer at zip dot com dot au  2008-03-16 06:41 -------
(In reply to comment #6)
> As a workaround can you try using all of the sysroot framework?

Thanks for looking at this Carlos. But the sysroot stuff is not really suited
to a non /usr layout. For example, with my --prefix=/temptools scenario, the
sysrooted toolchain will go looking for:

/temptools/usr/include
/temptools/usr/lib
/temptools/lib

whereas my layout is:

/temptools/include
/temptools/lib

And just to reiterate, according to  http://gcc.gnu.org/install/configure.html
the sysroot options apply *only* when building cross compilers. I know that
Alan Modra said he now builds sysrooted compilers on native hosts, so maybe the
GCC install docs need to be changed?

> Configure with
> --with-sysroot=/mnt/foo and --with-build-sysroot=/mnt/foo. Build with
> LDFLAGS_FOR_TARGET="--sysroot=/mnt/foo" and
> CPPFLAGS_FOR_TARGET="--sysroot=/mnt/foo".

Ok, I went ahead and tried with the above. The build system appears not to care
that the sysroot framework was in use on a native build. Great.

However, it doesn't work for my scenario. The build crapped out somewhere in
fixincludes:

The directory that should contain system headers does not exist:
  /temptools/usr/include
make[2]: *** [stmp-fixinc] Error 1

If I fudge past that error by doing a `mkdir /temptools/usr/include', it gets a
little further but craps out again:

In file included from ../../../gcc-4.3.0/libgcc/../gcc/libgcc2.c:33:
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:90:19: error: stdio.h: No such file
or directory
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:93:23: error: sys/types.h: No such
file or directory
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:96:19: error: errno.h: No such file
or directory
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:103:20: error: string.h: No such
file or directory
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:104:20: error: stdlib.h: No such
file or directory
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:105:20: error: unistd.h: No such
file or directory
../../../gcc-4.3.0/libgcc/../gcc/tsystem.h:111:18: error: time.h: No such file
or directory
make[2]: *** [_muldi3.o] Error 1

As mentioned above, the sysroot stuff is clearly designed for a file system
layout with /usr and DESTDIR style installs ie:

/sysroot/usr/include
/sysroot/usr/lib
/sysroot/lib

Oh well. T'was worth a try.

> I'm reading through the DIY instructions right now to make sense of your
> bootstrap process.

I have a feeling that slotting in a -B $prefix/lib somewhere in the
*FLAGS_FOR_TARGET could potentially solve my problem. Except that -B currently
doesn't process the multilibs.. damn.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[mmitchel at gcc dot gnu dot org]

------- Comment #8 from mmitchel at gcc dot gnu dot org  2008-03-21 17:19 -------
Greg --

I'm sorry it's taken me so long to comment on this issue.  I've been traveling
for most of the time since you reported this issue.

The change in driver behavior is intentional.  Using the sysroot flags
(including --with-build-sysroot) is indeed the intended way of doing what you
need to do.  You are right, however, that the driver will look for a /usr
subdirectory within the build sysroot.  I'm a little surprised that your GLIBC
installation isn't set up that way, but you can fake it by doing something
like:

  ln -s . $prefix/usr 

so that $prefix/usr/include is just another name for $prefix/include.

-- Mark


-- 

mmitchel at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mark at codesourcery dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[dirtyepic at gentoo dot org]

------- Comment #9 from dirtyepic at gentoo dot org  2008-03-22 20:32 -------
(In reply to comment #4)
> By building gcc you become a gcc developer, not a user

Nice.  We have about 30,000 new developers for you then.  Who do i talk to
about getting svn write access for them?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[carlos at codesourcery dot com]

------- Comment #10 from carlos at codesourcery dot com  2008-03-24 17:25 -------
Greg,

I've gone through your DIY instructions. Very well done. Using the sysroot
infrastructure is definitely the way forward for you. Looking at your existing
DIY instructions you probably want to:

1. Create a "Construct sysroot" step before the first stage gcc (3.5), this
will include creating $sysroot/usr/include.
2. Install the kernel headers (3.6) as part of the "Construct sysroot" step, or
as the next step.

You might want to verify that this is still needed:
~~~
echo "
/* Remove /usr/include from end of include search path.  */
#undef STANDARD_INCLUDE_DIR
#define STANDARD_INCLUDE_DIR 0" >> gcc/config/${DL_HEADER}
~~~
in step (3.10).


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[bonzini at gnu dot org]

------- Comment #11 from bonzini at gnu dot org  2008-04-02 14:08 -------
Carlos, I think Greg has a point here:

> the problem is that xgcc thinks it is a relocated compiler when it 
> runs from $objdir eg: when building libgcc* and other target libs, when in
> actual fact, it is not really a true relocated compiler at all.

Looking at http://gcc.gnu.org/ml/gcc/2006-10/msg00280.html it seems to me that
setenv GCC_EXEC_PREFIX "$GCC_EXEC_PREFIX" is always needed when we run in the
build directory.  This is easier said than done -- for example adding a
--exec-prefix option to the driver is not possible because gcc_exec_prefix is
set much earlier than at option-processing time.


-- 

bonzini at gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bonzini at gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[carlos at codesourcery dot com]

------- Comment #12 from carlos at codesourcery dot com  2008-04-02 19:20 -------
Paolo,

What's the test-case?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[bonzini at gnu dot org]

------- Comment #13 from bonzini at gnu dot org  2008-04-02 20:27 -------
Subject: Re:  Native GCC no longer searches $prefix/lib for startfiles when run
from $objdir

>  What's the test-case?

I'm talking of Greg's setting.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[carlos at codesourcery dot com]

------- Comment #14 from carlos at codesourcery dot com  2008-04-02 21:52 -------
Using the sysroot flags is the solution for Greg's scenario. In fact I would
say it makes his job easier.

I'm looking for a test case that doesn't mangle GCC's configure files, and is
broken with no easy alternative.

Do we have one?


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

[Bug driver/35532] Native GCC no longer searches $prefix/lib for startfiles when run from $objdir

[earnie at users dot sourceforge.net]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=35532

Earnie <earnie at users dot sourceforge.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |earnie at users dot sourceforge.ne
                   |                            |t

--- Comment #15 from Earnie <earnie at users dot sourceforge.net> ---
I know this is old but I have a similar issue with 2.8.1 in building a native
MinGW build.  The prev-gcc/xgcc seems to be working but the build of
genconstants with prev-gcc/xg++ is giving this issue.  I've tried various
--with-sysroot --with-build-sysroot and other methods with either the same or
different issues.  I was able to get a build by coping the missing crt2.0 and
libraries in the prev-gcc directory but that isn't a real solution.  Note that
MinGW has never had a /usr directory since /mingw is the replacement for /usr. 
I'm willing to try suggestions, but a native build should just build without
needing to play games.

~~~~~
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by configure, which was
generated by GNU Autoconf 2.64.  Invocation command line was

  $ ../../src/gcc-4.8.1/configure --prefix=/mingw --target=i686-pc-mingw32
--host=i686-pc-mingw32 --build=i686-pc-mingw32 --with-gnu-ld --without-pic
--enable-shared --enable-static --enable-lto --enable-libssp --disable-multilib
--enable-languages=c,c++,fortran,objc,obj-c++,ada --disable-sjlj-exceptions
--with-dwarf2 --disable-win32-registry --enable-libstdcxx-debug
--enable-version-specfic-runtime-libs --with-gmp=/mingw --with-mpc=/mingw
--with-mpfr=/mingw --with-system-zlib
--with-native-system-header-dir=/mingw/include --with-gnu-as
--enable-decimal-float=yes
~~~~~

-----

~~~~~
/usr/src/bld/gcc/./prev-gcc/xg++ -B/usr/src/bld/gcc/./prev-gcc/
-B/mingw/i686-pc-mingw32/bin/ -nostdinc++
-B/usr/src/bld/gcc/prev-i686-pc-mingw32/libstdc++-v3/src/.libs
-B/usr/src/bld/gcc/prev-i686-pc-mingw32/libstdc++-v3/libsupc++/.libs
-I/usr/src/bld/gcc/prev-i686-pc-mingw32/libstdc++-v3/include/i686-pc-mingw32
-I/usr/src/bld/gcc/prev-i686-pc-mingw32/libstdc++-v3/include
-I/usr/src/src/gcc-4.8.1/libstdc++-v3/libsupc++
-L/usr/src/bld/gcc/prev-i686-pc-mingw32/libstdc++-v3/src/.libs
-L/usr/src/bld/gcc/prev-i686-pc-mingw32/libstdc++-v3/libsupc++/.libs   -g -O2
-D__USE_MINGW_ACCESS -Wno-pedantic-ms-format -gtoggle -DIN_GCC  
-fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing
-Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long
-Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -DGENERATOR_FILE
-static-libstdc++ -static-libgcc -Wl,--stack,12582912 -o build/genconstants.exe
\
        build/genconstants.o build/read-md.o build/errors.o
.././libiberty/libiberty.a
h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find crt2.o: No such file or
directory

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmingw32

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmoldname

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmingwex

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmsvcrt

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -ladvapi32

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lshell32

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -luser32

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lkernel32

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmingw32

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmoldname

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmingwex

h:\p\giaw\mingw\i686-pc-mingw32\bin\ld.exe: cannot find -lmsvcrt

collect2.exe: error: ld returned 1 exit status

make[3]: *** [build/genconstants.exe] Error 1
make[3]: Leaving directory `/usr/src/bld/gcc/gcc'
make[2]: *** [all-stage2-gcc] Error 2
make[2]: Leaving directory `/usr/src/bld/gcc'
make[1]: *** [stage2-bubble] Error 2
make[1]: Leaving directory `/usr/src/bld/gcc'
make: *** [all] Error 2
~~~~~