Problem building glibc

Problem building glibc

[Michael Eager]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

I'm running into an odd problem building glibc.

I'm building glibc as simply as possible:
   $ cd /local/work/fsf-head/build/glibc
   $ ../../src/glibc/configure --prefix=/local/work/fsf-head/install/
   $ make

The make never finishes.  It enters sub-directories string
and runs make, then exits, runs make in other sub-dirs, then
re-enters sub-dir string, runs make, and then repeats this
until killed.

On the first entry into sub-dir string, all of the .o
files are built.  On the second and subsequent entries,
no object files are rebuilt, but limits.h is installed in
the install directory and locale-defines.h is written
in the build directory.

This is happening with top-of-tree builds of binutils
and gcc, using make-3.81.  I cannot reproduce this on
another system with gcc-4.7.2 and make-3.82.

I've attached a piece of the make log showing what
is happening when sub-dir string is entered the second
time.  Any suggestions are appreciated.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: glibc-make.log --]
[-- Type: text/x-log; name="glibc-make.log", Size: 2960 bytes --]

make[2]: Leaving directory `/local/work/fsf-head/src/glibc/malloc'
make  subdir=string -C string ..=../ subdir_lib
make[2]: Entering directory `/local/work/fsf-head/src/glibc/string'
/usr/bin/install -c -m 644 ../include/limits.h /local/work/fsf-head/install/include/limits.h
gawk -f ../scripts/gen-as-const.awk ../sysdeps/x86_64/locale-defines.sym \
	| gcc -S -o /local/work/fsf-head/build/glibc/locale-defines.hT3 -std=gnu99 -fgnu89-inline  -O2 -Wall -Werror -Wno-error=undef -Wundef -Wwrite-strings -fmerge-all-constants -frounding-math -g -Wstrict-prototypes          -I../include -I/local/work/fsf-head/build/glibc/string  -I/local/work/fsf-head/build/glibc  -I../sysdeps/unix/sysv/linux/x86_64/64  -I../sysdeps/unix/sysv/linux/x86_64  -I../sysdeps/unix/sysv/linux/x86  -I../sysdeps/unix/sysv/linux/wordsize-64  -I../sysdeps/x86_64/nptl  -I../sysdeps/unix/sysv/linux/include -I../sysdeps/unix/sysv/linux  -I../sysdeps/nptl  -I../sysdeps/pthread  -I../sysdeps/gnu  -I../sysdeps/unix/inet  -I../sysdeps/unix/sysv  -I../sysdeps/unix/x86_64  -I../sysdeps/unix  -I../sysdeps/posix  -I../sysdeps/x86_64/64  -I../sysdeps/x86_64/fpu/multiarch  -I../sysdeps/x86_64/fpu  -I../sysdeps/x86/fpu/include -I../sysdeps/x86/fpu  -I../sysdeps/x86_64/multiarch  -I../sysdeps/x86_64  -I../sysdeps/x86  -I../sysdeps/ieee754/ldbl-96  -I../sysdeps/ieee754/dbl-64/wordsize-64  -I../sysdeps/ieee754/dbl-64  -I../sysdeps/ieee754/flt-32  -I../sysdeps/wordsize-64  -I../sysdeps/ieee754  -I../sysdeps/generic  -I.. -I../libio -I.   -D_LIBC_REENTRANT -include /local/work/fsf-head/build/glibc/libc-modules.h -DMODULE_NAME=libc -include ../include/libc-symbols.h       -x c - \
		-MD -MP -MF /local/work/fsf-head/build/glibc/locale-defines.h.dT -MT '/local/work/fsf-head/build/glibc/locale-defines.h.d /local/work/fsf-head/build/glibc/locale-defines.h'
sed -n 's/^.*@@@name@@@\([^@]*\)@@@value@@@[^0-9Xxa-fA-F-]*\([0-9Xxa-fA-F-][0-9Xxa-fA-F-]*\).*@@@end@@@.*$/#define \1 \2/p' \
		/local/work/fsf-head/build/glibc/locale-defines.hT3 > /local/work/fsf-head/build/glibc/locale-defines.hT
rm -f /local/work/fsf-head/build/glibc/locale-defines.hT3
sed -e 's@ /local/work/fsf-head/build/glibc/@ $(common-objpfx)@g' -e 's@^/local/work/fsf-head/build/glibc/@$(common-objpfx)@g' -e 's@  *\.\.\/\([^ 	\]*\)@ $(..)\1@g' -e 's@^\.\.\/\([^ 	\]*\)@$(..)\1@g' \
	    /local/work/fsf-head/build/glibc/locale-defines.h.dT > /local/work/fsf-head/build/glibc/locale-defines.h.dT2
rm -f /local/work/fsf-head/build/glibc/locale-defines.h.dT
mv -f /local/work/fsf-head/build/glibc/locale-defines.h.dT2 /local/work/fsf-head/build/glibc/locale-defines.h.d
mv -f /local/work/fsf-head/build/glibc/locale-defines.hT /local/work/fsf-head/build/glibc/locale-defines.h
make[2]: Leaving directory `/local/work/fsf-head/src/glibc/string'
make[2]: Entering directory `/local/work/fsf-head/src/glibc/string'
/usr/bin/install -c -m 644 ../include/limits.h /local/work/fsf-head/install/include/limits.h

Re: Problem building glibc

[Roland McGrath]

I can't reproduce any problem with make-3.81, though I didn't try using
trunk binutils and gcc too.  As you can imagine, people are not commonly
hitting anything like this or we would have addressed it.  So you'll have
to do your own debugging.

The scenario of a header file being installed during the build suggests
that the include searching is going wrong.  Nothing from the install
directory should be used during the build.  I'd look at the .d (or .dt)
files in string/ and see which ones refer to anything in the install
directory, then figure out why it's coming out that way.

Finding limits.h from the wrong place doesn't really explain the infinite
iteration behavior.  But it's the most obvious place to look from what
you've described, and fixing the root cause of that might make everything
else shape up.

Re: Problem building glibc

[Richard Henderson]

On 05/19/2015 11:46 AM, Michael Eager wrote:
> I'm running into an odd problem building glibc.
> 
> I'm building glibc as simply as possible:
>   $ cd /local/work/fsf-head/build/glibc
>   $ ../../src/glibc/configure --prefix=/local/work/fsf-head/install/
>   $ make
> 
> The make never finishes.  It enters sub-directories string
> and runs make, then exits, runs make in other sub-dirs, then
> re-enters sub-dir string, runs make, and then repeats this
> until killed.

I've seen that before when the system clock is grossly off.

It happens most often when networked filesystems are involved, which would not
seem to be indicated with the "/local" prefix.  But I've also seen it in VMs
where something went wrong and the guest reads the RTC as 1970.


r~

Re: Problem building glibc

[Michael Eager]

On 05/19/2015 02:09 PM, Richard Henderson wrote:
> On 05/19/2015 11:46 AM, Michael Eager wrote:
>> I'm running into an odd problem building glibc.
>>
>> I'm building glibc as simply as possible:
>>    $ cd /local/work/fsf-head/build/glibc
>>    $ ../../src/glibc/configure --prefix=/local/work/fsf-head/install/
>>    $ make
>>
>> The make never finishes.  It enters sub-directories string
>> and runs make, then exits, runs make in other sub-dirs, then
>> re-enters sub-dir string, runs make, and then repeats this
>> until killed.
>
> I've seen that before when the system clock is grossly off.
>
> It happens most often when networked filesystems are involved, which would not
> seem to be indicated with the "/local" prefix.  But I've also seen it in VMs
> where something went wrong and the guest reads the RTC as 1970.

Yes, I've seen this in cross-network builds when the source
is updated on one system and built on another system when the
clocks are not synchronized.

Source, build, and install are all on the same mounted file system.

This is running in a VM.  As far as I can tell, time is in sync.

I built glibc last week on this system with no problem.
This behavior just started a day or two ago. I modified glibc and
running make didn't terminate.  I've backed out all my changes.


-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Problem building glibc

[Michael Eager]

On 05/19/2015 12:39 PM, Roland McGrath wrote:
> I can't reproduce any problem with make-3.81, though I didn't try using
> trunk binutils and gcc too.  As you can imagine, people are not commonly
> hitting anything like this or we would have addressed it.  So you'll have
> to do your own debugging.
>
> The scenario of a header file being installed during the build suggests
> that the include searching is going wrong.  Nothing from the install
> directory should be used during the build.  I'd look at the .d (or .dt)
> files in string/ and see which ones refer to anything in the install
> directory, then figure out why it's coming out that way.

There are a number of dependencies on headers in the install directory.
Most are the headers installed by gcc, under ...-linux-gnu/4.9.2.  If
I filter those out, there are dependencies for install/include/limits.h.
Once this is touched, almost everything needs to be remade.

Should I configure with --with-headers pointing somewhere else?

> Finding limits.h from the wrong place doesn't really explain the infinite
> iteration behavior.  But it's the most obvious place to look from what
> you've described, and fixing the root cause of that might make everything
> else shape up.

Running "make -d" I see this:

       Considering target file `/local/work/fsf-head/build/glibc/iconv/stamp.o'.
         Considering target file `/local/work/fsf-head/build/glibc/iconv/iconv_open.o'.
          Looking for an implicit rule for `/local/work/fsf-head/build/glibc/iconv/iconv_open.o'.
          Trying pattern rule with stem `iconv_open'.
          Trying implicit prerequisite `../sysdeps/unix/sysv/linux/x86_64/64/iconv_open.S'.
          Trying pattern rule with stem `iconv_open'.
          ...
            Prerequisite `/local/work/fsf-head/build/glibc/gnu/lib-names-64.stmp' is newer than 
target `/local/work/fsf-head/build/glibc/gnu/lib-names-64.h'.
          ...
            Prerequisite `/local/work/fsf-head/build/glibc/libc-modules.stmp' is newer than target 
`/local/work/fsf-head/build/glibc/libc-modules.h'.
           Must remake target `/local/work/fsf-head/build/glibc/libc-modules.h'.
          ...
             Pruning file `../include/limits.h'.
             Considering target file `force-install'.
              File `force-install' does not exist.
              Finished prerequisites of target file `force-install'.
             Must remake target `force-install'.
             Successfully remade target file `force-install'.
            Finished prerequisites of target file `/local/work/fsf-head/install/include/limits.h'.
            Prerequisite `../include/limits.h' is older than target 
`/local/work/fsf-head/install/include/limits.h'.
            Prerequisite `force-install' of target `/local/work/fsf-head/install/include/limits.h' 
does not exist.
           Must remake target `/local/work/fsf-head/install/include/limits.h'.

There are hundreds of "Must remake target" lines, including several for
install/include/limits.h.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Problem building glibc

[Roland McGrath]

> There are a number of dependencies on headers in the install directory.
> Most are the headers installed by gcc, under ...-linux-gnu/4.9.2.  

If the install directory contains the GCC installation too, then that is
perfectly normal.  What I meant was files installed by libc.

> If
> I filter those out, there are dependencies for install/include/limits.h.
> Once this is touched, almost everything needs to be remade.

That's not surprising given what you reported.  This is the crux of the
problem.  The build should not be finding limits.h there, but instead in
the libc source tree (libc/include/limits.h).  Because of the complex
#include_next dance between GCC's limits.h and libc's limits.h (via GCC's
syslimits.h), figuring out how this is going wrong could be nontrivial.  It
might be a new bug in trunk GCC's #include_next behavior, or it might be a
strange confluence of "correct" behavior and the particular circumstances
of your setup that we have just happened never to see before.

So pick some one file that gets limits.h into its .d (aka .dt) file, run
that one compilation command by hand, and figure out how the include nest
is (not) working.

> Should I configure with --with-headers pointing somewhere else?

That is only meant to be used for the kernel headers.


Thanks,
Roland

Re: Problem building glibc

[Michael Eager]

On 05/19/2015 04:40 PM, Roland McGrath wrote:
>> There are a number of dependencies on headers in the install directory.
>> Most are the headers installed by gcc, under ...-linux-gnu/4.9.2.
>
> If the install directory contains the GCC installation too, then that is
> perfectly normal.  What I meant was files installed by libc.
>
>> If
>> I filter those out, there are dependencies for install/include/limits.h.
>> Once this is touched, almost everything needs to be remade.
>
> That's not surprising given what you reported.  This is the crux of the
> problem.  The build should not be finding limits.h there, but instead in
> the libc source tree (libc/include/limits.h).  Because of the complex
> #include_next dance between GCC's limits.h and libc's limits.h (via GCC's
> syslimits.h), figuring out how this is going wrong could be nontrivial.  It
> might be a new bug in trunk GCC's #include_next behavior, or it might be a
> strange confluence of "correct" behavior and the particular circumstances
> of your setup that we have just happened never to see before.
>
> So pick some one file that gets limits.h into its .d (aka .dt) file, run
> that one compilation command by hand, and figure out how the include nest
> is (not) working.

Thanks for the suggestion.

I'll look at the gcc commit log and see if there is anything
that looks suspect, and perhaps build an older gcc and try
to bisect the problem.  Otherwise, a frontal attack.

-- 
Michael Eager	 eager@eagercon.com
1960 Park Blvd., Palo Alto, CA 94306  650-325-8077

Re: Problem building glibc

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On 19 May 2015 11:46, Michael Eager wrote:
> I'm running into an odd problem building glibc.
> 
> I'm building glibc as simply as possible:
>    $ cd /local/work/fsf-head/build/glibc
>    $ ../../src/glibc/configure --prefix=/local/work/fsf-head/install/
>    $ make
> 
> The make never finishes.  It enters sub-directories string
> and runs make, then exits, runs make in other sub-dirs, then
> re-enters sub-dir string, runs make, and then repeats this
> until killed.
> 
> On the first entry into sub-dir string, all of the .o
> files are built.  On the second and subsequent entries,
> no object files are rebuilt, but limits.h is installed in
> the install directory and locale-defines.h is written
> in the build directory.

i just ran into this issue (a different header, but same description) while
messing around with some of the internal headers, and then the issue stayed
sticky even after i backed out things.  i resolved the situation by blowing
away my ccache and rebuilding from scratch ...
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]