Toplevel configury, multilibs, new autoconf versions

Toplevel configury, multilibs, new autoconf versions

[Phil Edwards]

The next step in converting to the new autotools is making multilibs work.
I'm using the versions of the files that I put up for testing last night
(see the v3 list).  And I've temporarily turned i686-linux into a multilib
target using some harmless option (one of the -f optimizations), just
there to trigger the multilib build mechanisms.

The build is using the value of CXX passed around in the environment.
For libstdc++, it's a special case from the toplevel:

    libstdcxx_flags='`test ! -f $$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags || $(SHELL) $$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags --build-includes` -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
    raw_libstdcxx_flags=' -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'

Note the -L options; these get embedded into CXX_FOR_TARGET a few lines
later, which becomes CXX inside the library.

(Why does libstdc++ care about -L options?  Because we actually do link
at least one executable during the normal build, to say nothing of the
testsuite.)

Nothing anywhere adjusts the value of CXX's -L flags to search the multilib
dirs instead of the primary dir.  For a real multilib target, this means
that the program gets compiled with one option, but linked against the
standard primary library.  (For my fake i686 multilibs, this would have
succeeded, but the multilibs look to be built bottom up, so the primary
libstdc++.so hadn't been built yet when the multilib's executable tried
to search that directory, and the link failed.)


Ideas are solicited.  What we need to do seems clear, but I could use some
help deciding how to go about doing it.


Phil

-- 
If ye love wealth greater than liberty, the tranquility of servitude greater
than the animating contest for freedom, go home and leave us in peace.  We seek
not your counsel, nor your arms.  Crouch down and lick the hand that feeds you;
and may posterity forget that ye were our countrymen.            - Samuel Adams

Re: Toplevel configury, multilibs, new autoconf versions

[Alexandre Oliva]

On Jul 27, 2003, Phil Edwards <phil@jaj.com> wrote:

> It source's config-ml.in, which runs through the list of multilib subdirs,
> and does text substitution on the env variables.

And then it modifies CXX, and passes that to each multilib's
configure.  Since each multilib uses separate config.cache and
separate config.status, I don't see why the wrong CXX would be cached
in them.  Isn't config-ml.in doing its job properly?  Or is the
primary multilib inappropriately passing CXX to other multilibs'
makefiles?

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                 aoliva@{redhat.com, gcc.gnu.org}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Toplevel configury, multilibs, new autoconf versions

[Phil Edwards]

On Sat, Jul 26, 2003 at 04:58:55PM -0400, Phil Edwards wrote:
> > >    libstdcxx_flags='`test ! -f 
> > >$$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags || $(SHELL) 
> > >$$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags 
> > >--build-includes` -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src 
> > >-L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
> > >    raw_libstdcxx_flags=' -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src 
> > >-L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
> > >
> > >Note the -L options; these get embedded into CXX_FOR_TARGET a few lines
> > >later, which becomes CXX inside the library.
[...]
> Also, I'm having trouble understanding how this manages to work at present,
> i.e., with the "old" autotools.  The -L options are still hardcoded
> either way.

Okay, I've figured this part out, and why it's breaking.  I'm going to
need some help to make this work.

To make things clearer in this email, I'm going to pretend that the CXX
variable for a normal primary build is set to Primary.  (It's actully a long
"..../xgcc -B... -L..." string.)  So, when the primary v3 configure is run,
the toplevel does it via

    CXX=Primary $srcdir/configure --stuff

Clear?

Now, at the end of that run, config.status is generated and executed.
It source's config-ml.in, which runs through the list of multilib subdirs,
and does text substitution on the env variables.  Pretend we have one
multilib, and I'll call the CXX variable for that one Multilib:

    # config-ml.in does this
    Multilib=`echo Primary | sed 's/references-to-primary-dir/references-to-multilib-dir'`

    cd up-and-over-to-multilib-dir/libstdc++-v3
    CXX=Multilib $srcdir/configure --stuff

Where "--stuff" is everything that was passed to configure.  This list of
args is stored in config.status.  Remember that for later.

That's why hardcoding things like -L into CXX at the top level still work
for multilibs.  Still clear?


Since the days of autoconf 2.13, it has been obverved that people try to
do things like this, in general:

    $ CC=foo ./configure
    $ make
        [something breaks, requiring a reconfigure]
    $ ./configure
        [oops, CC wasn't set this time, different compilers are found
        during the tests, hilarity ensues]

To protect the users from themselves, autoconf 2.5x takes note of "precious"
variables in the environment, and remembers them.

    $ CC=foo ./configure
    $ make
        [something breaks, requiring a reconfigure]
    $ ./configure
    configure: error: you're being a dumbass!

To be both safe and helpful, the configure script actually stores those
variables into the generated config.status file -- pay attention now --
as if they had been specified on the command line.  The shell variable
which stores command-line arguments is augmented with those "precious"
environment variables.


So when a multilib run is fired off with configure generated by 2.57,
the created primary config.status source's config-ml.in, which then does
this just as before:

    Multilib=`echo Primary | sed '.....'`

    CXX=Multilib $srcdir/configure --stuff 'CXX=Primary'

When configure examines the command line and sees variable assignments,
it performs them without examining the current environment.  It does this
for safety, because it thinks the user is rerunning configure.

That sound you hear is configure wailing, "MY PRECIOUSSSSSSSS!" as it
overwrites Multilib with Primary.


I've made a couple of half-hearted attempts to defeat this from within
libstdc++'s acinclude.m4, but autoconf is too smart for me.


Phil

-- 
If ye love wealth greater than liberty, the tranquility of servitude greater
than the animating contest for freedom, go home and leave us in peace.  We seek
not your counsel, nor your arms.  Crouch down and lick the hand that feeds you;
and may posterity forget that ye were our countrymen.            - Samuel Adams

Re: Toplevel configury, multilibs, new autoconf versions

[Phil Edwards]

On Sat, Jul 26, 2003 at 01:16:10AM -0400, Nathanael Nerode wrote:
> >The build is using the value of CXX passed around in the environment.
> >For libstdc++, it's a special case from the toplevel:
> >
> >    libstdcxx_flags='`test ! -f 
> >$$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags || $(SHELL) 
> >$$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags 
> >--build-includes` -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src 
> >-L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
> >    raw_libstdcxx_flags=' -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src 
> >-L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
> >
> >Note the -L options; these get embedded into CXX_FOR_TARGET a few lines
> >later, which becomes CXX inside the library.
> 
> So the testsuite_flags stuff isn't the problem?  That was the part I've 
> been afraid to tweak.  :-)

It's not the problem /now/.  --build-includes emits a bunch of -I's.
It's the -L's that are the problem, and those are embedded in the toplevel
configure.in.


> I think 'preinstall' is the correct solution for this.  libstdc++ should 
> be changed to preinstall the libraries into preinstall-${target}/lib.
> 
> Then after multilibbing is moved to the top level, the 
> non-multilib version would always have preinstall-${target}/lib added
> to the LD_LIBRARY_PATH passed down from the top.  For the multilibbed 
> targets, each one would have a different, appropriate 
> preinstall-${target}/xx/lib added to LD_LIBRARY_PATH.

And LD_RUN_PATH too, I hope.


> OK, so this is pie-in-the-sky.  But maybe you can find a way to adapt 
> this idea to the current state of things?

Not at the present time.  I've got too many local changes already to try
and do preinstalling.  (FWIW, both Benjamin and I -- and maybe others too,
I simply don't know -- agree that a staging area for installation is needed.)

Also, I'm having trouble understanding how this manages to work at present,
i.e., with the "old" autotools.  The -L options are still hardcoded
either way.


Phil

-- 
If ye love wealth greater than liberty, the tranquility of servitude greater
than the animating contest for freedom, go home and leave us in peace.  We seek
not your counsel, nor your arms.  Crouch down and lick the hand that feeds you;
and may posterity forget that ye were our countrymen.            - Samuel Adams

Re: Toplevel configury, multilibs, new autoconf versions

[Nathanael Nerode]

>The next step in converting to the new autotools is making multilibs 
>work.

I tend to think the 'correct' way to do things is to move multilibbing 
to the top level.  ;-)  But that might be too much work to do *before* 
the autoconf upgrade.

>The build is using the value of CXX passed around in the environment.
>For libstdc++, it's a special case from the toplevel:
>
>    libstdcxx_flags='`test ! -f 
>$$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags || $(SHELL) 
>$$r/$(TARGET_SUBDIR)/libstdc++-v3/scripts/testsuite_flags 
>--build-includes` -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src 
>-L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
>    raw_libstdcxx_flags=' -L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src 
>-L$$r/$(TARGET_SUBDIR)/libstdc++-v3/src/.libs'
>
>Note the -L options; these get embedded into CXX_FOR_TARGET a few lines
>later, which becomes CXX inside the library.

So the testsuite_flags stuff isn't the problem?  That was the part I've 
been afraid to tweak.  :-)

I think 'preinstall' is the correct solution for this.  libstdc++ should 
be changed to preinstall the libraries into preinstall-${target}/lib.

Then after multilibbing is moved to the top level, the 
non-multilib version would always have preinstall-${target}/lib added
to the LD_LIBRARY_PATH passed down from the top.  For the multilibbed 
targets, each one would have a different, appropriate 
preinstall-${target}/xx/lib added to LD_LIBRARY_PATH.

OK, so this is pie-in-the-sky.  But maybe you can find a way to adapt 
this idea to the current state of things?

-- 
Nathanael Nerode  <neroden at gcc.gnu.org>
http://home.twcny.rr.com/nerode/neroden/fdl.html