Deciphering flags in CXX_FOR_TARGET

Deciphering flags in CXX_FOR_TARGET

[Nathanael Nerode]

In the process of unravelling the build process for gcc and src (so that
I can make a nice autoconfiscation), I've hit another confusing point.

CXX_FOR_TARGET is set to a rather complicated piece of shell code.
Some of this conditionalizes on which directory is being configured,
which is fine.

One bit, however, is quite odd.  It adds certain flags to
CXX_FOR_TARGET.  These flags are generated by
`libstdc++-v3/testsuite_flags --build-includes`.  This is only done if
'testsuite_flags' exists, which is only true if libstdc++-v3 has been
configured.

Due to the dependency structure of the Makefile, these are only
guaranteed to be used by the build of target-gperf, a program
which doesn't reside in gcc or src.  With others, it depends on make's
choice of order, which is extremely undesirable.

I presume that 'check' targets might depend on these flags, because
'check' is assumed to be run entirely after 'make all' (although the
dependencies are not explicit).  

'install' targets might, but it seems very wrong for an 'install' target
to depend on compiler flags; similarly I doubt this is an issue for dvi,
info, clean.

This little issue is the cause of an unbelievable amount of messiness.

Can someone explain to me what these flags from 'testsuite_flags' are,
in fact, for?  

(I'm guessing they're only used for 'check', and in fact
only for 'check' in certain subdirectories.  If that's the case, I can
make a much cleaner implementation for my autoconfiscation.  I'd also
want to know which subdirectories of course, though I can probably
manage without that information.)

Thanks in advance.

--Nathanael

Re: Deciphering flags in CXX_FOR_TARGET

[Nathanael Nerode]

On Tue, May 07, 2002 at 12:51:55PM -0700, Geoff Keating wrote:
> 
> > One bit, however, is quite odd.  It adds certain flags to
> > CXX_FOR_TARGET.  These flags are generated by
> > `libstdc++-v3/testsuite_flags --build-includes`.  This is only done if
> > 'testsuite_flags' exists, which is only true if libstdc++-v3 has been
> > configured.
> 
> If you don't get a better answer, my suggestion would be to delete the
> inclusion of these flags from a local combined tree, and try to build
> and test.  If the flags are necessary, something should break...
> 
> I suspect they're necessary for building libstdc++-v3 or libjava, and
> that the weirdness is actually a bug.
Heh.  Actually, that's the one thing I'm absolutely sure they're not
necessary for, since they're explicitly avoided in those two contexts!

Re: Deciphering flags in CXX_FOR_TARGET

[Alexandre Oliva]

On May  7, 2002, Nathanael Nerode <neroden@doctormoo.dyndns.org> wrote:

> Due to the dependency structure of the Makefile, these are only
> guaranteed to be used by the build of target-gperf

Which is probably the only C++ application at the moment.  If not,
we're missing Makefile dependencies.  Any application that links with
libstdc++-v3 or uses its headers must be configured after libstdc++-v3
is built.

> With others, it depends on make's choice of order, which is
> extremely undesirable.

For target programs or libraries that don't use CXX or CXXFLAGS, this
doesn't make any difference.

> Can someone explain to me what these flags from 'testsuite_flags' are,
> in fact, for?  

They're flags that have to be passed to g++ in the build tree, such
that it finds libstdc++ in the build tree.  Without these flags, it
would only look at the install tree, if that much, and we'd end up
using the wrong version of libstdc++ to build libraries that depend on
it, or to test libstdc++ itself.

> If that's the case, I can make a much cleaner implementation for my
> autoconfiscation.

I think the right way to clean this up properly, which was suggested
earlier by someone else (Per?), is to get compilers and target
libraries built in a tree that mirrors the install tree, such that the
compiler is able to use the intelligence it already has to locate
headers and libraries yet to be installed.  This would obviate the
hideous hack in CXX_FOR_TARGET.  Unfortunately, we can't drop it
before we move to this saner build structure.

> I'd also want to know which subdirectories of course, though I can
> probably manage without that information.)

If you find AC_PROG_CXX in a target package's configure, these flags
probably matter.  (I say probably because libgcj, for example, isn't
linked with libstdc++-v3, even though it contains C++ code.  I'm not
sure about dependencies on headers, though).

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Deciphering flags in CXX_FOR_TARGET

[Nathanael Nerode]

On Tue, May 07, 2002 at 07:14:59PM -0300, Alexandre Oliva wrote:
> I think the right way to clean this up properly, which was suggested
> earlier by someone else (Per?), is to get compilers and target
> libraries built in a tree that mirrors the install tree, such that the
> compiler is able to use the intelligence it already has to locate
> headers and libraries yet to be installed.  This would obviate the
> hideous hack in CXX_FOR_TARGET.  Unfortunately, we can't drop it
> before we move to this saner build structure.

Hmm.  A new structure might also allow for the cleanup of dozens
of less hideous hacks; things which look for 'gcc/xgcc', 'ld/ld-new',
etc.  These, I can more easily deal with, but getting rid of all the
special-case code would be a win.

Perhaps an 'install-for-build' in each subdirectory which would
copy/move/link/symlink its output files to a tree mirroring the install
tree would do the trick?

Perhaps we could just convince each subdirectory to put its files in
a reasonable place without calling 'install', but that sounds harder.

More thoughts from the wise?

--Nathanael

Re: Deciphering flags in CXX_FOR_TARGET

[Alexandre Oliva]

On May  7, 2002, Nathanael Nerode <neroden@doctormoo.dyndns.org> wrote:

> Perhaps an 'install-for-build' in each subdirectory which would
> copy/move/link/symlink its output files to a tree mirroring the install
> tree would do the trick?

Nope.  We can't install into the build tree, we really have to build
the tools into the tree-for-install, and install them from there.

This is critical for correct functioning of libtool-build libraries
and executables.  They're created with a notion of where they're
expected to be installed, and installing them elsewhere doesn't always
produce correct results.  But libtool can tell `build' from `install'
files, so if we build them in a location in the build tree in which
they will work, and then install them from there, things will work
fine.

> Perhaps we could just convince each subdirectory to put its files in
> a reasonable place without calling 'install', but that sounds harder.

Unfortunately this is what we have to do for correct functioning, at
least for pieces that use libtool.

There's a longer message about this in a thread started by Per
Bothner, IIRC, in late February this year.  You may want to have a
look into that.  I've never wanted to ask you to do that cause it
might scare you away, but since you got this far, and you appear to be
interested in tackling this problem, I think it will do more good to
get you involved now :-)

Thanks,

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer

Re: Deciphering flags in CXX_FOR_TARGET

[Gabriel Dos Reis]

Nathanael Nerode <neroden@doctormoo.dyndns.org> writes:

| On Tue, May 07, 2002 at 12:51:55PM -0700, Geoff Keating wrote:
| > 
| > > One bit, however, is quite odd.  It adds certain flags to
| > > CXX_FOR_TARGET.  These flags are generated by
| > > `libstdc++-v3/testsuite_flags --build-includes`.  This is only done if
| > > 'testsuite_flags' exists, which is only true if libstdc++-v3 has been
| > > configured.
| > 
| > If you don't get a better answer, my suggestion would be to delete the
| > inclusion of these flags from a local combined tree, and try to build
| > and test.  If the flags are necessary, something should break...
| > 
| > I suspect they're necessary for building libstdc++-v3 or libjava, and
| > that the weirdness is actually a bug.
| Heh.  Actually, that's the one thing I'm absolutely sure they're not
| necessary for, since they're explicitly avoided in those two contexts!

I'm not sure I understand your comment.  Maybe the name of script is
ill-choosen but at the time it was first created that name was
descriptive. 

Initially, we used it to generare flags necessary to properly build
and test libstdc++-v3.  Then as things were growing we factorized all
the places where we needed to know which flags are appropriate to use
libstdc++ in that script.  In short, any program that needs to locate
g++ and the library headers should invoke that script to figure out
which flags to use.  That is true for the testsuite; that should be
true for libgcj (we already had that discussion in the past).

-- Gaby

Re: Deciphering flags in CXX_FOR_TARGET

[Nathanael Nerode]

On Wed, May 08, 2002 at 11:43:44AM +0200, Gabriel Dos Reis wrote:
> Nathanael Nerode <neroden@doctormoo.dyndns.org> writes:
> 
> | On Tue, May 07, 2002 at 12:51:55PM -0700, Geoff Keating wrote:
> | > 
> | > > One bit, however, is quite odd.  It adds certain flags to
> | > > CXX_FOR_TARGET.  These flags are generated by
> | > > `libstdc++-v3/testsuite_flags --build-includes`.  This is only done if
> | > > 'testsuite_flags' exists, which is only true if libstdc++-v3 has been
> | > > configured.
> | > 
> | > If you don't get a better answer, my suggestion would be to delete the
> | > inclusion of these flags from a local combined tree, and try to build
> | > and test.  If the flags are necessary, something should break...
> | > 
> | > I suspect they're necessary for building libstdc++-v3 or libjava, and
> | > that the weirdness is actually a bug.
> | Heh.  Actually, that's the one thing I'm absolutely sure they're not
> | necessary for, since they're explicitly avoided in those two contexts!
> 
> I'm not sure I understand your comment.  Maybe the name of script is
> ill-choosen but at the time it was first created that name was
> descriptive. 
If you look carefully at the top level configure.in, you will note the
comment 'Don't use libstdc++-v3's flags to configure/build itself.'
The way the macros are set up explicitly avoids using the flags from the
testsuite_flags script when the directory being dealt with is 'libjava'
or 'libstdc++-v3'.  This is presumably because the script doesn't exist
until libstdc++-v3 is configured.

Presumably its flags are used for c++ tests, but they'd have to be
explicitly referenced in the *subdirectory* libstdc++-v3, since they
don't get passed down from the top level for that subdirectory.

> Initially, we used it to generare flags necessary to properly build
> and test libstdc++-v3.  Then as things were growing we factorized all
> the places where we needed to know which flags are appropriate to use
> libstdc++ in that script.  In short, any program that needs to locate
> g++ and the library headers should invoke that script to figure out
> which flags to use.  That is true for the testsuite; that should be
> true for libgcj (we already had that discussion in the past).
> 
> -- Gaby

Thanks for the historical info.

Re: Deciphering flags in CXX_FOR_TARGET

[Nathanael Nerode]

On Tue, May 07, 2002 at 09:24:11PM -0300, Alexandre Oliva wrote:
> On May  7, 2002, Nathanael Nerode <neroden@doctormoo.dyndns.org> wrote:
> 
> > Perhaps an 'install-for-build' in each subdirectory which would
> > copy/move/link/symlink its output files to a tree mirroring the install
> > tree would do the trick?
> 
> Nope.  We can't install into the build tree, we really have to build
> the tools into the tree-for-install, and install them from there.
> 
> This is critical for correct functioning of libtool-build libraries
> and executables.  They're created with a notion of where they're
> expected to be installed, and installing them elsewhere doesn't always
> produce correct results.  But libtool can tell `build' from `install'
> files, so if we build them in a location in the build tree in which
> they will work, and then install them from there, things will work
> fine.
> 
> > Perhaps we could just convince each subdirectory to put its files in
> > a reasonable place without calling 'install', but that sounds harder.
> 
> Unfortunately this is what we have to do for correct functioning, at
> least for pieces that use libtool.
> 
> There's a longer message about this in a thread started by Per
> Bothner, IIRC, in late February this year.  You may want to have a
> look into that.  I've never wanted to ask you to do that cause it
> might scare you away, but since you got this far, and you appear to be
> interested in tackling this problem, I think it will do more good to
> get you involved now :-)

I think I'll hit this problem *after* I get the autoconfiscation
working.  I've got a workaround for the bizarre CXX_FOR_TARGET flags
issue which I think I can make work.  (It means I'm moving code from the
configure file to the Makefile, the opposite of what I usually try to
do...)

Thanks for the info.

Re: Deciphering flags in CXX_FOR_TARGET

[Gabriel Dos Reis]

Nathanael Nerode <neroden@doctormoo.dyndns.org> writes:

[...]

| > | > I suspect they're necessary for building libstdc++-v3 or libjava, and
| > | > that the weirdness is actually a bug.
| > | Heh.  Actually, that's the one thing I'm absolutely sure they're not
| > | necessary for, since they're explicitly avoided in those two contexts!
| > 
| > I'm not sure I understand your comment.  Maybe the name of script is
| > ill-choosen but at the time it was first created that name was
| > descriptive. 
| If you look carefully at the top level configure.in, you will note the
| comment 'Don't use libstdc++-v3's flags to configure/build itself.'

That comment is right since the flags output by testsuite_flags do
depend on information passed down from the toplevel configure.in.

| The way the macros are set up explicitly avoids using the flags from the
| testsuite_flags script when the directory being dealt with is 'libjava'
| or 'libstdc++-v3'.  This is presumably because the script doesn't exist
| until libstdc++-v3 is configured.

testsuite_flags is generated from testsuite_flags.in during
libstdc++-v3 configuration.

| Presumably its flags are used for c++ tests,

Mostly.  But any client built at the same time and that needs to
locate V3 headers has to invoke that script in order to get the
correct values.

-- Gaby