Obsolete building in source dir?

Obsolete building in source dir?

[Giovanni Bajo]

Hello,

Do we still need to support builds in source directory? It looks like it does
not work anymore not even on primary targets because of recent changes. We
discouraged doing it for a long time already, and no developers probably test
this so it is going to be often (always) broken. Maybe is it time to obsolete
(or forbid) this feature for 4.0.0, by having configure emit an error and exit?

Giovanni Bajo

Re: Obsolete building in source dir?

[DJ Delorie]

"Giovanni Bajo" <giovannibajo@libero.it> writes:
> Do we still need to support builds in source directory?

Since every other source package on the planet (it seems) supports
building in the source directory, gcc should too.  To do otherwise
would require an SC decision, as it would make gcc incompatible with
everything else.

> It looks like it does not work anymore not even on primary targets
> because of recent changes.

So?  Treat it like any other bug.  Find out who broke it, and get them
to fix it.

Re: Obsolete building in source dir?

[Joe Buck]

On Sun, Sep 12, 2004 at 11:07:12PM -0400, DJ Delorie wrote:
> "Giovanni Bajo" <giovannibajo@libero.it> writes:
> > Do we still need to support builds in source directory?
> 
> Since every other source package on the planet (it seems) supports
> building in the source directory, gcc should too.  To do otherwise
> would require an SC decision, as it would make gcc incompatible with
> everything else.

The SC cannot force people to submit patches to fix the problem;
building in the source directory has been at least partially broken
for years.

I don't think that this is an SC matter.  The SC only has the power to
say no (as in demanding that a patch that breaks functionality not
be accepted), likewise the RM. 

Re: Obsolete building in source dir?

[Zack Weinberg]

Joe Buck <Joe.Buck@synopsys.COM> writes:

> On Sun, Sep 12, 2004 at 11:07:12PM -0400, DJ Delorie wrote:
>> "Giovanni Bajo" <giovannibajo@libero.it> writes:
>> > Do we still need to support builds in source directory?
>> 
>> Since every other source package on the planet (it seems) supports
>> building in the source directory, gcc should too.  To do otherwise
>> would require an SC decision, as it would make gcc incompatible with
>> everything else.
>
> The SC cannot force people to submit patches to fix the problem;
> building in the source directory has been at least partially broken
> for years.

Could someone remind me what was wrong with the "if configure notices
it's been invoked in the source directory, create a build directory
behind the user's back" idea?  That wouldn't break all the time.

zw

Re: Obsolete building in source dir?

[Joseph S. Myers]

On Tue, 14 Sep 2004, Zack Weinberg wrote:

> Could someone remind me what was wrong with the "if configure notices
> it's been invoked in the source directory, create a build directory
> behind the user's back" idea?  That wouldn't break all the time.

http://gcc.gnu.org/ml/gcc-patches/2002-06/msg01540.html

has that patch and subsequent discussion leading to its reversion.

I'd be happy with both doing that again and converting relative srcdir to 
an absolute path unconditionally to get rid of bugs such as 13993 and 
reduce the number of variations Makefiles need to handle to one (building 
out of srcdir with an absolute path to srcdir).

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
  http://www.srcf.ucam.org/~jsm28/gcc/#c90status - status of C90 for GCC 4.0
    jsm@polyomino.org.uk (personal mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)

Re: Obsolete building in source dir?

[DJ Delorie]

> > Since every other source package on the planet (it seems) supports
> > building in the source directory, gcc should too.  To do otherwise
> > would require an SC decision, as it would make gcc incompatible with
> > everything else.
> 
> The SC cannot force people to submit patches to fix the problem;
> building in the source directory has been at least partially broken
> for years.

The SC decision is "do we *want* to support building in srcdir, or
should we reject such attempts?"  Once we know what the SC wants, we
can find someone to make the changes (probably me).

There are a few of us who actively fix build-in-src bugs, mostly
because we feel it's important to act like other GNU packages, but we
can stop feeling that way if the SC deems it so.

> I don't think that this is an SC matter.  The SC only has the power to
> say no (as in demanding that a patch that breaks functionality not
> be accepted), likewise the RM. 

I think it's appropriate for the SC to get past the political issue of
*wanting* to support it or not.  It's somewhat controversial (based on
historical arguments about it) whether the effort spent on maintaining
it is worth it.

Last time I worked on it, I think there was one libfoo directory that
still needed a tweak; libiberty builds in srcdir just fine, and only
directories that get built more than once are of concern anyway.

Re: Obsolete building in source dir?

[DJ Delorie]

> Could someone remind me what was wrong with the "if configure
> notices it's been invoked in the source directory, create a build
> directory behind the user's back" idea?  That wouldn't break all the
> time.

It's the "behind the user's back" that's a problem.  I think that we
should either make it do what the user expected, or tell the user they
can't do that, but to do something the user doesn't expect is bad.

Plus, the toplevel stuff is shared among many projects, and only gcc
has a problem with building in srcdir.

Re: Obsolete building in source dir?

[Joe Buck]

On Tue, Sep 14, 2004 at 03:43:51PM -0400, DJ Delorie wrote:
> 
> > > Since every other source package on the planet (it seems) supports
> > > building in the source directory, gcc should too.  To do otherwise
> > > would require an SC decision, as it would make gcc incompatible with
> > > everything else.
> > 
> > The SC cannot force people to submit patches to fix the problem;
> > building in the source directory has been at least partially broken
> > for years.
> 
> The SC decision is "do we *want* to support building in srcdir, or
> should we reject such attempts?"  Once we know what the SC wants, we
> can find someone to make the changes (probably me).

I would consider it a technical matter, not a big enough issue for the
SC to step in.  So s/SC/GWP maintainers/.  If folks with the authority
to approve patches like your patches, cool.  I'm not intent on using
the SC to browbeat them into it if they consider that too many kludges
are required.

Re: Obsolete building in source dir?

[Zack Weinberg]

DJ Delorie <dj@redhat.com> writes:

>> Could someone remind me what was wrong with the "if configure
>> notices it's been invoked in the source directory, create a build
>> directory behind the user's back" idea?  That wouldn't break all the
>> time.
>
> It's the "behind the user's back" that's a problem.  I think that we
> should either make it do what the user expected, or tell the user they
> can't do that, but to do something the user doesn't expect is bad.

Given all the weird stuff we do _anyway_ - invoking subconfigures from
the Makefile instead of the top level configure, building stuff
multiple times with different compilers, etc. etc. - I don't see this
as that much more surprising.  Just to be clearly documented.

> Plus, the toplevel stuff is shared among many projects, and only gcc
> has a problem with building in srcdir.

This raises the question of why gcc has the problem and the other
projects don't.  I venture to speculate it's because of the bootstrap
logic.  If I'm right, then toplevel bootstrap offers a chance to make
it go away (or, turnabout, to spread the problem to more projects).

zw

Re: Obsolete building in source dir?

[DJ Delorie]

> This raises the question of why gcc has the problem and the other
> projects don't.

I suspect it's because gcc is the only project where the organization
of the build tree doesn't match the organization of the source tree.
Between host/build/target modules and multilibs, we move stuff around
a lot.  Keeping track of where we put it is where we lose the most.  I
suspect that making srcdir absolute will fix most of the hard
problems, and we can conditional that on finding gcc in the source
tree.

if [ -d $srcdir/gcc ]
then
  srcdir=`cd $srcdir; $PWDCMD`
fi

Similarly for the build directory.  The down side is that you can't
move a build any more, but I've found that to be true already.

Re: Obsolete building in source dir?

[Loren James Rittle]

> I would consider it a technical matter, not a big enough issue for the
> SC to step in. [...]

Agreed, the documentation already tells users how to get best results.
Those that refuse to read it or believe it, can report a bug when their
build stops half-way through.  That is how all bugs, not seen by gcc
developers, are fixed.  In any event:

http://people.freebsd.org/~ljrittle/daily-gcc-bootstrap-reports/ref4/STATUS

gcc 3.1   20020513, objdir configuration, gmake, bootstrapped OK
gcc 3.1   20020513, srcdir configuration, gmake, bootstrapped OK
gcc 3.1.1         , objdir configuration, gmake, bootstrapped OK
gcc 3.1.1         , srcdir configuration, gmake, bootstrapped OK
gcc 3.2   20020813, objdir configuration, gmake, bootstrapped OK
gcc 3.2   20020813, srcdir configuration, gmake, bootstrapped OK
gcc 3.2.1         , objdir configuration, gmake, bootstrapped OK
gcc 3.2.1         , srcdir configuration, gmake, bootstrapped OK
gcc 3.2.2 20030129, objdir configuration, gmake, bootstrapped OK
gcc 3.2.2 20030129, srcdir configuration, gmake, bootstrapped OK
gcc 3.3   20030421, srcdir configuration, gmake, bootstrapped OK
gcc 3.3   20030421, objdir configuration, gmake, bootstrapped OK
gcc 3.3           , srcdir configuration, gmake, bootstrapped OK
gcc 3.3           , objdir configuration, gmake, bootstrapped OK
gcc 3.3.1 20030803, srcdir configuration, gmake, bootstrapped OK
gcc 3.3.1 20030803, objdir configuration, gmake, bootstrapped OK
[This testing machine non-functional during 3.3.2 & 3.3.3 & 3.4 release cycles.]
gcc 3.4.1         , srcdir configuration, gmake, bootstrapped OK
gcc 3.4.1         , objdir configuration, gmake, bootstrapped OK
gcc 3.4.2         , srcdir configuration, gmake, bootstrapped OK
gcc 3.4.2         , objdir configuration, gmake, bootstrapped OK

(The dated versions are close to a release.)

I have no idea about the state of mainline but...  Once a 4.0.0 branch
is cut and a daily "srcdir configuration" tester is moved to it, I'm
sure we will fix any bugs before the release.  If we don't, then I
will propose the needed documentation patch that I always pull out
whenever it is required...

Regards,
Loren

Re: Obsolete building in source dir?

[Richard Earnshaw]

On Tue, 2004-09-14 at 20:47, DJ Delorie wrote:
> > Could someone remind me what was wrong with the "if configure
> > notices it's been invoked in the source directory, create a build
> > directory behind the user's back" idea?  That wouldn't break all the
> > time.
> 
> It's the "behind the user's back" that's a problem.  I think that we
> should either make it do what the user expected, or tell the user they
> can't do that, but to do something the user doesn't expect is bad.

If we print out 'creating build directory in <target>-build...
configuring in <target>-build' then it's no-longer 'behind their back'.

R.

Re: Obsolete building in source dir?

[DJ Delorie]

> If we print out 'creating build directory in <target>-build...
> configuring in <target>-build' then it's no-longer 'behind their back'.

Assuming the user catches that line in the heaping gobs of output
configure produces (especially when builds are automated).  Better to
say:

$ ./configure
configure: Please create a separate build directory for builds including gcc.

$

... so that the user KNOWS.

But, I'd prefer to just fix the bugs.  That's what we usually do (and
have historically done) when we find them.

I'm amenable to making $srcdir aboslute when it's relative, too.  That
should paper over a number of PRs in a relatively harmless way,
breaking only net installs where the mount points differ from machine
to machine (which is a bad practice anyway IMHO).

Re: Obsolete building in source dir?

[Richard Earnshaw]

On Wed, 2004-09-15 at 14:08, DJ Delorie wrote:
> > If we print out 'creating build directory in <target>-build...
> > configuring in <target>-build' then it's no-longer 'behind their back'.
> 
> Assuming the user catches that line in the heaping gobs of output
> configure produces (especially when builds are automated).  Better to
> say:
> 
> $ ./configure
> configure: Please create a separate build directory for builds including gcc.
> 
> $
> 
> ... so that the user KNOWS.

Why does the user have to KNOW this?  Why would they care?  We don't
assume that they have to know a lot of other things.  All they want to
do is build the compiler so they can install it.  I haven't seen a
single reason why they should care that this is done in a subdirectory
of the source rather than in the sources themselves.

R.

Re: Obsolete building in source dir?

[Phil Edwards]

On Wed, Sep 15, 2004 at 02:28:14PM +0100, Richard Earnshaw wrote:
> On Wed, 2004-09-15 at 14:08, DJ Delorie wrote:
> > > If we print out 'creating build directory in <target>-build...
> > > configuring in <target>-build' then it's no-longer 'behind their back'.
> > 
> > Assuming the user catches that line in the heaping gobs of output
> > configure produces (especially when builds are automated).  Better to
> > say:

This is a strawman argument.

You are proposing the existence of a user who

    - is building a compiler from scratch
    - is automating the build (!)
    - gives a flying rat's ass about the location of the build directory

but who also

    - has never read the installation instructions
    - isn't on the mailing lists
    - doesn't log the output of configure and
    - can't read config.log

I refuse to believe that such a user exists until you provide concrete
evidence.


> Why does the user have to KNOW this?  Why would they care?  We don't
> assume that they have to know a lot of other things.  All they want to
> do is build the compiler so they can install it.  I haven't seen a
> single reason why they should care that this is done in a subdirectory
> of the source rather than in the sources themselves.

Ditto.  I've never heard any argument why this should matter to the end
user in the slightest.

They don't care that, for example, our source code comments are written in
grammatically proper English.  /We/ care.  Likewise, changing how in-srcdir
builds work isn't being done for the user's convenience.  It's for /us/.


-- 
<The_Vulture> evilgeek: actually it's <: and :>, <% and %>
<evilgeek> oh, right. digraphs are always happy

Re: Obsolete building in source dir?

[Alexandre Oliva]

On Sep 14, 2004, DJ Delorie <dj@redhat.com> wrote:

>> Could someone remind me what was wrong with the "if configure
>> notices it's been invoked in the source directory, create a build
>> directory behind the user's back" idea?  That wouldn't break all the
>> time.

> It's the "behind the user's back" that's a problem.  I think that we
> should either make it do what the user expected, or tell the user they
> can't do that, but to do something the user doesn't expect is bad.

What if we changed the top-level machinery such that, instead of using
a directory such as `gcc' to build GCC for the host, it used say
`host-gcc', just like we use `build-libiberty' for libiberty built for
the build machine?  Then nothing would be configured/built in the
source tree, except for the top-level, that isn't sensitive to these
matters.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: Obsolete building in source dir?

[Ralf Corsepius]

On Wed, 2004-09-15 at 22:29, Alexandre Oliva wrote:
> On Sep 14, 2004, DJ Delorie <dj@redhat.com> wrote:
> 
> >> Could someone remind me what was wrong with the "if configure
> >> notices it's been invoked in the source directory, create a build
> >> directory behind the user's back" idea?  That wouldn't break all the
> >> time.
> 
> > It's the "behind the user's back" that's a problem.  I think that we
> > should either make it do what the user expected, or tell the user they
> > can't do that, but to do something the user doesn't expect is bad.
> 
> What if we changed the top-level machinery such that, instead of using
> a directory such as `gcc' to build GCC for the host, it used say
> `host-gcc', just like we use `build-libiberty' for libiberty built for
> the build machine?  Then nothing would be configured/built in the
> source tree, except for the top-level, that isn't sensitive to these
> matters.
Instead of doing this you could consider to systematically using
$host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
no matter if cross-compiling or not.

Ralf

Re: Obsolete building in source dir?

[DJ Delorie]

> Instead of doing this you could consider to systematically using
> $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
> no matter if cross-compiling or not.

Except we can't, because the toplevel machinery is shared among many
projects.  We have to be careful not to change those without their
permission.

Re: Obsolete building in source dir?

[Matthias B.]

On Tue, 14 Sep 2004 15:47:24 -0400 DJ Delorie <dj@redhat.com> wrote:

> It's the "behind the user's back" that's a problem.  I think that we
> should either make it do what the user expected, or tell the user they
> can't do that, but to do something the user doesn't expect is bad.

As a user I agree with that argument and I think the easiest solution
would be to do it on the packaging level. Rather than releasing a tarball
like this

gcc-3.4.1/
  INSTALL/
  boehm-gc/
  ...


distribute one like this:

gcc-3.4.1/
  configure
  Makefile
  gcc-build/
  gcc-src/
     INSTALL/
     boehm-gc/
     ...


where configure and Makefile do the obvious thing and cd into gcc-build
before delegating to ../gcc-src/{configure,Makefile}

As I understand it, the desire to support building from srcdir is more
accurately described as a desire to support building from "the directory
created when extracting the tarball" in order to streamline the build
process for users of the release tarballs. The above packaging solution
achieves that without doing anything behind the user's back or interfering
with the GCC build system.

MSB

-- 
Conform, go crazy, or become an artist.

Re: Obsolete building in source dir?

[Ralf Corsepius]

On Thu, 2004-09-16 at 18:38, DJ Delorie wrote:
> > Instead of doing this you could consider to systematically using
> > $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
> > no matter if cross-compiling or not.
> 
> Except we can't, because the toplevel machinery is shared among many
> projects.  We have to be careful not to change those without their
> permission.
AFAIU, your concern only applies to subpackages which support "native
compilation only". Do such packages exist within the "uberbaum"?

I am aware about fundamental issues with adalib and about potential
issues with tcl/tk based subpackages, nevertheless all of them can be
expected to support VPATH builds, so I don't see that moving their
--srcdir would break building them. 

I am aware that some configure scripts and Makefiles apply nasty tricks
such as test'ing for presence of tools in neighboring build-directories.
These checks/tests probably would require to be adapted in some cases. 

Ralf

Re: Obsolete building in source dir?

[Alexandre Oliva]

On Sep 16, 2004, Ralf Corsepius <rc040203@freenet.de> wrote:

> Instead of doing this you could consider to systematically using
> $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
> no matter if cross-compiling or not.

This wouldn't quite work if we actually need different builds for say
build, host and target, but they're all the same.

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: Obsolete building in source dir?

[DJ Delorie]

> > Except we can't, because the toplevel machinery is shared among many
> > projects.  We have to be careful not to change those without their
> > permission.
> AFAIU, your concern only applies to subpackages which support "native
> compilation only".

No, it's a policy issue.  We can't decide to change the way their
projects work without their agreement.

Re: Obsolete building in source dir?

[Kai Henningsen]

aoliva@redhat.com (Alexandre Oliva)  wrote on 17.09.04 in <oru0twubfc.fsf@free.redhat.lsd.ic.unicamp.br>:

> On Sep 16, 2004, Ralf Corsepius <rc040203@freenet.de> wrote:
>
> > Instead of doing this you could consider to systematically using
> > $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
> > no matter if cross-compiling or not.
>
> This wouldn't quite work if we actually need different builds for say
> build, host and target, but they're all the same.

That's trivial to fix by using a {host,build,target}- prefix.

MfG Kai

Re: Obsolete building in source dir?

[Alexandre Oliva]

On Sep 18, 2004, kaih@khms.westfalen.de (Kai Henningsen) wrote:

> aoliva@redhat.com (Alexandre Oliva)  wrote on 17.09.04 in <oru0twubfc.fsf@free.redhat.lsd.ic.unicamp.br>:
>> On Sep 16, 2004, Ralf Corsepius <rc040203@freenet.de> wrote:
>> 
>> > Instead of doing this you could consider to systematically using
>> > $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
>> > no matter if cross-compiling or not.
>> 
>> This wouldn't quite work if we actually need different builds for say
>> build, host and target, but they're all the same.

> That's trivial to fix by using a {host,build,target}- prefix.

Which brings us back to exactly what I had suggested before Ralf's
counter-proposal :-)

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: Obsolete building in source dir?

[Kai Henningsen]

aoliva@redhat.com (Alexandre Oliva)  wrote on 18.09.04 in <or8yb77c6m.fsf@livre.redhat.lsd.ic.unicamp.br>:

> On Sep 18, 2004, kaih@khms.westfalen.de (Kai Henningsen) wrote:
>
> > aoliva@redhat.com (Alexandre Oliva)  wrote on 17.09.04 in
> > <oru0twubfc.fsf@free.redhat.lsd.ic.unicamp.br>:
> >> On Sep 16, 2004, Ralf Corsepius <rc040203@freenet.de> wrote:
> >>
> >> > Instead of doing this you could consider to systematically using
> >> > $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
> >> > no matter if cross-compiling or not.
> >>
> >> This wouldn't quite work if we actually need different builds for say
> >> build, host and target, but they're all the same.
>
> > That's trivial to fix by using a {host,build,target}- prefix.
>
> Which brings us back to exactly what I had suggested before Ralf's
> counter-proposal :-)

Wrong. You proposed build-gcc, he proposed $buils(_alias)-gcc, I proposed  
build-$build(_alias)-gcc.

MfG Kai

Re: Obsolete building in source dir?

[Ralf Corsepius]

On Sun, 2004-09-19 at 14:01, Kai Henningsen wrote:
> aoliva@redhat.com (Alexandre Oliva)  wrote on 18.09.04 in <or8yb77c6m.fsf@livre.redhat.lsd.ic.unicamp.br>:
> 
> > On Sep 18, 2004, kaih@khms.westfalen.de (Kai Henningsen) wrote:
> >
> > > aoliva@redhat.com (Alexandre Oliva)  wrote on 17.09.04 in
> > > <oru0twubfc.fsf@free.redhat.lsd.ic.unicamp.br>:
> > >> On Sep 16, 2004, Ralf Corsepius <rc040203@freenet.de> wrote:
> > >>
> > >> > Instead of doing this you could consider to systematically using
> > >> > $host(_alias), $build(_alias), $target(_alias) builddir-subdirectories,
> > >> > no matter if cross-compiling or not.
> > >>
> > >> This wouldn't quite work if we actually need different builds for say
> > >> build, host and target, but they're all the same.
> >
> > > That's trivial to fix by using a {host,build,target}- prefix.
> >
> > Which brings us back to exactly what I had suggested before Ralf's
> > counter-proposal :-)
> 
> Wrong. You proposed build-gcc, he proposed $buils(_alias)-gcc,
No, I proposed
$({build|host|target})/<subpackage>

or

$({build|host|target}_alias)/<subpackage>

i.e. to extend the building scheme that _already_ is applied for
building target-subpackages in case of cross-compilation (aka.
--target-subdir)

I am already applying a variant this configuration scheme in other
(non-gcc) packages.

>  I proposed  
> build-$build(_alias)-gcc.

Ralf

Re: Obsolete building in source dir?

[Marc Espie]

I have trouble seeing gcc not building in src dir being a problem.
All the distributors I know don't have any issue with building stuff
with srcdir somewhere and objdir somewhere. Heck, most of them currently
insist on it anyways.

Does anyone have an instance of a current vendor build system that can't
cope with srcdir != objdir ?

Individual users can be very easily unconfused by the documentation.

Re: Obsolete building in source dir?

[Per Bothner]

Marc Espie wrote:

> I have trouble seeing gcc not building in src dir being a problem.

gcc building is src dir is not important.

Having (./configure && make && make install) work is important, IMO,
since that is the "standard" way to build GNU (and lotsof other)
software.
-- 
	--Per Bothner
per@bothner.com   http://per.bothner.com/