Require GNU Make Somehow?

Require GNU Make Somehow?

[Craig Burley]

One problem I've seen a lot is people configuring for separate
source and build directories, then forgetting to use GNU make.

The result is often a bug report that, at first glance (especially
to me, since I'm still just learning how these things work),
seems to be a "real problem", but is easily addressed by asking
the person "are you using GNU make".

To cut down on these kinds of problems, would it be possibly
to find a reliable, sure-fire way to detect and report this
situation ASAP to the user, using very clear language?

I'm thinking of something like this:

  # ../sourcedir/configure ...
  <configure does its work>
  # make ...
  Sorry: you are not using GNU make, FOOBAR make, or BLETCH make,
    and most other variants of make are known to have problems
    with configurations that have separate build and source
    directories.  Please retry with a version of make that is
    known to work.
  Exit 1
  #

Whether it's even desirable is, I would think, issue #1.  I'd
sure like it, just to cut down on the bug reports I get.

If "yes", #2 is probably how to do it in the way that is most
certain to get the "right" answer.

My guess is that doing it "right" means modifying autoconf,
maybe automake, and the variations gcc/egcs might use that
aren't "vanilla", to do the following:

  -  Detect that source and build directories are different

  -  If so, produce a makefile setup that "slides through" just
     fine when processed by a known-working make, but that
     produces a diagnostic like the above when processed by
     any other make

Ideally, it'd be easy for a "knowledgable" user to edit the
makefile setup to allow a not-known-working make to work
on that setup.

Also, just how to get the makefile setup to do its rejecting
regardless of the target(s) specified on the command line
is something I can't figure out offhand.  Maybe just handling
the "common" targets is fine; maybe just putting explicit
dependencies on a check-for-known-working-make target is fine;
I don't know.

Opinions on the desirability, at least?

        tq vm, (burley)

Re: Require GNU Make Somehow?

[Joe Buck]

> One problem I've seen a lot is people configuring for separate
> source and build directories, then forgetting to use GNU make.

Some makes are bad enough that building in the same directory doesn't
work either.  The correct autoconf way of doing things seems to be to
run tests on the make that is found, to see if it has problems.

>   # ../sourcedir/configure ...
>   <configure does its work>
>   # make ...
>   Sorry: you are not using GNU make, FOOBAR make, or BLETCH make,

No, because the next release of FOOBAR make may fix the bug.  The
other is that many people have GNU make installed as gmake; if so,
the generated makefile might try to feed itself to gmake.

> My guess is that doing it "right" means modifying autoconf,
> maybe automake, and the variations gcc/egcs might use that
> aren't "vanilla", to do the following:

No mods to autoconf or automake would be needed.  Adding new stuff to
configure.in should be enough.  configure.in can ask for any test it
wants; autoconf provides a fairly general mechanism for executing a
command and seeing if it successful.  The user might see

Testing 'make' to see if VPATH works ... no
Sorry, can't build with --srcdir different from the build directory.
You'll need a better make program, such as GNU make.

Or

Testing 'make' to see if VPATH works ... no
Testing 'gmake' to see if VPATH works ... yes
************ IMPORTANT! *******************
************ IMPORTANT! *******************
You *must* build with gmake rather than with make!!!!!
************ IMPORTANT! *******************
************ IMPORTANT! *******************


>   -  Detect that source and build directories are different

Easy to do in configure.in

>   -  If so, produce a makefile setup that "slides through" just
>      fine when processed by a known-working make, but that
>      produces a diagnostic like the above when processed by
>      any other make

Better to just have configure fail if make is not powerful enough
(telling people what their options are).

Re: Require GNU Make Somehow?

[Craig Burley]

>> One problem I've seen a lot is people configuring for separate
>> source and build directories, then forgetting to use GNU make.
>
>Some makes are bad enough that building in the same directory doesn't
>work either.  The correct autoconf way of doing things seems to be to
>run tests on the make that is found, to see if it has problems.

Eek, well, let me explain a bit more what I'm concerned about.

If joe luser is stuck with a make so buggy that even building in
the same directory doesn't work, I'm hoping that he is already
aware of the deficiencies or will suitably address them as soon
as he discovers them.

But, right now, I'm a bit worried (and maybe too much so) that,
for example, the cleaning up I've done of the g77 configure and
build procedures for g77 0.5.23 (which will be based on gcc 2.8)
will start "breaking" for people who have "basically working"
makes, perhaps even working well enough to have previously
worked in build-separate-from-source builds.  Ideally I'd like
to forestall a flood of bug reports from various systems, each
looking menacingly different from the other, all of which turn
out to be "oops, gotta use gmake", though I think it's too late
for me to try and do that in g77, especially since it's best done
in gcc/egcs and other "top-level" products.

>>   # ../sourcedir/configure ...
>>   <configure does its work>
>>   # make ...
>>   Sorry: you are not using GNU make, FOOBAR make, or BLETCH make,
>
>No, because the next release of FOOBAR make may fix the bug.  The
>other is that many people have GNU make installed as gmake; if so,
>the generated makefile might try to feed itself to gmake.

(Actually the wording was intended to say "you are not using one of:
GNU make; FOOBAR make; or BLETCH make", which goes to show how
hard it is to write unambiguously.  Even so, of course, the next
release of FOOBAR make might add a *new* bug.  ;-)

Basically I was just looking for whether it would be a) worthwhile
and b) possible to, up-front, distribute products and packages
with whatever knowledge we happen to possess at the time that
a given version, or variant, of make will *not* work on
a particular configuration -- knowledge encapsulated such that,
if the user attempts the buggy thing, he gets a nice, quick,
clear diagnostic saying "this won't work because..." rather than,
50 minutes into the build, "don't know how to build gcc/f/runtime/foo",
which will trigger a bug report 99% of the time or so.

The latter sentence, "if so, the generated makefile might try to
feed itself to gmake", I'm not sure how to interpret.  If you mean
we could design this proposed mechanism so that, instead of complaining
and exiting, it complained (maybe) and tried to find a GNU make
to continue the process, then that's a rather interesting idea!

>> My guess is that doing it "right" means modifying autoconf,
>> maybe automake, and the variations gcc/egcs might use that
>> aren't "vanilla", to do the following:
>
>No mods to autoconf or automake would be needed.  Adding new stuff to
>configure.in should be enough.  configure.in can ask for any test it
>wants; autoconf provides a fairly general mechanism for executing a
>command and seeing if it successful.  The user might see
>
>Testing 'make' to see if VPATH works ... no
>Sorry, can't build with --srcdir different from the build directory.
>You'll need a better make program, such as GNU make.
>
>Or
>
>Testing 'make' to see if VPATH works ... no
>Testing 'gmake' to see if VPATH works ... yes
>************ IMPORTANT! *******************
>************ IMPORTANT! *******************
>You *must* build with gmake rather than with make!!!!!
>************ IMPORTANT! *******************
>************ IMPORTANT! *******************

Right, this is definitely worth thinking about, though it's a
bit "friendlier" in some ways and less-so in others than what I
was thinking about.  My idea was less about producing more stuff
for the user who is doing things "right" to read, but something
more like a thing for the user who does the wrong thing to
quickly, effectively, and clearly "stub his toe" on.

E.g. I'd like to "catch" the case where people just have scripts
that do "../sourcedir/configure ...; make ...", in which case
they don't see the warnings.  Ideally they'll look for them
if the build crashes 50 minutes and 50KB of output later, but
when we're also outputting messages like "mv: cc1obj: Not found"
by the handful and expecting people to ignore them, it doesn't
surprise me that sometimes users feel they need only worry about
the last 20 or so lines before the crash-point when looking for
the cause of, or reporting, a build-time crash.

>>   -  Detect that source and build directories are different
>
>Easy to do in configure.in

Agreed.  Now, if there was a little boilerplate toe-stubbing
code I could stick into g77's configure.in that would stay nice
and quiet *except* when a different-directory configuration
was being made by a known-bad make, I'd love to know about it!
Presumably something to put into the generated Makefile, for example.

One thought I had was to simply produce a gnuMakefile, or whatever
that name is, instead of a Makefile, as the normal output of
the configuration process in *that* case (the different-directory
case), and to produce a "dummy" Makefile with "stub" targets and
the "@echo" statements that generate the nice diagnostic.

But I don't like this solution because, surely, there are non-GNU
makes that do implement VPATH and what-not correctly, and it's
hardly fair to give their users the impression that it doesn't
"work" just because it doesn't read gnuMakefile in preference to
Makefile.

(Maybe I should be asking some kind of "GNU tools" mailing list.
Forget the name of the one(s) I'm on, since I rarely see email
come from them.)

>>   -  If so, produce a makefile setup that "slides through" just
>>      fine when processed by a known-working make, but that
>>      produces a diagnostic like the above when processed by
>>      any other make
>
>Better to just have configure fail if make is not powerful enough
>(telling people what their options are).

In that case, they'd either have to put gmake in their path early
enough as `make', or be offered a way to override the default
`make' with a command name/path of their own choosing, to cause
configuration to work.

So, again, I'd still be a little worried about the case where
they do that (e.g. "MAKE=gmake ../sourcedir/configure ...") but
later "forget" about the requirement and just type "make",
or forget to make sure their path is still set up correctly.

I don't mind one bit if the ideas you are suggesting are implemented
without worrying about these nits of mine, because IMO that'd solve
most of the problems anyway, and I'm not even clear on how many
more times this problem is going to keep happening.

Guess I have a bit of a belt-and-suspenders mindset towards safety,
plus a vivid imagination when I get bug reports from people who
see buggy behavior in g77 on configurations almost exactly identical
to mine, but I can't reproduce the problem at all, since, often,
when it is finally learned what the problem is, it's a very subtle
configuration or library mismatch problem that stemmed from some
kind of build problem.

But, imagination aside, the "oops, forgot to use gmake" problem is
real.  I got *two* reports in one day that I think were due to that,
from two different g77 alpha testers.  Each of them sent their bug
reports to me privately instead of to the list -- in my experience,
that means they just didn't want to embarrass me in front of the
other testers (g77 alpha testers are *incredibly* polite, at least
on that list) -- and, when I couldn't offer any help (since I
couldn't reproduce the problem), one of them later followed up with
"oops, my fault, forgot to use gmake that one time", so I assume
that was the case with the other person as well.

So, I'd like to, at the very least, move the discovery of that
mistake up much earlier in time so people waste less time waiting
for the computer to discover the problem.  A general rule of thumb
I like to operate by is: make the computer do the repetitive,
trivial memorization kinds of work, let the human do the reasoning
and be allowed to forget things sometimes.  Remember that gmake
is required instead of make on a particular one of the five
systems one might be supporting in a given instalation really
is a job for the computer, ideally.

Kind of like that "build directory is subdirectory of source
directory" bug situation, I think it's always a good idea, when
a programmer knows some combination of user-controllable settings
won't work right, to at least try and ensure that the user gets
told "this won't work" as clearly and as soon as possible after
engaging those particular settings.

This philosophizing is probably overkill for this particular issue,
but I find it's useful to keep in mind when designing all sorts
of systems, to avoid unnecessary problems down the line, so
maybe it'll be worthwhile advice to somebody.

        tq vm, (burley)

Re: Require GNU Make Somehow?

[Dave Love]

>>>>> "Craig" == Craig Burley <burley@gnu.org> writes:

 >>> One problem I've seen a lot is people configuring for separate
 >>> source and build directories, then forgetting to use GNU make.

I agree with Craig that this should be addressed, especially as I'm
one of the g77 lusers he's referring to (trying to test in adverse
circumstances) and I'm supposed to be an expert.  There have also been
N bug reports on the egcs list through not using GNU make.

Joe Buck:

 >> Some makes are bad enough that building in the same directory
 >> doesn't work either.

I know this is the case for egcs.  Is it also for gcc 2.8, which I
haven't tried to build in srcdir?  I don't know of any make which
failed for gcc 2.7+g77 in srcdir.  

 >> The correct autoconf way of doing things seems to be to run tests
 >> on the make that is found, to see if it has problems.

Is it even clear what the set of problems is, though?  egcs says you
must use GNU make (though I'm inclined to regard that as a bug).  It
should be strightforward just to test that and complain.

 Craig> But, right now, I'm a bit worried (and maybe too much so)
 Craig> that, for example, the cleaning up I've done of the g77
 Craig> configure and build procedures for g77 0.5.23 (which will be
 Craig> based on gcc 2.8) will start "breaking" for people who have
 Craig> "basically working" makes, perhaps even working well enough to
 Craig> have previously worked in build-separate-from-source builds.

Actually, I don't think any make I've used worked outside srcdir for
building g77+gcc-2.7, though the g77 stuff itself was OK for that, at
least at one time; it was the base gcc which failed.

 Craig> Agreed.  Now, if there was a little boilerplate toe-stubbing
 Craig> code I could stick into g77's configure.in that would stay
 Craig> nice and quiet *except* when a different-directory
 Craig> configuration was being made by a known-bad make, I'd love to
 Craig> know about it!

I'm confused about a g77 configure.in as I thought there was policy
against one, but there used to be a test in config-lang.in for srcdir
!= builddir.  That just warned about the need for GNU make at
configure time.  I'll look into sorting something out for g77's
makefile, but it presumably won't kick in until g77 starts building.

 >> Better to just have configure fail if make is not powerful enough
 >> (telling people what their options are).

 Craig> In that case, they'd either have to put gmake in their path
 Craig> early enough as `make', or be offered a way to override the
 Craig> default `make' with a command name/path of their own choosing,
 Craig> to cause configuration to work.

Indeed.  If I want GNU make, I'd be (should be) typing `gmake' on a
non-GNU system and a test is needed at compile time, not configure
time.

Re: Require GNU Make Somehow?

[Craig Burley]

>I'm confused about a g77 configure.in as I thought there was policy
>against one, but there used to be a test in config-lang.in for srcdir
>!= builddir.  That just warned about the need for GNU make at
>configure time.  I'll look into sorting something out for g77's
>makefile, but it presumably won't kick in until g77 starts building.

Oops, I meant config-lang.in, not configure.in, you're right.  Yes,
the test and message have gone away; I don't mind resurrecting
them, but feel that's really the job of the top-level makefile
to do, in this case, gcc's and egcs's, and thinking down that
avenue led to my original email on this topic.

>Indeed.  If I want GNU make, I'd be (should be) typing `gmake' on a
>non-GNU system and a test is needed at compile time, not configure
>time.

Right, though it might be most straightforward and reliable to
have configure time lay a sort of "trap" that is either avoided
by using GNU make at compile time, or triggered by not using
GNU make then, and the effect of the trap is to print a nice
message and exit.

        tq vm, (burley)

Re: Require GNU Make Somehow?

[Jim Meyering]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

Craig Burley <burley@gnu.org> writes:
| >I'm confused about a g77 configure.in as I thought there was policy
| >against one, but there used to be a test in config-lang.in for srcdir
| >!= builddir.  That just warned about the need for GNU make at
| >configure time.  I'll look into sorting something out for g77's
| >makefile, but it presumably won't kick in until g77 starts building.
|
| Oops, I meant config-lang.in, not configure.in, you're right.  Yes,
| the test and message have gone away; I don't mind resurrecting
| them, but feel that's really the job of the top-level makefile
| to do, in this case, gcc's and egcs's, and thinking down that
| avenue led to my original email on this topic.
|
| >Indeed.  If I want GNU make, I'd be (should be) typing `gmake' on a
| >non-GNU system and a test is needed at compile time, not configure
| >time.
|
| Right, though it might be most straightforward and reliable to
| have configure time lay a sort of "trap" that is either avoided
| by using GNU make at compile time, or triggered by not using
| GNU make then, and the effect of the trap is to print a nice
| message and exit.

This isn't quite the same, since GNU make isn't required to
build the fileutils, but it might be useful...


[-- Attachment #2: GNUmakefile --]
[-- Type: text/x-makefile, Size: 653 bytes --]

# Having a separate GNUmakefile lets me `include' the dynamically
# generated rules created via Makefile.maint as well as Makefile.maint itself.
# This makefile is used only if you run GNU Make.
# It is necessary if you want to build targets usually of interest
# only to the maintainer.

have-Makefile := $(shell test -f Makefile && echo yes)

# If the user runs GNU make but has not yet run ./configure,
# give them a diagnostic.
ifeq ($(have-Makefile),yes)

include Makefile
include $(srcdir)/Makefile.maint

else

all:
	@echo There seems to be no Makefile in this directory.
	@echo "You must run ./configure before running \`make'."
	@exit 1

endif

Re: Require GNU Make Somehow?

[Craig Burley]

>This isn't quite the same, since GNU make isn't required to
>build the fileutils, but it might be useful...

Thanks!  We're going to try a simple solution in the next g77 0.5.23 alpha
release, which is a release that will be based on gcc 2.8:

# We want to try to catch the common trap of building outside srcdir
# without GNU make.  We do that by adding a phony target to the
# all the LANGUAGES dependencies, and doing the check in the phony target.
$(LANGUAGES): gnumake-check
gnumake-check:
        @if [ $(srcdir) != . ]; then \
          case `$(MAKE) --version 2>/dev/null` in \
          *GNU*) true ;; \
          *) echo 1>&2 ''; \
             echo 1>&2 'You must use GNU make to build outside the source directory.'; \
             echo 1>&2 ''; \
             false;; \
          esac; \
        else true; \
        fi

(This goes into g77's Make-lang.in, a fragment that gets pulled into
the "master" Makefile for gcc when configuring with g77 present.
$(LANGUAGES) is the list of languages the user wants to build; I'd
thought about making this just "F77 f77" to catch the problem only
when g77 itself gets built, but then realized that happens well
into a build, so this approach, which Dave Love came up with, seems
best, because it kicks back on error quite quickly after issuing
the non-GNU `make' command.)

        tq vm, (burley)

Re: Require GNU Make Somehow?

[Todd P. Whitesel]

Craig Burley <burley@gnu.org> writes:

>Guess I have a bit of a belt-and-suspenders mindset towards safety,
>plus a vivid imagination when I get bug reports from people who
>see buggy behavior in g77 on configurations almost exactly identical
>to mine, but I can't reproduce the problem at all, since, often,
>when it is finally learned what the problem is, it's a very subtle
>configuration or library mismatch problem that stemmed from some
>kind of build problem.

I have a lot of sympathy for this viewpoint. I have been burnt time and
time again by programs that outsmarted themselves trying to figure out
what I want so they could keep going and only obscure the error when it
finally fails hard. What I _want_ is for them to stop dead with an error
message that points directly at something in my source or build directory.

At my previous job there were a few of us who periodically sanitized
source directories to localize the places where configuration-dependent
things were stored. These few places were well publicized and were easy
to grep through or even browse manually. I could teach new employees how
to diagnose and fix most port-related bugs in about a day. This would
happen acceptably often because of people throwing in time/date code or
yet another directory slash piece of code; we'd drag it over to the
"toxic code dump" modules, and either fix it outright or leave asserts and
comments in it for the next guy. I can't vouch for the rest of the product
however -- this was Green Hills MULTI.

This scheme worked well because it was easy to communicate to new people what
to do when they ran into a problem, and new ports were much easier because
95% of what you changed was next to something else you'd have to change. We
accepted the inevitability of configuration bugs and made it a priority to
ensure that diagnosis and treatment of those bugs was fairly obvious.

So far I have found that configure is largely the opposite. If it does not
just work, then your chances of finding the problem are largely dependent
upon your ability to simulate bash scripts in your head. Eventually you get
enough of an idea what's going on in there and you can fix things without
going for primal scream therapy -- after which you tolerate configure and
even learn to admire it, but the fact remains that you've now joined the
ranks of the priesthood, and will be forever different from the other kids.

Could we have some kind of paranoid configure mode that did this for, say, gdb:

    $CC ${srcdir}/gmake/*.c -g -o ./gnu_make
    ./gnu_make CC=$CC SRCDIR=${srcdir} <etc.> gnu_sed gnu_bash gnu_grep <etc.>
    ./gnu_bash configure
    ./gnu_make all-gdb

For bootstrapping on a system with suspect /bin programs but a passable cc,
this would eliminate a lot of the unknowns. Once it's available to people it
can be used as a standard "sanity check" to determine if it's your system or
the GNU source tree that is having problems.

Todd Whitesel
toddpw @ ugcs.caltech.edu