Re: hierarchical projects with configure scripts

Re: hierarchical projects with configure scripts

[Sergio Durigan Junior]

[ Adding gdb-patches since this is an issue with GDB's Makefile as
well.  ]

On Wednesday, August 29 2018, Bruno Haible wrote:

> Hi Sergio, all,
>
>> I managed to reproduce this bug, and also to find the reason for it.
>
> Thanks for explaining it!
>
>> Basically, the gnulib machinery to identify the compiler features and
>> modify its flags is working OK, as can be seen by this excerpt from
>> gnulib/Makefile:
>> 
>>   ...
>>   CC=gcc -std=gnu11
>>   ...
>> 
>> Also, if I enter the gnulib build directory (which, for GDB, is located
>> at "gdb/build-gnulib/") and run "make", everything works fine.  The
>> problem happens when GDB's Makefile invokes gnulib's.  Here's how it
>> works.
>> 
>> GDB's Makefile uses this incantation to build gnulib:
>> 
>>   all-lib: $(GNULIB_BUILDDIR)/Makefile
>>           @$(MAKE) $(FLAGS_TO_PASS) DO=all DODIRS=$(GNULIB_BUILDDIR) subdir_do
>>   .PHONY: all-lib
>> 
>> The "subdir_do" rule is:
>> 
>>   subdir_do: force
>>           @for i in $(DODIRS); do \
>>                   ...
>>                   if [ -f ./$$i/Makefile ] ; then \
>>                           if (cd ./$$i; \
>>                                   $(MAKE) $(FLAGS_TO_PASS) $(DO)) ; then true ; \
>>                           else exit 1 ; fi ; \
>>                   else true ; fi ; \
>>           done
>> 
>> Which is correct, and should work at first glance.  However,
>> FLAGS_TO_PASS contains:
>> 
>>   FLAGS_TO_PASS = \
>>           ...
>>           "CC=$(CC)" \
>>           "CFLAGS=$(CFLAGS)" \
>>           "CXX=$(CXX)" \
>>           "CXX_DIALECT=$(CXX_DIALECT)" \
>>           "CXXFLAGS=$(CXXFLAGS)" \
>>           ...
>> 
>> Which ends up overriding gnulib's CC/CXX variables.  That's why we don't
>> see the "-std=gnu11" there.
>
> Other packages with separate configure scripts in subdirectories (e.g.
> GNU clisp) have similar issues.
>
> Namely, we have a conflict between
>   (a) the requirement that (CC, CFLAGS) for build is the same as
>       (CC, CFLAGS) for configure, in every subdirectory, and
>   (b) the desire of developers to be able to rebuild an entire source
>       tree with "make clean; make CFLAGS='-O0 -ggdb'" or so.
>
> In small packages the solution can be to merge all the configure
> scripts into the top-level one. This greatly reduces the configure
> times as well. But this is not feasible for GNU gdb or GNU gettext.
>
> Maybe the solution can be inspired by the line of thought Paul started
> in [1]. Namely:
>   Define a *small* set of variables that influence the configure
>   results. Currently these are CC, CFLAGS, LDFLAGS, but not CPPFLAGS.
> Then, can we define a set of variables that can be passed down from the
> top-level Makefile the subordinate Makefiles?
> These two sets of variables must be disjoint, and that is the problem
> here, because users would like to use CFLAGS to pass optimization and
> debugging flags down the build tree, after the configuration is complete.

Bruno,

Thanks for the further explanation and for proposing a solution.

I'm including the rest of the GDB developers in this discussion because
this involves tweaking our build system, so I'd like more people to be
able to give opinions here.

I'm still inclined to go the "easy way" and do what I proposed above:
create a "FLAGS_TO_PASS_TO_GNULIB" which would omit the CC/CXX (and
possibly other) variables, even if it's just to unbreak the s390x build
(I still have a gnulib issue happening with mingw to investigate, sigh).

> If we could redesign the GNU build system from scratch, we could
> define a variable GLOBAL_CFLAGS or RECURSIVE_CFLAGS that could be passed
> down by 'make'. And all Makefiles would have to use
>   $(CC) $(CFLAGS) $(GLOBAL_CFLAGS)
> instead of just
>   $(CC) $(CFLAGS).
>
> This would be the proper solution, but is a lot of work in the
> GNU Coding Standards, in Automake, and Autoconf.
>
>> Unless someone has a better idea, I'll propose a patch to not pass
>> CC/CXX to gnulib's Makefile on GDB.
>
> It's not only CC, CXX, but also CFLAGS, CXXFLAGS, LDFLAGS.

Right.  In our specific case, I think just omitting CC/CXX would be
enough (the "c99" module doesn't seem to affect other variables).

> Bruno
>
> [1] https://lists.gnu.org/archive/html/bug-gnulib/2017-11/msg00014.html

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: hierarchical projects with configure scripts

[Tom Tromey]

>>>>> "Sergio" == Sergio Durigan Junior <sergiodj@redhat.com> writes:

>>> "CC=$(CC)" \
>>> "CFLAGS=$(CFLAGS)" \
>>> "CXX=$(CXX)" \
>>> "CXX_DIALECT=$(CXX_DIALECT)" \
>>> "CXXFLAGS=$(CXXFLAGS)" \
>>> ...

>>> Which ends up overriding gnulib's CC/CXX variables.  That's why we don't
>>> see the "-std=gnu11" there.

>> Maybe the solution can be inspired by the line of thought Paul started
>> in [1]. Namely:
>> Define a *small* set of variables that influence the configure
>> results. Currently these are CC, CFLAGS, LDFLAGS, but not CPPFLAGS.
>> Then, can we define a set of variables that can be passed down from the
>> top-level Makefile the subordinate Makefiles?
>> These two sets of variables must be disjoint, and that is the problem
>> here, because users would like to use CFLAGS to pass optimization and
>> debugging flags down the build tree, after the configuration is complete.

Sergio> I'm still inclined to go the "easy way" and do what I proposed above:
Sergio> create a "FLAGS_TO_PASS_TO_GNULIB" which would omit the CC/CXX (and
Sergio> possibly other) variables, even if it's just to unbreak the s390x build
Sergio> (I still have a gnulib issue happening with mingw to investigate, sigh).

Historically the GNU and/or configure and/or automake rule was that
variables like CC, CXX, CFLAGS (etc) were for users.  So, following this
rule, I think it's correct for gdb to pass these to sub-configures and
sub-makes.

I haven't looked at the gnulib code here, but it seems to me that if
gnulib wants to find special compiler flags to build itself, then I
think those should be stuck into some other-named variable, not CFLAGS.
gdb does this itself as well, for example warning flags aren't added to
CXXFLAGS but some other variable.  In automake the convention is to name
these internal things AM_mumble, like AM_CFLAGS.  There's some
discussion in (info "(automake) Flag Variables Ordering")

Tom

Re: hierarchical projects with configure scripts

[Sergio Durigan Junior]

On Friday, August 31 2018, Tom Tromey wrote:

>>>>>> "Sergio" == Sergio Durigan Junior <sergiodj@redhat.com> writes:
>
>>>> "CC=$(CC)" \
>>>> "CFLAGS=$(CFLAGS)" \
>>>> "CXX=$(CXX)" \
>>>> "CXX_DIALECT=$(CXX_DIALECT)" \
>>>> "CXXFLAGS=$(CXXFLAGS)" \
>>>> ...
>
>>>> Which ends up overriding gnulib's CC/CXX variables.  That's why we don't
>>>> see the "-std=gnu11" there.
>
>>> Maybe the solution can be inspired by the line of thought Paul started
>>> in [1]. Namely:
>>> Define a *small* set of variables that influence the configure
>>> results. Currently these are CC, CFLAGS, LDFLAGS, but not CPPFLAGS.
>>> Then, can we define a set of variables that can be passed down from the
>>> top-level Makefile the subordinate Makefiles?
>>> These two sets of variables must be disjoint, and that is the problem
>>> here, because users would like to use CFLAGS to pass optimization and
>>> debugging flags down the build tree, after the configuration is complete.
>
> Sergio> I'm still inclined to go the "easy way" and do what I proposed above:
> Sergio> create a "FLAGS_TO_PASS_TO_GNULIB" which would omit the CC/CXX (and
> Sergio> possibly other) variables, even if it's just to unbreak the s390x build
> Sergio> (I still have a gnulib issue happening with mingw to investigate, sigh).
>
> Historically the GNU and/or configure and/or automake rule was that
> variables like CC, CXX, CFLAGS (etc) were for users.  So, following this
> rule, I think it's correct for gdb to pass these to sub-configures and
> sub-makes.

Hm, fair enough.  Actually, after experimenting with this yesterday, I
found that it's not enough to remove CC/CXX from the list of variables
passed down to gnulib.  I still haven't found a fix for this problem.

> I haven't looked at the gnulib code here, but it seems to me that if
> gnulib wants to find special compiler flags to build itself, then I
> think those should be stuck into some other-named variable, not CFLAGS.
> gdb does this itself as well, for example warning flags aren't added to
> CXXFLAGS but some other variable.  In automake the convention is to name
> these internal things AM_mumble, like AM_CFLAGS.  There's some
> discussion in (info "(automake) Flag Variables Ordering")

That's a good idea.  I'd like to know what the gnulib folks think of it.
I'll try to take a look at the gnulib code and see if I can come up with
a patch for this.

Also, Paul Eggert suggested something yesterday:

  Another possibility that may be simpler for GDB, is to change its
  configure.ac files to require C99 or later everywhere. At this point
  it's more trouble than it's worth to tweak source code or makefiles to
  cater to compilers operating in C89 mode. Just tell your C compiler to
  support C99-or-better everywhere, and your life will surely be
  simpler.

I don't know if just requiring C99 or later would be enough to solve
this problem, but it's something to consider for GDB, I think.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: hierarchical projects with configure scripts

[Tom Tromey]

>>>>> "Sergio" == Sergio Durigan Junior <sergiodj@redhat.com> writes:

>   Another possibility that may be simpler for GDB, is to change its
>   configure.ac files to require C99 or later everywhere. At this point
>   it's more trouble than it's worth to tweak source code or makefiles to
>   cater to compilers operating in C89 mode. Just tell your C compiler to
>   support C99-or-better everywhere, and your life will surely be
>   simpler.

Sergio> I don't know if just requiring C99 or later would be enough to solve
Sergio> this problem, but it's something to consider for GDB, I think.

I personally think it would be fine -- C99 is nearly 20 years old now,
surely we can afford to upgrade -- but I think this would have to be run
by the binutils list as well.

Also there's the question of how it would be implemented.  Like, if it
relied on modifying CFLAGS, then that would still interfere how this is
supposed to be a user-controlled variable.

Tom

Re: hierarchical projects with configure scripts

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>,  Bruno Haible <bruno@clisp.org>,  Paul Eggert <eggert@cs.ucla.edu>,  bug-gnulib@gnu.org,  gdb-patches@sourceware.org
> Date: Sat, 01 Sep 2018 00:05:26 -0600
> 
> Sergio> I don't know if just requiring C99 or later would be enough to solve
> Sergio> this problem, but it's something to consider for GDB, I think.
> 
> I personally think it would be fine -- C99 is nearly 20 years old now,

A nit: it should be okay to require C99 for the compiler, but we
should not require that for the library (including header files),
because MinGW, for example, uses a C99-compliant GCC, but the headers
and the function from the standard C library are those from Windows,
which are not 100% C99.

Re: hierarchical projects with configure scripts

[Paul Eggert]

Eli Zaretskii wrote:
> A nit: it should be okay to require C99 for the compiler, but we
> should not require that for the library (including header files),
> because MinGW, for example, uses a C99-compliant GCC, but the headers
> and the function from the standard C library are those from Windows,
> which are not 100% C99.

Gnulib doesn't assume 100% C99, only the common features that I think even 
Microsoft supports nowadays. (Even C11 is not 100% C99 compatible....)