[RFC][top-level] Add configure test-case

[RFC][top-level] Add configure test-case

[Tom de Vries]

Hi,

As reported in reopened PR18632, recent commit 228cf97dd3c ("Merge configure.ac
from gcc project") dropped commit 69961a84c9b ("Don't build
readline/libreadline.a, when --with-system-readline is supplied"), due to the
patch only being applied to binutils-gdb repo, and not to gcc repo.

I wondered if I could write a test-case that would regress because of this,
such that we at least don't silently regress.

I came up with a test-case named toplevel.exp, that can be run like this:
...
 $ cd src
 $ runtest toplevel.exp
 ...
 Running ./toplevel.exp ...

                 ===  Summary ===

 # of expected passes            1
...
and leaves files $src/testrun.{log,sum}.

Tested on x86_64-linux.

Any comments?

Thanks,
- Tom

[top-level] Add configure test-case

---
 toplevel.exp | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/toplevel.exp b/toplevel.exp
new file mode 100644
index 00000000000..db224b3f0b1
--- /dev/null
+++ b/toplevel.exp
@@ -0,0 +1,53 @@
+#   Copyright (C) 2022 Free Software Foundation, Inc.
+#
+# This file is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.
+
+# We're assuming this is run in the top-level source directory, like so:
+# $ cd src
+# $ runtest toplevel.exp
+set rootdir [pwd]
+set srcdir $rootdir
+
+# We're putting the build dir inside the source dir, that may need to be
+# changed at some point.
+set builddir $rootdir/build.tmp
+exec rm -Rf $builddir
+exec mkdir $builddir
+
+cd $builddir
+
+set test "configure --with-system-readline"
+set res [catch {exec $srcdir/configure --with-system-readline} output]
+
+set error_info_re \
+    [list \
+	 "This configuration is not supported in the following subdirectories:" \
+	 "(\[^\n\]+ )?readline( \[^\n\]+)?" \
+	 ""]
+set error_info_re [join $error_info_re "\n"]
+
+# Note: $res == 1 and $errorCode == "NONE" means that configure succeeded but
+# wrote something to stderr, which is available in $errorInfo.
+if { $res == 1
+     && $errorCode == "NONE"
+     && [regexp $error_info_re $errorInfo] == 1 } {
+    pass $test
+} else {
+    verbose -log "configure output: $output"
+    fail $test
+}
+
+# Cleanup.
+exec rm -Rf $builddir

Re: [RFC][top-level] Add configure test-case

[Nick Clifton]

Hi Tom,

> I came up with a test-case named toplevel.exp, that can be run like this:
> ...
>   $ cd src
>   $ runtest toplevel.exp

Could it also be run as part of "make check" ?

> and leaves files $src/testrun.{log,sum}.

I would prefer the results to appear as $src/testsuite/toplevel.{log,sum},
ie in a directory of their own.  I know that the ld and binutils testsuites
do not do this, but the gas one does, and so do the gcc testsuites.

I assume that you are also be suggesting this change on the gcc mailing list ?

Cheers
   Nick

Re: [RFC][top-level] Add configure test-case

[Alan Modra]

On Thu, Oct 20, 2022 at 05:10:28PM +0200, Tom de Vries via Binutils wrote:
> Hi,
> 
> As reported in reopened PR18632, recent commit 228cf97dd3c ("Merge configure.ac
> from gcc project") dropped commit 69961a84c9b ("Don't build
> readline/libreadline.a, when --with-system-readline is supplied"), due to the
> patch only being applied to binutils-gdb repo, and not to gcc repo.

Sorry about that.

> I wondered if I could write a test-case that would regress because of this,
> such that we at least don't silently regress.
> 
> I came up with a test-case named toplevel.exp, that can be run like this:

While I applaud anything that helps stop Alan making silly mistakes,
the real problem is either
a) that top-level binutils/gdb patches don't get applied to the gcc
   git repository in a timely manner, or
b) that we try to keep them in sync.

Maybe we should just give up on (b)?

-- 
Alan Modra
Australia Development Lab, IBM

Re: [RFC][top-level] Add configure test-case

[Joseph Myers]

On Mon, 7 Nov 2022, Alan Modra via Binutils wrote:

> a) that top-level binutils/gdb patches don't get applied to the gcc
>    git repository in a timely manner, or

If a toplevel patch is approved for either repository, I think you should 
treat it as approved for the other one without needing separate review.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [RFC][top-level] Add configure test-case

[Alan Modra]

On Mon, Nov 07, 2022 at 06:23:45PM +0000, Joseph Myers wrote:
> On Mon, 7 Nov 2022, Alan Modra via Binutils wrote:
> 
> > a) that top-level binutils/gdb patches don't get applied to the gcc
> >    git repository in a timely manner, or
> 
> If a toplevel patch is approved for either repository, I think you should 
> treat it as approved for the other one without needing separate review.

Thanks Joseph, that's how I see it too.  Of course with the
understanding that binutils-gdb can't be used as a back door way of
sneaking in a gcc-specific change.

Can I get agreement among the gcc build maintainers that such a
policy is acceptable?

-- 
Alan Modra
Australia Development Lab, IBM

Re: [RFC][top-level] Add configure test-case

[Alexandre Oliva]

On Nov  8, 2022, Alan Modra via Gdb-patches <gdb-patches@sourceware.org> wrote:

> On Mon, Nov 07, 2022 at 06:23:45PM +0000, Joseph Myers wrote:
>> On Mon, 7 Nov 2022, Alan Modra via Binutils wrote:
>> 
>> > a) that top-level binutils/gdb patches don't get applied to the gcc
>> >    git repository in a timely manner, or
>> 
>> If a toplevel patch is approved for either repository, I think you should 
>> treat it as approved for the other one without needing separate review.

> Thanks Joseph, that's how I see it too.  Of course with the
> understanding that binutils-gdb can't be used as a back door way of
> sneaking in a gcc-specific change.

> Can I get agreement among the gcc build maintainers that such a
> policy is acceptable?

FTR, II've long assumed that this cooperation in maintaining the
top-level build machinery worked both ways already.  Reducing divergence
is a plus IMHO.

-- 
Alexandre Oliva, happy hacker                https://FSFLA.org/blogs/lxo/
   Free Software Activist                       GNU Toolchain Engineer
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about <https://stallmansupport.org>