run one test

run one test

[DJ Delorie]

One of the small items that was mentioned at Cauldron was "how do to
re-run just one test?"

While it can be done with a suitable command line make invocation,
this is a lot easier...

I'll turn it into a real patch once we bikeshed the target name ;-)

diff --git a/Makefile b/Makefile
index 67ddd01bfe..c424d9ee02 100644
--- a/Makefile
+++ b/Makefile
@@ -499,3 +499,11 @@ FORCE:
 
 iconvdata/% localedata/% po/%: FORCE
 	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
+
+# Convenience target to rerun one test, from the top of the build tree
+# Example: make onetest t=wcsmbs/test-wcsnlen
+onetest :
+	@-rm -f $(objpfx)$t.out
+	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
+	@cat $(objpfx)$t.test-result
+	@cat $(objpfx)$t.out

Re: run one test

[Carlos O'Donell]

On 9/25/19 4:41 PM, DJ Delorie wrote:
> One of the small items that was mentioned at Cauldron was "how do to
> re-run just one test?"
> 
> While it can be done with a suitable command line make invocation,
> this is a lot easier...
> 
> I'll turn it into a real patch once we bikeshed the target name ;-)
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..c424d9ee02 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -499,3 +499,11 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onetest :
> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out
> 

Does this avoid rebuilding *anything* else?

One of the problems with the suggested fix on the wiki is that it can
put your tree into a bad state because of missing dependencies you
might rebuild only part of glibc and fail.

For example if $t.out depends on libpthread.so, then what happens
if you've touched a file under nptl/? Likewise for libdl.so?

I'm not saying we shouldn't do this, but that we should document the
limitations of this command.

And I suggest 'make onecheck' to match 'make xcheck' and 'make check' :-)

-- 
Cheers,
Carlos.

Re: run one test

[DJ Delorie]

"Carlos O'Donell" <carlos@redhat.com> writes:
> Does this avoid rebuilding *anything* else?

It uses the internal dependency tree to generate the *.out file, but
does not artificially limit what that tree might rebuild.

> One of the problems with the suggested fix on the wiki is that it can
> put your tree into a bad state because of missing dependencies you
> might rebuild only part of glibc and fail.
>
> For example if $t.out depends on libpthread.so, then what happens
> if you've touched a file under nptl/? Likewise for libdl.so?

It does not do the implied "make" that "make check" does, if that's what
you're asking.  It literally just reruns the one test - rebuilding the
test itself if needed, but not rebuilding libc.so because there's no
dependency on it.

So you'd need to run "make && make onetest t=..." if you want to rebuild
libc.so first.

> And I suggest 'make onecheck' to match 'make xcheck' and 'make check' :-)

Let the bikeshedding begin!  :-)

Re: run one test

[Szabolcs Nagy]

On 25/09/2019 21:41, DJ Delorie wrote:
> 
> One of the small items that was mentioned at Cauldron was "how do to
> re-run just one test?"
> 
> While it can be done with a suitable command line make invocation,
> this is a lot easier...

looks useful to me.

unfortunately there is currently no easy way to
discover such make targets. (maybe there could be a
make target that prints out usage information for
the most commonly used make targets)

it can be convenient to rerun a set of tests too, e.g.
by providing a list or pattern like 'math/test-double-*',
which can be implemented in a similar way.

same for benchmarks.

for now the proposed patch looks good enough though.

> 
> I'll turn it into a real patch once we bikeshed the target name ;-)
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..c424d9ee02 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -499,3 +499,11 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onetest :
> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out
> 

[v2, with help] Re: run one test

[DJ Delorie]

Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:
> unfortunately there is currently no easy way to
> discover such make targets. (maybe there could be a
> make target that prints out usage information for
> the most commonly used make targets)

I added a help option, with mini-help.  I do NOT volunteer to write a
comprehensive one-page help doc for our Makefile, but invite everyone to
add their favorite targets willy-nilly after the initial commit :-)

The obvious next question is "how does the user know to run 'make
help'?"  but I tried to give them a hint there too.

> it can be convenient to rerun a set of tests too, e.g.
> by providing a list or pattern like 'math/test-double-*',
> which can be implemented in a similar way.

I think we can do that with wildcards and file matching, but I don't
know off the top of my head how to wrap the onetest code into that kind
of expansion.  Is there a compelling case for needing more than one but
less than a subdir of tests rerun?

> same for benchmarks.

I've not run the benchmarks myself, but... yeah, probably.

> for now the proposed patch looks good enough though.

Same patch, with help system, attached...

diff --git a/Makefile b/Makefile
index 67ddd01bfe..0465609de0 100644
--- a/Makefile
+++ b/Makefile
@@ -26,8 +26,17 @@ include Makeconfig
 
 
 # This is the default target; it makes everything except the tests.
-.PHONY: all
-all: lib others
+.PHONY: all help minihelp
+all: minihelp lib others
+
+help:
+	@sed '0,/^help-starts-here$$/d' Makefile.help
+
+minihelp:
+	@echo
+	@echo type \"make help\" for help with common GLIBC makefile targets
+	@echo
+
 \f
 ifneq ($(AUTOCONF),no)
 
@@ -499,3 +508,11 @@ FORCE:
 
 iconvdata/% localedata/% po/%: FORCE
 	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
+
+# Convenience target to rerun one test, from the top of the build tree
+# Example: make onetest t=wcsmbs/test-wcsnlen
+onetest :
+	@-rm -f $(objpfx)$t.out
+	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
+	@cat $(objpfx)$t.test-result
+	@cat $(objpfx)$t.out
diff --git a/Makefile.help b/Makefile.help
new file mode 100644
index 0000000000..76b3418adb
--- /dev/null
+++ b/Makefile.help
@@ -0,0 +1,41 @@
+# Copyright (C) 2019 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <https://www.gnu.org/licenses/>.
+
+This is the file that gets printed when the user runs "make help",
+starting just after the "help-starts-here" line.
+
+help-starts-here
+
+all
+	The usual default; builds everything but doesn't run the
+	tests.
+tests
+	Runs the testsuite.  Also "make check" is a shortcut for this
+
+onetest
+	Runs one test.  Use like this:
+		make onetest t=wcsmbs/test-wcsnlen
+	Note that this will rebuild the test if needed, but will not
+	rebuild what "make all" would have rebuilt.
+
+--
+Other useful hints:
+
+$ rm $(objpfx)testroot.pristine/install.stamp
+	Forces the testroot to be reinstalled the next time you run
+	the testsuite (or just rm -rf testroot.pristine)
+

Re: [v2, with help] Re: run one test

[Carlos O'Donell]

On 9/30/19 3:37 PM, DJ Delorie wrote:
> Szabolcs Nagy <Szabolcs.Nagy@arm.com> writes:
>> unfortunately there is currently no easy way to
>> discover such make targets. (maybe there could be a
>> make target that prints out usage information for
>> the most commonly used make targets)
> 
> I added a help option, with mini-help.  I do NOT volunteer to write a
> comprehensive one-page help doc for our Makefile, but invite everyone to
> add their favorite targets willy-nilly after the initial commit :-)

That's fine. I can add some notes on the targets I added for locales which
are useful to distro maintainers.

See my comments.

> The obvious next question is "how does the user know to run 'make
> help'?"  but I tried to give them a hint there too.
> 
>> it can be convenient to rerun a set of tests too, e.g.
>> by providing a list or pattern like 'math/test-double-*',
>> which can be implemented in a similar way.
> 
> I think we can do that with wildcards and file matching, but I don't
> know off the top of my head how to wrap the onetest code into that kind
> of expansion.  Is there a compelling case for needing more than one but
> less than a subdir of tests rerun?
> 
>> same for benchmarks.
> 
> I've not run the benchmarks myself, but... yeah, probably.
> 
>> for now the proposed patch looks good enough though.
> 
> Same patch, with help system, attached...
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..0465609de0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -26,8 +26,17 @@ include Makeconfig
>  
>  
>  # This is the default target; it makes everything except the tests.
> -.PHONY: all
> -all: lib others
> +.PHONY: all help minihelp
> +all: minihelp lib others
> +
> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help
> +
> +minihelp:
> +	@echo
> +	@echo type \"make help\" for help with common GLIBC makefile targets

s/GLIBC/glibc/g

> +	@echo
> +
>  \f
>  ifneq ($(AUTOCONF),no)
>  
> @@ -499,3 +508,11 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onetest :
> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out

Bikeshed. Suggest "onecheck" alias given that 'make test' is the same
as 'make check.'

> diff --git a/Makefile.help b/Makefile.help
> new file mode 100644
> index 0000000000..76b3418adb
> --- /dev/null
> +++ b/Makefile.help
> @@ -0,0 +1,41 @@
> +# Copyright (C) 2019 Free Software Foundation, Inc.
> +# This file is part of the GNU C Library.
> +
> +# The GNU C Library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +
> +# The GNU C Library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with the GNU C Library; if not, see
> +# <https://www.gnu.org/licenses/>.

OK.

> +
> +This is the file that gets printed when the user runs "make help",
> +starting just after the "help-starts-here" line.
> +
> +help-starts-here
> +
> +all
> +	The usual default; builds everything but doesn't run the
> +	tests.
> +tests
> +	Runs the testsuite.  Also "make check" is a shortcut for this
> +
> +onetest

s/onetest/onecheck/g

Or add a onecheck alias.

> +	Runs one test.  Use like this:
> +		make onetest t=wcsmbs/test-wcsnlen
> +	Note that this will rebuild the test if needed, but will not
> +	rebuild what "make all" would have rebuilt.

OK.

> +
> +--
> +Other useful hints:
> +
> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)
> +
> 

OK.

-- 
Cheers,
Carlos.

Re: [v2, with help] Re: run one test

[DJ Delorie]

"Carlos O'Donell" <carlos@redhat.com> writes:
> Bikeshed. Suggest "onecheck" alias given that 'make test' is the same
> as 'make check.'

Can add.  I was thinking we already have "make nss/tests" and "make
tests", we call them "tests" and "testsuite", and "make check" is just a
convenience target:

  # Support the GNU standard name for this target.
  .PHONY: check
  check: tests

Or we could call it "make test" (note singular) but I suspect this would
be confusing, and "make check" (note singular!) vs "make checks".

Re: [v2, with help] Re: run one test

[Carlos O'Donell]

On 9/30/19 3:58 PM, DJ Delorie wrote:
> "Carlos O'Donell" <carlos@redhat.com> writes:
>> Bikeshed. Suggest "onecheck" alias given that 'make test' is the same
>> as 'make check.'
> 
> Can add.  I was thinking we already have "make nss/tests" and "make
> tests", we call them "tests" and "testsuite", and "make check" is just a
> convenience target:
> 
>   # Support the GNU standard name for this target.
>   .PHONY: check
>   check: tests
> 
> Or we could call it "make test" (note singular) but I suspect this would
> be confusing, and "make check" (note singular!) vs "make checks".

In summary:

make check = make tests
make onecheck = make test

Or is the tests vs test too confusing?

-- 
Cheers,
Carlos.

Re: [v3, with help] Re: run one test

[DJ Delorie]

From a6f669fc4091899c8ebc50159c9e4edc5cc4bdeb Mon Sep 17 00:00:00 2001
From: DJ Delorie <dj@redhat.com>
Date: Mon, 30 Sep 2019 16:04:52 -0400
Subject: Add onetest convenience target and makefile help text

Adds "make onetest" (or "make onecheck") for re-running just
one test.  Also adds "make help" for help with our Makefile
targets, and adds a mini-help when you just run "make".

diff --git a/Makefile b/Makefile
index 67ddd01bfe..f84450e60d 100644
--- a/Makefile
+++ b/Makefile
@@ -26,8 +26,17 @@ include Makeconfig
 
 
 # This is the default target; it makes everything except the tests.
-.PHONY: all
-all: lib others
+.PHONY: all help minihelp
+all: minihelp lib others
+
+help:
+	@sed '0,/^help-starts-here$$/d' Makefile.help
+
+minihelp:
+	@echo
+	@echo type \"make help\" for help with common glibc makefile targets
+	@echo
+
 \f
 ifneq ($(AUTOCONF),no)
 
@@ -499,3 +508,12 @@ FORCE:
 
 iconvdata/% localedata/% po/%: FORCE
 	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
+
+# Convenience target to rerun one test, from the top of the build tree
+# Example: make onetest t=wcsmbs/test-wcsnlen
+onecheck : onetest
+onetest :
+	@-rm -f $(objpfx)$t.out
+	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
+	@cat $(objpfx)$t.test-result
+	@cat $(objpfx)$t.out
diff --git a/Makefile.help b/Makefile.help
new file mode 100644
index 0000000000..445085be03
--- /dev/null
+++ b/Makefile.help
@@ -0,0 +1,41 @@
+# Copyright (C) 2019 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <https://www.gnu.org/licenses/>.
+
+This is the file that gets printed when the user runs "make help",
+starting just after the "help-starts-here" line.
+
+help-starts-here
+
+all
+	The usual default; builds everything but doesn't run the
+	tests.
+tests
+	Runs the testsuite.  Also "make check" is a shortcut for this
+
+onetest  (or onecheck)
+	Runs one test.  Use like this:
+		make onetest t=wcsmbs/test-wcsnlen
+	Note that this will rebuild the test if needed, but will not
+	rebuild what "make all" would have rebuilt.
+
+--
+Other useful hints:
+
+$ rm $(objpfx)testroot.pristine/install.stamp
+	Forces the testroot to be reinstalled the next time you run
+	the testsuite (or just rm -rf testroot.pristine)
+

Re: [v3, with help] Re: run one test

[Carlos O'Donell]

On 9/30/19 4:06 PM, DJ Delorie wrote:
> 
> From a6f669fc4091899c8ebc50159c9e4edc5cc4bdeb Mon Sep 17 00:00:00 2001
> From: DJ Delorie <dj@redhat.com>
> Date: Mon, 30 Sep 2019 16:04:52 -0400
> Subject: Add onetest convenience target and makefile help text
> 
> Adds "make onetest" (or "make onecheck") for re-running just
> one test.  Also adds "make help" for help with our Makefile
> targets, and adds a mini-help when you just run "make".
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..f84450e60d 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -26,8 +26,17 @@ include Makeconfig
>  
>  
>  # This is the default target; it makes everything except the tests.
> -.PHONY: all
> -all: lib others
> +.PHONY: all help minihelp
> +all: minihelp lib others
> +
> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help
> +
> +minihelp:
> +	@echo
> +	@echo type \"make help\" for help with common glibc makefile targets
> +	@echo
> +
>  \f
>  ifneq ($(AUTOCONF),no)
>  
> @@ -499,3 +508,12 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onecheck : onetest
> +onetest :

I take this back, this looks dumb when I say it out loud. There isn't a gnu
standard here, so why not just stick to "test"?

e.g. make test t=foo/bar?

In summary:

make check == make tests == Run all standard tests.
make test == Run one test.


> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out
> diff --git a/Makefile.help b/Makefile.help
> new file mode 100644
> index 0000000000..445085be03
> --- /dev/null
> +++ b/Makefile.help
> @@ -0,0 +1,41 @@
> +# Copyright (C) 2019 Free Software Foundation, Inc.
> +# This file is part of the GNU C Library.
> +
> +# The GNU C Library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +
> +# The GNU C Library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with the GNU C Library; if not, see
> +# <https://www.gnu.org/licenses/>.
> +
> +This is the file that gets printed when the user runs "make help",
> +starting just after the "help-starts-here" line.
> +
> +help-starts-here
> +
> +all
> +	The usual default; builds everything but doesn't run the
> +	tests.
> +tests

s/tests/check (or tests)/g

> +	Runs the testsuite.  Also "make check" is a shortcut for this

s/Runs the testsuite. .../Runs the standard set of tests./g.

Should list the preferred GNU target first.

> +
> +onetest (or onecheck)
Adjust according to above.

> +	Runs one test.  Use like this:
> +		make onetest t=wcsmbs/test-wcsnlen
> +	Note that this will rebuild the test if needed, but will not
> +	rebuild what "make all" would have rebuilt.
> +
> +--
> +Other useful hints:
> +
> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)
> +
> 


-- 
Cheers,
Carlos.

Re: [v4, with help] Re: run one test

[DJ Delorie]

Latest iteration...

From b8c545e96f32420a1ebab0056e0a4b5dfe19cc1b Mon Sep 17 00:00:00 2001
From: DJ Delorie <dj@redhat.com>
Date: Mon, 30 Sep 2019 16:04:52 -0400
Subject: Add run-one-test convenience target and makefile help text

Adds "make test" for re-running just one test.  Also adds
"make help" for help with our Makefile targets, and adds a
mini-help when you just run "make".

diff --git a/Makefile b/Makefile
index 67ddd01bfe..1f3e088408 100644
--- a/Makefile
+++ b/Makefile
@@ -26,8 +26,17 @@ include Makeconfig
 
 
 # This is the default target; it makes everything except the tests.
-.PHONY: all
-all: lib others
+.PHONY: all help minihelp
+all: minihelp lib others
+
+help:
+	@sed '0,/^help-starts-here$$/d' Makefile.help
+
+minihelp:
+	@echo
+	@echo type \"make help\" for help with common glibc makefile targets
+	@echo
+
 \f
 ifneq ($(AUTOCONF),no)
 
@@ -499,3 +508,12 @@ FORCE:
 
 iconvdata/% localedata/% po/%: FORCE
 	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
+
+# Convenience target to rerun one test, from the top of the build tree
+# Example: make test t=wcsmbs/test-wcsnlen
+.PHONY: test
+test :
+	@-rm -f $(objpfx)$t.out
+	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
+	@cat $(objpfx)$t.test-result
+	@cat $(objpfx)$t.out
diff --git a/Makefile.help b/Makefile.help
new file mode 100644
index 0000000000..49815e9b9e
--- /dev/null
+++ b/Makefile.help
@@ -0,0 +1,42 @@
+# Copyright (C) 2019 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <https://www.gnu.org/licenses/>.
+
+This is the file that gets printed when the user runs "make help",
+starting just after the "help-starts-here" line.
+
+help-starts-here
+
+all
+	The usual default; builds everything but doesn't run the
+	tests.
+
+check (or tests)
+	Runs the standard set of tests.
+
+test
+	Runs one test.  Use like this:
+		make test t=wcsmbs/test-wcsnlen
+	Note that this will rebuild the test if needed, but will not
+	rebuild what "make all" would have rebuilt.
+
+--
+Other useful hints:
+
+$ rm $(objpfx)testroot.pristine/install.stamp
+	Forces the testroot to be reinstalled the next time you run
+	the testsuite (or just rm -rf testroot.pristine)
+

Re: [v4, with help] Re: run one test

[Carlos O'Donell]

On 10/2/19 2:44 PM, DJ Delorie wrote:
> 
> Latest iteration...
> 

Reviewing other feedback. Looks good.

Szabolcs said it was good.

Florian and Andreas' comments are resolved or implemented in other ways.

My opinion is that we can play with this a bit and see what works.

This is my OK for master.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

> From b8c545e96f32420a1ebab0056e0a4b5dfe19cc1b Mon Sep 17 00:00:00 2001
> From: DJ Delorie <dj@redhat.com>
> Date: Mon, 30 Sep 2019 16:04:52 -0400
> Subject: Add run-one-test convenience target and makefile help text
> 
> Adds "make test" for re-running just one test.  Also adds
> "make help" for help with our Makefile targets, and adds a
> mini-help when you just run "make".
> 
> diff --git a/Makefile b/Makefile
> index 67ddd01bfe..1f3e088408 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -26,8 +26,17 @@ include Makeconfig
>  
>  
>  # This is the default target; it makes everything except the tests.
> -.PHONY: all
> -all: lib others
> +.PHONY: all help minihelp
> +all: minihelp lib others
> +
> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help

OK. Resolved Florian's comment about $ojbdir.

> +
> +minihelp:
> +	@echo
> +	@echo type \"make help\" for help with common glibc makefile targets
> +	@echo
> +

OK.

>  \f
>  ifneq ($(AUTOCONF),no)
>  
> @@ -499,3 +508,12 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make test t=wcsmbs/test-wcsnlen
> +.PHONY: test
> +test :
> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out

OK, resolved differently from Andreas' suggestion, using t=TESTNAME.

> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out

OK.

> diff --git a/Makefile.help b/Makefile.help
> new file mode 100644
> index 0000000000..49815e9b9e
> --- /dev/null
> +++ b/Makefile.help
> @@ -0,0 +1,42 @@
> +# Copyright (C) 2019 Free Software Foundation, Inc.
> +# This file is part of the GNU C Library.
> +
> +# The GNU C Library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +
> +# The GNU C Library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with the GNU C Library; if not, see
> +# <https://www.gnu.org/licenses/>.
> +
> +This is the file that gets printed when the user runs "make help",
> +starting just after the "help-starts-here" line.
> +
> +help-starts-here
> +
> +all
> +	The usual default; builds everything but doesn't run the
> +	tests.
> +
> +check (or tests)
> +	Runs the standard set of tests.
> +
> +test
> +	Runs one test.  Use like this:
> +		make test t=wcsmbs/test-wcsnlen
> +	Note that this will rebuild the test if needed, but will not
> +	rebuild what "make all" would have rebuilt.
> +
> +--
> +Other useful hints:
> +
> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)

OK.

> +
> 


-- 
Cheers,
Carlos.

Re: [v4, with help] Re: run one test

[DJ Delorie]

"Carlos O'Donell" <carlos@redhat.com> writes:
> This is my OK for master.

Thanks.  Pushed!

Re: [v2, with help] Re: run one test

[Andreas Schwab]

On Sep 30 2019, DJ Delorie <dj@redhat.com> wrote:

> @@ -499,3 +508,11 @@ FORCE:
>  
>  iconvdata/% localedata/% po/%: FORCE
>  	$(MAKE) $(PARALLELMFLAGS) -C $(@D) $(@F)
> +
> +# Convenience target to rerun one test, from the top of the build tree
> +# Example: make onetest t=wcsmbs/test-wcsnlen
> +onetest :
.PHONY: onetest

> +	@-rm -f $(objpfx)$t.out
> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
> +	@cat $(objpfx)$t.test-result
> +	@cat $(objpfx)$t.out

Or make the test name part of the target, so that you can use $*.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [v2, with help] Re: run one test

[DJ Delorie]

Andreas Schwab <schwab@linux-m68k.org> writes:
>> +onetest :
> .PHONY: onetest

Fixed.

>> +	@-rm -f $(objpfx)$t.out
>> +	$(MAKE) subdir=$(dir $t) -C $(dir $t) ..=../ $(objpfx)$t.out
>> +	@cat $(objpfx)$t.test-result
>> +	@cat $(objpfx)$t.out
>
> Or make the test name part of the target, so that you can use $*.

How do you mean?  Like "make onetest.wcsmbs/test-wcsnlen" ?

I tried just "make wcsmbs/test-wcsnlen" but that got complicated and
messy before it got working.

Re: [v2, with help] Re: run one test

[Andreas Schwab]

On Sep 30 2019, DJ Delorie <dj@redhat.com> wrote:

> How do you mean?  Like "make onetest.wcsmbs/test-wcsnlen" ?

Yes.  Or retest-... or rerun-....

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: [v2, with help] Re: run one test

[Florian Weimer]

* DJ Delorie:

> +help:
> +	@sed '0,/^help-starts-here$$/d' Makefile.help

That doesn't look like something that would work in a build directory,
where Makefile.help will not be present in the current directory.

> +$ rm $(objpfx)testroot.pristine/install.stamp
> +	Forces the testroot to be reinstalled the next time you run
> +	the testsuite (or just rm -rf testroot.pristine)

It may make sense to put the actual path there, or otherwise avoid the
invalid shell syntax.

Thanks,
Florian

Re: [v2, with help] Re: run one test

[DJ Delorie]

Florian Weimer <fweimer@redhat.com> writes:
>> +help:
>> +	@sed '0,/^help-starts-here$$/d' Makefile.help
>
> That doesn't look like something that would work in a build directory,
> where Makefile.help will not be present in the current directory.

It works just fine because the build Makefile just cd's to the source
directory and re-runs the target.

all .DEFAULT:
        $(MAKE) -r PARALLELMFLAGS="$(PARALLELMFLAGS)" -C $(srcdir) objdir=`pwd` $@

>> +$ rm $(objpfx)testroot.pristine/install.stamp
>> +	Forces the testroot to be reinstalled the next time you run
>> +	the testsuite (or just rm -rf testroot.pristine)
>
> It may make sense to put the actual path there, or otherwise avoid the
> invalid shell syntax.

I don't want to start filtering in real paths; I'll edit it.