* [PATCH] test-in-container: Install locales into the test container
@ 2019-07-09 17:32 Florian Weimer
2019-07-09 18:07 ` DJ Delorie
0 siblings, 1 reply; 30+ messages in thread
From: Florian Weimer @ 2019-07-09 17:32 UTC (permalink / raw)
To: libc-alpha
The testing time impact of this change is significant: It increases
total test time from ~13 to ~15 minutes for me. However, I do not
see any other way to run locale-dependent tests in a container.
2019-07-09 Florian Weimer <fweimer@redhat.com>
* Makefile (testroot.pristine/install.stamp): Install locales.
diff --git a/Makefile b/Makefile
index dc5de7aa6b..23c4326980 100644
--- a/Makefile
+++ b/Makefile
@@ -402,6 +402,7 @@ ifeq ($(run-built-tests),yes)
done
endif
$(MAKE) install DESTDIR=$(objpfx)testroot.pristine
+ $(MAKE) localedata/install-locales DESTDIR=$(objpfx)testroot.pristine
touch $(objpfx)testroot.pristine/install.stamp
tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 17:32 [PATCH] test-in-container: Install locales into the test container Florian Weimer
@ 2019-07-09 18:07 ` DJ Delorie
2019-07-09 20:52 ` Florian Weimer
0 siblings, 1 reply; 30+ messages in thread
From: DJ Delorie @ 2019-07-09 18:07 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
Florian Weimer <fweimer@redhat.com> writes:
> The testing time impact of this change is significant: It increases
> total test time from ~13 to ~15 minutes for me. However, I do not
> see any other way to run locale-dependent tests in a container.
This is also the only way to auto-test "make install-locales" too :-)
> $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
> + $(MAKE) localedata/install-locales DESTDIR=$(objpfx)testroot.pristine
> touch $(objpfx)testroot.pristine/install.stamp
This matches what's documented in INSTALL so LGTM.
Reviewed-by: DJ Delorie <dj@redhat.com>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 18:07 ` DJ Delorie
@ 2019-07-09 20:52 ` Florian Weimer
2019-07-09 20:59 ` DJ Delorie
2019-07-09 21:08 ` [PATCH] test-in-container: Install locales into the test container Carlos O'Donell
0 siblings, 2 replies; 30+ messages in thread
From: Florian Weimer @ 2019-07-09 20:52 UTC (permalink / raw)
To: DJ Delorie; +Cc: libc-alpha
* DJ Delorie:
> Florian Weimer <fweimer@redhat.com> writes:
>> The testing time impact of this change is significant: It increases
>> total test time from ~13 to ~15 minutes for me. However, I do not
>> see any other way to run locale-dependent tests in a container.
>
> This is also the only way to auto-test "make install-locales" too :-)
>
>> $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
>> + $(MAKE) localedata/install-locales DESTDIR=$(objpfx)testroot.pristine
>> touch $(objpfx)testroot.pristine/install.stamp
>
> This matches what's documented in INSTALL so LGTM.
>
> Reviewed-by: DJ Delorie <dj@redhat.com>
The problem is that the delay also applies to partial runs of the test
suite:
$ time make -j8 subdirs=resolv check
goes from:
real 0m8.792s
user 0m15.486s
sys 0m6.017s
to:
real 2m5.588s
user 13m20.667s
sys 0m44.631s
Compare this to the time of a clean glibc build on this machine:
real 1m42.489s
user 8m20.556s
sys 2m6.281s
This is a bit embarrassing because the pristine container constructed in
this way is not even correct (bug 24794).
I think applying this patch as-is would make glibc development
substantially more difficult for me.
Thanks,
Florian
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 20:52 ` Florian Weimer
@ 2019-07-09 20:59 ` DJ Delorie
2019-07-09 21:02 ` Florian Weimer
2019-07-09 21:08 ` [PATCH] test-in-container: Install locales into the test container Carlos O'Donell
1 sibling, 1 reply; 30+ messages in thread
From: DJ Delorie @ 2019-07-09 20:59 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
Florian Weimer <fweimer@redhat.com> writes:
> The problem is that the delay also applies to partial runs of the test
> suite:
The first time, or each other time?
> $ time make -j8 subdirs=resolv check
This does (should :) create the pristine testroot, but only once.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 20:59 ` DJ Delorie
@ 2019-07-09 21:02 ` Florian Weimer
2019-07-09 21:11 ` Partial test suite run builds corrupt test-in-container testroot Carlos O'Donell
0 siblings, 1 reply; 30+ messages in thread
From: Florian Weimer @ 2019-07-09 21:02 UTC (permalink / raw)
To: DJ Delorie; +Cc: libc-alpha
* DJ Delorie:
> Florian Weimer <fweimer@redhat.com> writes:
>> The problem is that the delay also applies to partial runs of the test
>> suite:
>
> The first time, or each other time?
The first time. It's still bad in some cases (depending on what I'm
working).
>> $ time make -j8 subdirs=resolv check
>
> This does (should :) create the pristine testroot, but only once.
Yes, and its contents stays incomplete.
Thanks,
Florian
^ permalink raw reply [flat|nested] 30+ messages in thread
* Partial test suite run builds corrupt test-in-container testroot
2019-07-09 21:02 ` Florian Weimer
@ 2019-07-09 21:11 ` Carlos O'Donell
2019-07-10 0:36 ` DJ Delorie
0 siblings, 1 reply; 30+ messages in thread
From: Carlos O'Donell @ 2019-07-09 21:11 UTC (permalink / raw)
To: Florian Weimer, DJ Delorie; +Cc: libc-alpha
On 7/9/19 5:02 PM, Florian Weimer wrote:
> * DJ Delorie:
>>> $ time make -j8 subdirs=resolv check
>>
>> This does (should :) create the pristine testroot, but only once.
>
> Yes, and its contents stays incomplete.
I'm changing the subject because this is a distinct htread.
Thanks for filling this:
https://sourceware.org/bugzilla/show_bug.cgi?id=24794
We should work to solve this, since it's a use case that many
developers expect to work.
--
Cheers,
Carlos.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Partial test suite run builds corrupt test-in-container testroot
2019-07-09 21:11 ` Partial test suite run builds corrupt test-in-container testroot Carlos O'Donell
@ 2019-07-10 0:36 ` DJ Delorie
2019-07-10 5:50 ` Florian Weimer
0 siblings, 1 reply; 30+ messages in thread
From: DJ Delorie @ 2019-07-10 0:36 UTC (permalink / raw)
To: fweimer, libc-alpha
I did a "make" followed by "make libio/tests" and compared with a "make;
make check" and they produced identical testroots (including iconvconfig).
I tried a "./configure; make libio/tests" and it failed pretty quickly
(shell-container not found) since support/ hadn't been built.
Please provide more detailed reproduction instructions, as I don't see a
"corrupt testroot" yet.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Partial test suite run builds corrupt test-in-container testroot
2019-07-10 0:36 ` DJ Delorie
@ 2019-07-10 5:50 ` Florian Weimer
2019-07-10 16:34 ` DJ Delorie
0 siblings, 1 reply; 30+ messages in thread
From: Florian Weimer @ 2019-07-10 5:50 UTC (permalink / raw)
To: DJ Delorie; +Cc: libc-alpha
* DJ Delorie:
> I did a "make" followed by "make libio/tests" and compared with a "make;
> make check" and they produced identical testroots (including iconvconfig).
>
> I tried a "./configure; make libio/tests" and it failed pretty quickly
> (shell-container not found) since support/ hadn't been built.
>
> Please provide more detailed reproduction instructions, as I don't see a
> "corrupt testroot" yet.
You need to build the tree first before running the test suite subset.
For example:
make -j8
make -j8 check subdirs=libio
Thanks,
Florian
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Partial test suite run builds corrupt test-in-container testroot
2019-07-10 5:50 ` Florian Weimer
@ 2019-07-10 16:34 ` DJ Delorie
2019-07-10 16:35 ` Florian Weimer
0 siblings, 1 reply; 30+ messages in thread
From: DJ Delorie @ 2019-07-10 16:34 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
Florian Weimer <fweimer@redhat.com> writes:
> You need to build the tree first before running the test suite subset.
> For example:
>
> make -j8
> make -j8 check subdirs=libio
Yup, as I said:
>> I did a "make" followed by "make libio/tests" and compared with a "make;
>> make check" and they produced identical testroots (including iconvconfig).
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Partial test suite run builds corrupt test-in-container testroot
2019-07-10 16:34 ` DJ Delorie
@ 2019-07-10 16:35 ` Florian Weimer
2019-07-10 16:38 ` DJ Delorie
0 siblings, 1 reply; 30+ messages in thread
From: Florian Weimer @ 2019-07-10 16:35 UTC (permalink / raw)
To: DJ Delorie; +Cc: libc-alpha
* DJ Delorie:
> Florian Weimer <fweimer@redhat.com> writes:
>> You need to build the tree first before running the test suite subset.
>> For example:
>>
>> make -j8
>> make -j8 check subdirs=libio
>
> Yup, as I said:
>
>>> I did a "make" followed by "make libio/tests" and compared with a "make;
>>> make check" and they produced identical testroots (including iconvconfig).
Huh. What's your make version?
Thanks,
Florian
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Partial test suite run builds corrupt test-in-container testroot
2019-07-10 16:35 ` Florian Weimer
@ 2019-07-10 16:38 ` DJ Delorie
2019-07-19 11:52 ` Florian Weimer
0 siblings, 1 reply; 30+ messages in thread
From: DJ Delorie @ 2019-07-10 16:38 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
Florian Weimer <fweimer@redhat.com> writes:
> Huh. What's your make version?
4.2.1 - Fedora 30
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Partial test suite run builds corrupt test-in-container testroot
2019-07-10 16:38 ` DJ Delorie
@ 2019-07-19 11:52 ` Florian Weimer
2019-07-22 20:45 ` [PATCH] test-container: Install with $(all-subdirs) [BZ #24794] Tulio Magno Quites Machado Filho
0 siblings, 1 reply; 30+ messages in thread
From: Florian Weimer @ 2019-07-19 11:52 UTC (permalink / raw)
To: DJ Delorie; +Cc: libc-alpha
* DJ Delorie:
> Florian Weimer <fweimer@redhat.com> writes:
>> Huh. What's your make version?
>
> 4.2.1 - Fedora 30
Here's what I did. I got a box from Beaker with Fedora 30.
# dnf install screen
# dnf download --source glibc
# dnf builddep glibc-*.src.rpm
# screen
# dnf install git
# git clone --depth 1 git://sourceware.org/git/glibc.git git
# mkdir build
# cd build
# ../git/configure --prefix=/usr
# time nohup make -j24
# time nohup make -j24 subdirs=libio check
The output ends with:
FAIL: libio/tst-wfile-ascii
Summary of test results:
1 FAIL
87 PASS
make[1]: *** [Makefile:411: tests] Error 1
make[1]: Leaving directory '/root/git'
make: *** [Makefile:9: check] Error 2
The test log contains:
tst-wfile-ascii.c:37: numeric comparison failure
left: 256 (0x100); from: system (iconvconfig)
right: 0 (0x0); from: 0
error: 1 test failures
Locale settings (inherited via SSH):
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_US.UTF-8
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
Root vs non-root doesn't make a difference.
Thanks,
Florian
^ permalink raw reply [flat|nested] 30+ messages in thread* [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-19 11:52 ` Florian Weimer
@ 2019-07-22 20:45 ` Tulio Magno Quites Machado Filho
2019-07-22 20:56 ` DJ Delorie
0 siblings, 1 reply; 30+ messages in thread
From: Tulio Magno Quites Machado Filho @ 2019-07-22 20:45 UTC (permalink / raw)
To: Florian Weimer, DJ Delorie, libc-alpha
Whenever a sub-make is created, it inherits the variable subdirs from its
parent. This is also true when make check is called with a restricted
list of subdirs. In this scenario, make install is executed "partially"
and testroot.pristine ends up with an incomplete installation.
2019-07-22 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
[BZ #24794]
* Makefile (testroot.pristine/install.stamp): Pass
subdirs='$(all-subdirs)' to make install.
---
Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index a4ed747cef..9fbf705200 100644
--- a/Makefile
+++ b/Makefile
@@ -401,7 +401,8 @@ ifeq ($(run-built-tests),yes)
$(test-wrapper) cp $$dso $(objpfx)testroot.pristine$$dso ;\
done
endif
- $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
+ $(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
+ subdirs='$(all-subdirs)'
touch $(objpfx)testroot.pristine/install.stamp
tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
--
2.14.5
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-22 20:45 ` [PATCH] test-container: Install with $(all-subdirs) [BZ #24794] Tulio Magno Quites Machado Filho
@ 2019-07-22 20:56 ` DJ Delorie
2019-07-23 16:29 ` Tulio Magno Quites Machado Filho
0 siblings, 1 reply; 30+ messages in thread
From: DJ Delorie @ 2019-07-22 20:56 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho; +Cc: fweimer, libc-alpha
Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com> writes:
> Whenever a sub-make is created, it inherits the variable subdirs from its
> parent. This is also true when make check is called with a restricted
> list of subdirs. In this scenario, make install is executed "partially"
> and testroot.pristine ends up with an incomplete installation.
"obvious in retrospect" Sigh.
> 2019-07-22 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
>
> [BZ #24794]
> * Makefile (testroot.pristine/install.stamp): Pass
> subdirs='$(all-subdirs)' to make install.
LGTM. Thanks!
Reviewed-by: DJ Delorie <dj@redhat.com>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-22 20:56 ` DJ Delorie
@ 2019-07-23 16:29 ` Tulio Magno Quites Machado Filho
2019-07-24 14:48 ` Stefan Liebler
0 siblings, 1 reply; 30+ messages in thread
From: Tulio Magno Quites Machado Filho @ 2019-07-23 16:29 UTC (permalink / raw)
To: DJ Delorie, fweimer, libc-alpha
DJ Delorie <dj@redhat.com> writes:
> Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com> writes:
>> Whenever a sub-make is created, it inherits the variable subdirs from its
>> parent. This is also true when make check is called with a restricted
>> list of subdirs. In this scenario, make install is executed "partially"
>> and testroot.pristine ends up with an incomplete installation.
>
> "obvious in retrospect" Sigh.
>
>> 2019-07-22 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
>>
>> [BZ #24794]
>> * Makefile (testroot.pristine/install.stamp): Pass
>> subdirs='$(all-subdirs)' to make install.
>
> LGTM. Thanks!
>
> Reviewed-by: DJ Delorie <dj@redhat.com>
Pushed as 35e038c1d2ccb.
Thanks!
--
Tulio Magno
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-23 16:29 ` Tulio Magno Quites Machado Filho
@ 2019-07-24 14:48 ` Stefan Liebler
2019-07-24 15:32 ` Tulio Magno Quites Machado Filho
0 siblings, 1 reply; 30+ messages in thread
From: Stefan Liebler @ 2019-07-24 14:48 UTC (permalink / raw)
To: libc-alpha
Hi,
starting with this commit, I see the following fail:
cat nss/tst-nss-files-hosts-long.out
error: tst-nss-files-hosts-long.c:35: ahostsv4 failed
error: 1 test failures
Does anybody else also see this fail?
While running "make check" I've recognized the output of
support/shell-container.c:
sh: execing getent failed: No such file or directory File
sh: execing /usr/bin/makedb failed: No such file or directory File
And indeed getent does not exist in <build-dir>/testroot.root or
<build-dir>/testroot.pristine. Without this commit, it was available.
In older logs, I've found these commands:
/usr/bin/install -c build/nss/getent
build/testroot.pristine/usr/bin/getent.new
/usr/bin/install -c build/nss/makedb
build/testroot.pristine/usr/bin/makedb.new
mv -f build/testroot.pristine/usr/bin/getent.new
build/testroot.pristine/usr/bin/getent
mv -f build/testroot.pristine/usr/bin/makedb.new
build/testroot.pristine/usr/bin/makedb
Can anybody help?
Bye
Stefan
On 7/23/19 6:29 PM, Tulio Magno Quites Machado Filho wrote:
> DJ Delorie <dj@redhat.com> writes:
>
>> Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com> writes:
>>> Whenever a sub-make is created, it inherits the variable subdirs from its
>>> parent. This is also true when make check is called with a restricted
>>> list of subdirs. In this scenario, make install is executed "partially"
>>> and testroot.pristine ends up with an incomplete installation.
>>
>> "obvious in retrospect" Sigh.
>>
>>> 2019-07-22 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
>>>
>>> [BZ #24794]
>>> * Makefile (testroot.pristine/install.stamp): Pass
>>> subdirs='$(all-subdirs)' to make install.
>>
>> LGTM. Thanks!
>>
>> Reviewed-by: DJ Delorie <dj@redhat.com>
>
> Pushed as 35e038c1d2ccb.
>
> Thanks!
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-24 14:48 ` Stefan Liebler
@ 2019-07-24 15:32 ` Tulio Magno Quites Machado Filho
2019-07-24 20:40 ` Carlos O'Donell
` (2 more replies)
0 siblings, 3 replies; 30+ messages in thread
From: Tulio Magno Quites Machado Filho @ 2019-07-24 15:32 UTC (permalink / raw)
To: Stefan Liebler, libc-alpha
Stefan Liebler <stli@linux.ibm.com> writes:
> starting with this commit, I see the following fail:
> cat nss/tst-nss-files-hosts-long.out
> error: tst-nss-files-hosts-long.c:35: ahostsv4 failed
> error: 1 test failures
>
> Does anybody else also see this fail?
I reproduced it here.
I'm sorry, I thought it was a bug hidden by the bug fixed in my previous
patch.
> While running "make check" I've recognized the output of
> support/shell-container.c:
> sh: execing getent failed: No such file or directory File
> sh: execing /usr/bin/makedb failed: No such file or directory File
>
> And indeed getent does not exist in <build-dir>/testroot.root or
> <build-dir>/testroot.pristine. Without this commit, it was available.
>
> In older logs, I've found these commands:
> /usr/bin/install -c build/nss/getent
> build/testroot.pristine/usr/bin/getent.new
> /usr/bin/install -c build/nss/makedb
> build/testroot.pristine/usr/bin/makedb.new
> mv -f build/testroot.pristine/usr/bin/getent.new
> build/testroot.pristine/usr/bin/getent
> mv -f build/testroot.pristine/usr/bin/makedb.new
> build/testroot.pristine/usr/bin/makedb
>
> Can anybody help?
I'll take a look and see what's missing.
--
Tulio Magno
^ permalink raw reply [flat|nested] 30+ messages in thread* Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-24 15:32 ` Tulio Magno Quites Machado Filho
@ 2019-07-24 20:40 ` Carlos O'Donell
2019-07-24 23:50 ` DJ Delorie
2019-07-25 0:03 ` [PATCH] test-container: Install with $(sorted-subdirs) " Tulio Magno Quites Machado Filho
2 siblings, 0 replies; 30+ messages in thread
From: Carlos O'Donell @ 2019-07-24 20:40 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho, Stefan Liebler, libc-alpha
On 7/24/19 11:32 AM, Tulio Magno Quites Machado Filho wrote:
> Stefan Liebler <stli@linux.ibm.com> writes:
>
>> starting with this commit, I see the following fail:
>> cat nss/tst-nss-files-hosts-long.out
>> error: tst-nss-files-hosts-long.c:35: ahostsv4 failed
>> error: 1 test failures
>>
>> Does anybody else also see this fail?
>
> I reproduced it here.
> I'm sorry, I thought it was a bug hidden by the bug fixed in my previous
> patch.
>
>> While running "make check" I've recognized the output of
>> support/shell-container.c:
>> sh: execing getent failed: No such file or directory File
>> sh: execing /usr/bin/makedb failed: No such file or directory File
>>
>> And indeed getent does not exist in <build-dir>/testroot.root or
>> <build-dir>/testroot.pristine. Without this commit, it was available.
>>
>> In older logs, I've found these commands:
>> /usr/bin/install -c build/nss/getent
>> build/testroot.pristine/usr/bin/getent.new
>> /usr/bin/install -c build/nss/makedb
>> build/testroot.pristine/usr/bin/makedb.new
>> mv -f build/testroot.pristine/usr/bin/getent.new
>> build/testroot.pristine/usr/bin/getent
>> mv -f build/testroot.pristine/usr/bin/makedb.new
>> build/testroot.pristine/usr/bin/makedb
>>
>> Can anybody help?
>
> I'll take a look and see what's missing.
Regression tests catching regressions! ;-)
--
Cheers,
Carlos.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]
2019-07-24 15:32 ` Tulio Magno Quites Machado Filho
2019-07-24 20:40 ` Carlos O'Donell
@ 2019-07-24 23:50 ` DJ Delorie
2019-07-25 0:03 ` [PATCH] test-container: Install with $(sorted-subdirs) " Tulio Magno Quites Machado Filho
2 siblings, 0 replies; 30+ messages in thread
From: DJ Delorie @ 2019-07-24 23:50 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho; +Cc: stli, libc-alpha
Tulio Magno Quites Machado Filho <tuliom@ascii.art.br> writes:
> I'll take a look and see what's missing.
How about this? The initial setting of subdirs is from Makeconfig:
subdirs = $(sorted-subdirs)
With this change, tst-nss-files-hosts-long passes, where
it failed before with a clean testroot install.
diff --git a/Makefile b/Makefile
index 9fbf705200..ac1125853b 100644
--- a/Makefile
+++ b/Makefile
@@ -402,7 +402,7 @@ ifeq ($(run-built-tests),yes)
done
endif
$(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
- subdirs='$(all-subdirs)'
+ subdirs='$(sorted-subdirs)'
touch $(objpfx)testroot.pristine/install.stamp
tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
^ permalink raw reply [flat|nested] 30+ messages in thread
* [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-24 15:32 ` Tulio Magno Quites Machado Filho
2019-07-24 20:40 ` Carlos O'Donell
2019-07-24 23:50 ` DJ Delorie
@ 2019-07-25 0:03 ` Tulio Magno Quites Machado Filho
2019-07-25 1:32 ` DJ Delorie
` (2 more replies)
2 siblings, 3 replies; 30+ messages in thread
From: Tulio Magno Quites Machado Filho @ 2019-07-25 0:03 UTC (permalink / raw)
To: Stefan Liebler, libc-alpha
Commit 35e038c1d2ccb3a75395662f9c4f28d85a61444f started to use an
incomplete list of subdirs based on $(all-subdirs) causing
testroot.pristine to miss files from nss.
Tested if the list of files in testroot.pristine remains the same.
2019-07-24 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
[BZ #24794]
* Makeconfig (all-subdirs): Improved source comments.
* Makefile (testroot.pristine/install.stamp): Pass
subdirs='$(sorted-subdirs)' to make install.
---
Makeconfig | 6 +++---
Makefile | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Makeconfig b/Makeconfig
index 0e386fbc19..fd36c58c04 100644
--- a/Makeconfig
+++ b/Makeconfig
@@ -1267,9 +1267,9 @@ else
libsupport = $(common-objpfx)support/libsupport.a
endif
-# These are the subdirectories containing the library source. The order
-# is more or less arbitrary. The sorting step will take care of the
-# dependencies.
+# This is a partial list of subdirectories containing the library source.
+# The order is more or less arbitrary. The sorting step will take care of the
+# dependencies and generate sorted-subdirs dynamically.
all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
stdlib stdio-common libio malloc string wcsmbs time dirent \
grp pwd posix io termios resource misc socket sysvipc gmon \
diff --git a/Makefile b/Makefile
index 9fbf705200..ac1125853b 100644
--- a/Makefile
+++ b/Makefile
@@ -402,7 +402,7 @@ ifeq ($(run-built-tests),yes)
done
endif
$(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
- subdirs='$(all-subdirs)'
+ subdirs='$(sorted-subdirs)'
touch $(objpfx)testroot.pristine/install.stamp
tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
--
2.14.5
^ permalink raw reply [flat|nested] 30+ messages in thread* Re: [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-25 0:03 ` [PATCH] test-container: Install with $(sorted-subdirs) " Tulio Magno Quites Machado Filho
@ 2019-07-25 1:32 ` DJ Delorie
2019-07-25 12:46 ` Szabolcs Nagy
2019-07-25 13:24 ` Stefan Liebler
2 siblings, 0 replies; 30+ messages in thread
From: DJ Delorie @ 2019-07-25 1:32 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho; +Cc: stli, libc-alpha
Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com> writes:
> - subdirs='$(all-subdirs)'
> + subdirs='$(sorted-subdirs)'
Considering I posted this as a suggested fix an hour before you did, I'm
going to say LGTM. Of course, I said that last time too, so maybe we
need an independent confirmation? ;-)
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-25 0:03 ` [PATCH] test-container: Install with $(sorted-subdirs) " Tulio Magno Quites Machado Filho
2019-07-25 1:32 ` DJ Delorie
@ 2019-07-25 12:46 ` Szabolcs Nagy
2019-07-25 12:56 ` Carlos O'Donell
2019-07-25 13:24 ` Stefan Liebler
2 siblings, 1 reply; 30+ messages in thread
From: Szabolcs Nagy @ 2019-07-25 12:46 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho, Stefan Liebler, libc-alpha; +Cc: nd
On 25/07/2019 01:03, Tulio Magno Quites Machado Filho wrote:
> Commit 35e038c1d2ccb3a75395662f9c4f28d85a61444f started to use an
> incomplete list of subdirs based on $(all-subdirs) causing
> testroot.pristine to miss files from nss.
>
> Tested if the list of files in testroot.pristine remains the same.
fwiw with this patch aarch64 buildbot would go back to green.
>
> 2019-07-24 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
>
> [BZ #24794]
> * Makeconfig (all-subdirs): Improved source comments.
> * Makefile (testroot.pristine/install.stamp): Pass
> subdirs='$(sorted-subdirs)' to make install.
> ---
> Makeconfig | 6 +++---
> Makefile | 2 +-
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/Makeconfig b/Makeconfig
> index 0e386fbc19..fd36c58c04 100644
> --- a/Makeconfig
> +++ b/Makeconfig
> @@ -1267,9 +1267,9 @@ else
> libsupport = $(common-objpfx)support/libsupport.a
> endif
>
> -# These are the subdirectories containing the library source. The order
> -# is more or less arbitrary. The sorting step will take care of the
> -# dependencies.
> +# This is a partial list of subdirectories containing the library source.
> +# The order is more or less arbitrary. The sorting step will take care of the
> +# dependencies and generate sorted-subdirs dynamically.
> all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
> stdlib stdio-common libio malloc string wcsmbs time dirent \
> grp pwd posix io termios resource misc socket sysvipc gmon \
> diff --git a/Makefile b/Makefile
> index 9fbf705200..ac1125853b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -402,7 +402,7 @@ ifeq ($(run-built-tests),yes)
> done
> endif
> $(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
> - subdirs='$(all-subdirs)'
> + subdirs='$(sorted-subdirs)'
> touch $(objpfx)testroot.pristine/install.stamp
>
> tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-25 12:46 ` Szabolcs Nagy
@ 2019-07-25 12:56 ` Carlos O'Donell
2019-07-29 13:36 ` Tulio Magno Quites Machado Filho
0 siblings, 1 reply; 30+ messages in thread
From: Carlos O'Donell @ 2019-07-25 12:56 UTC (permalink / raw)
To: Szabolcs Nagy, Tulio Magno Quites Machado Filho, Stefan Liebler,
libc-alpha
Cc: nd
On 7/25/19 8:46 AM, Szabolcs Nagy wrote:
> On 25/07/2019 01:03, Tulio Magno Quites Machado Filho wrote:
>> Commit 35e038c1d2ccb3a75395662f9c4f28d85a61444f started to use an
>> incomplete list of subdirs based on $(all-subdirs) causing
>> testroot.pristine to miss files from nss.
>>
>> Tested if the list of files in testroot.pristine remains the same.
>
> fwiw with this patch aarch64 buildbot would go back to green.
This is good enough confirmation for me.
Please install the fix so we can get clean builds for the release.
>>
>> 2019-07-24 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
>>
>> [BZ #24794]
>> * Makeconfig (all-subdirs): Improved source comments.
>> * Makefile (testroot.pristine/install.stamp): Pass
>> subdirs='$(sorted-subdirs)' to make install.
>> ---
>> Makeconfig | 6 +++---
>> Makefile | 2 +-
>> 2 files changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/Makeconfig b/Makeconfig
>> index 0e386fbc19..fd36c58c04 100644
>> --- a/Makeconfig
>> +++ b/Makeconfig
>> @@ -1267,9 +1267,9 @@ else
>> libsupport = $(common-objpfx)support/libsupport.a
>> endif
>>
>> -# These are the subdirectories containing the library source. The order
>> -# is more or less arbitrary. The sorting step will take care of the
>> -# dependencies.
>> +# This is a partial list of subdirectories containing the library source.
>> +# The order is more or less arbitrary. The sorting step will take care of the
>> +# dependencies and generate sorted-subdirs dynamically.
>> all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
>> stdlib stdio-common libio malloc string wcsmbs time dirent \
>> grp pwd posix io termios resource misc socket sysvipc gmon \
>> diff --git a/Makefile b/Makefile
>> index 9fbf705200..ac1125853b 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -402,7 +402,7 @@ ifeq ($(run-built-tests),yes)
>> done
>> endif
>> $(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
>> - subdirs='$(all-subdirs)'
>> + subdirs='$(sorted-subdirs)'
>> touch $(objpfx)testroot.pristine/install.stamp
>>
>> tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
>>
>
--
Cheers,
Carlos.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-25 12:56 ` Carlos O'Donell
@ 2019-07-29 13:36 ` Tulio Magno Quites Machado Filho
2022-01-27 15:25 ` H.J. Lu
0 siblings, 1 reply; 30+ messages in thread
From: Tulio Magno Quites Machado Filho @ 2019-07-29 13:36 UTC (permalink / raw)
To: Carlos O'Donell, Szabolcs Nagy, Stefan Liebler, libc-alpha, dj; +Cc: nd
Carlos O'Donell <carlos@redhat.com> writes:
> On 7/25/19 8:46 AM, Szabolcs Nagy wrote:
>> On 25/07/2019 01:03, Tulio Magno Quites Machado Filho wrote:
>>> Commit 35e038c1d2ccb3a75395662f9c4f28d85a61444f started to use an
>>> incomplete list of subdirs based on $(all-subdirs) causing
>>> testroot.pristine to miss files from nss.
>>>
>>> Tested if the list of files in testroot.pristine remains the same.
>>
>> fwiw with this patch aarch64 buildbot would go back to green.
>
> This is good enough confirmation for me.
>
> Please install the fix so we can get clean builds for the release.
I've just pushed it to master.
DJ,
I took the freedom to add your name to the ChangeLog file too as you proposed
the same fix. ;-)
Pushed as 354e4c1adddb1.
--
Tulio Magno
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-29 13:36 ` Tulio Magno Quites Machado Filho
@ 2022-01-27 15:25 ` H.J. Lu
0 siblings, 0 replies; 30+ messages in thread
From: H.J. Lu @ 2022-01-27 15:25 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho, Libc-stable Mailing List
Cc: Carlos O'Donell, Szabolcs Nagy, Stefan Liebler, libc-alpha,
DJ Delorie, nd
On Mon, Jul 29, 2019 at 6:37 AM Tulio Magno Quites Machado Filho
<tuliom@linux.ibm.com> wrote:
>
> Carlos O'Donell <carlos@redhat.com> writes:
>
> > On 7/25/19 8:46 AM, Szabolcs Nagy wrote:
> >> On 25/07/2019 01:03, Tulio Magno Quites Machado Filho wrote:
> >>> Commit 35e038c1d2ccb3a75395662f9c4f28d85a61444f started to use an
> >>> incomplete list of subdirs based on $(all-subdirs) causing
> >>> testroot.pristine to miss files from nss.
> >>>
> >>> Tested if the list of files in testroot.pristine remains the same.
> >>
> >> fwiw with this patch aarch64 buildbot would go back to green.
> >
> > This is good enough confirmation for me.
> >
> > Please install the fix so we can get clean builds for the release.
>
> I've just pushed it to master.
>
> DJ,
> I took the freedom to add your name to the ChangeLog file too as you proposed
> the same fix. ;-)
>
> Pushed as 354e4c1adddb1.
>
> --
> Tulio Magno
I am backporting this to older release branches.
--
H.J.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-container: Install with $(sorted-subdirs) [BZ #24794]
2019-07-25 0:03 ` [PATCH] test-container: Install with $(sorted-subdirs) " Tulio Magno Quites Machado Filho
2019-07-25 1:32 ` DJ Delorie
2019-07-25 12:46 ` Szabolcs Nagy
@ 2019-07-25 13:24 ` Stefan Liebler
2 siblings, 0 replies; 30+ messages in thread
From: Stefan Liebler @ 2019-07-25 13:24 UTC (permalink / raw)
To: Tulio Magno Quites Machado Filho, libc-alpha
Hi Tulio,
with this patch, testroot.pristine contains the missing getent / makedb
executables and libnss_* libraries.
Now nss/tst-nss-files-hosts-long and all other container tests are passing.
Thanks,
Stefan
On 7/25/19 2:03 AM, Tulio Magno Quites Machado Filho wrote:
> Commit 35e038c1d2ccb3a75395662f9c4f28d85a61444f started to use an
> incomplete list of subdirs based on $(all-subdirs) causing
> testroot.pristine to miss files from nss.
>
> Tested if the list of files in testroot.pristine remains the same.
>
> 2019-07-24 Tulio Magno Quites Machado Filho <tuliom@linux.ibm.com>
>
> [BZ #24794]
> * Makeconfig (all-subdirs): Improved source comments.
> * Makefile (testroot.pristine/install.stamp): Pass
> subdirs='$(sorted-subdirs)' to make install.
> ---
> Makeconfig | 6 +++---
> Makefile | 2 +-
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/Makeconfig b/Makeconfig
> index 0e386fbc19..fd36c58c04 100644
> --- a/Makeconfig
> +++ b/Makeconfig
> @@ -1267,9 +1267,9 @@ else
> libsupport = $(common-objpfx)support/libsupport.a
> endif
>
> -# These are the subdirectories containing the library source. The order
> -# is more or less arbitrary. The sorting step will take care of the
> -# dependencies.
> +# This is a partial list of subdirectories containing the library source.
> +# The order is more or less arbitrary. The sorting step will take care of the
> +# dependencies and generate sorted-subdirs dynamically.
> all-subdirs = csu assert ctype locale intl catgets math setjmp signal \
> stdlib stdio-common libio malloc string wcsmbs time dirent \
> grp pwd posix io termios resource misc socket sysvipc gmon \
> diff --git a/Makefile b/Makefile
> index 9fbf705200..ac1125853b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -402,7 +402,7 @@ ifeq ($(run-built-tests),yes)
> done
> endif
> $(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
> - subdirs='$(all-subdirs)'
> + subdirs='$(sorted-subdirs)'
> touch $(objpfx)testroot.pristine/install.stamp
>
> tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
>
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 20:52 ` Florian Weimer
2019-07-09 20:59 ` DJ Delorie
@ 2019-07-09 21:08 ` Carlos O'Donell
2019-07-09 23:55 ` DJ Delorie
1 sibling, 1 reply; 30+ messages in thread
From: Carlos O'Donell @ 2019-07-09 21:08 UTC (permalink / raw)
To: Florian Weimer, DJ Delorie; +Cc: libc-alpha
On 7/9/19 4:52 PM, Florian Weimer wrote:
> I think applying this patch as-is would make glibc development
> substantially more difficult for me.
Where do you draw the line? Eventually the testing is going to
slow down your edit/build/test cycle. If this is a problem for
developers then we need some kind of objective "line" that we
try not to cross.
Should we have a staggered approach?
make check-sanity
-- Run as many tests as fit into 10s.
-- Run all tests which are not support_become_root,
test-in-container, or xcheck tests, since these
can be expensive.
make check
-- No time limit for testing.
-- Run all tests which are not xcheck tests.
make xhceck
-- No time limit for testing.
-- Run all tests including xcheck tests.
--
Cheers,
Carlos.
^ permalink raw reply [flat|nested] 30+ messages in thread* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 21:08 ` [PATCH] test-in-container: Install locales into the test container Carlos O'Donell
@ 2019-07-09 23:55 ` DJ Delorie
2019-07-12 4:02 ` Carlos O'Donell
2019-07-12 5:49 ` Florian Weimer
0 siblings, 2 replies; 30+ messages in thread
From: DJ Delorie @ 2019-07-09 23:55 UTC (permalink / raw)
To: Carlos O'Donell; +Cc: fweimer, libc-alpha
"Carlos O'Donell" <carlos@redhat.com> writes:
> Where do you draw the line?
Originally I wanted the "make install" to trigger just before the first
test that actually needed it, but I found it very difficult to do that.
Eventually more and more tests will run inside the container, and that
optimization would be useless anyway.
Perhaps we could at least add $(PARALLELMFLAGS) to the "make install" ?
If it's not already propogated by some other means? Or otherwise
optimize the install process, to limit the overhead during testing?
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 23:55 ` DJ Delorie
@ 2019-07-12 4:02 ` Carlos O'Donell
2019-07-12 5:49 ` Florian Weimer
1 sibling, 0 replies; 30+ messages in thread
From: Carlos O'Donell @ 2019-07-12 4:02 UTC (permalink / raw)
To: DJ Delorie; +Cc: fweimer, libc-alpha
On 7/9/19 7:54 PM, DJ Delorie wrote:
> "Carlos O'Donell" <carlos@redhat.com> writes:
>> Where do you draw the line?
>
> Originally I wanted the "make install" to trigger just before the first
> test that actually needed it, but I found it very difficult to do that.
> Eventually more and more tests will run inside the container, and that
> optimization would be useless anyway.
>
> Perhaps we could at least add $(PARALLELMFLAGS) to the "make install" ?
> If it's not already propogated by some other means? Or otherwise
> optimize the install process, to limit the overhead during testing?
My point is that no matter how good you optimize, eventually there will
be enough tests that the time will creep up again.
I want Florian to be productive, and I want a build system that supports
a straight forward edit/build/test workflow.
My question is: How long is too long to wait for the results of testing
when you are doing a edit/build/test/debug cycle?
Should 'make check' be running enough tests to fill X seconds?
--
Cheers,
Carlos.
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: [PATCH] test-in-container: Install locales into the test container
2019-07-09 23:55 ` DJ Delorie
2019-07-12 4:02 ` Carlos O'Donell
@ 2019-07-12 5:49 ` Florian Weimer
1 sibling, 0 replies; 30+ messages in thread
From: Florian Weimer @ 2019-07-12 5:49 UTC (permalink / raw)
To: DJ Delorie; +Cc: Carlos O'Donell, libc-alpha
* DJ Delorie:
> Perhaps we could at least add $(PARALLELMFLAGS) to the "make install" ?
> If it's not already propogated by some other means? Or otherwise
> optimize the install process, to limit the overhead during testing?
$(MAKE) already implies $(PARALLELMFLAGS).
Thanks,
Florian
^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2022-01-27 15:26 UTC | newest]
Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-09 17:32 [PATCH] test-in-container: Install locales into the test container Florian Weimer
2019-07-09 18:07 ` DJ Delorie
2019-07-09 20:52 ` Florian Weimer
2019-07-09 20:59 ` DJ Delorie
2019-07-09 21:02 ` Florian Weimer
2019-07-09 21:11 ` Partial test suite run builds corrupt test-in-container testroot Carlos O'Donell
2019-07-10 0:36 ` DJ Delorie
2019-07-10 5:50 ` Florian Weimer
2019-07-10 16:34 ` DJ Delorie
2019-07-10 16:35 ` Florian Weimer
2019-07-10 16:38 ` DJ Delorie
2019-07-19 11:52 ` Florian Weimer
2019-07-22 20:45 ` [PATCH] test-container: Install with $(all-subdirs) [BZ #24794] Tulio Magno Quites Machado Filho
2019-07-22 20:56 ` DJ Delorie
2019-07-23 16:29 ` Tulio Magno Quites Machado Filho
2019-07-24 14:48 ` Stefan Liebler
2019-07-24 15:32 ` Tulio Magno Quites Machado Filho
2019-07-24 20:40 ` Carlos O'Donell
2019-07-24 23:50 ` DJ Delorie
2019-07-25 0:03 ` [PATCH] test-container: Install with $(sorted-subdirs) " Tulio Magno Quites Machado Filho
2019-07-25 1:32 ` DJ Delorie
2019-07-25 12:46 ` Szabolcs Nagy
2019-07-25 12:56 ` Carlos O'Donell
2019-07-29 13:36 ` Tulio Magno Quites Machado Filho
2022-01-27 15:25 ` H.J. Lu
2019-07-25 13:24 ` Stefan Liebler
2019-07-09 21:08 ` [PATCH] test-in-container: Install locales into the test container Carlos O'Donell
2019-07-09 23:55 ` DJ Delorie
2019-07-12 4:02 ` Carlos O'Donell
2019-07-12 5:49 ` Florian Weimer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).