From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 48) id 64EF43858017; Sat, 25 Feb 2023 19:04:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 64EF43858017 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1677351888; bh=553Ij+HiYIm9kjjzYVgv5Ramr1GhbCXlmVYMYqnHeF4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Ez8cEA98isLr0peAnYyTHTu4W+N+FNBAg5jrR8I5k6G0CJz6fuQnl12Ov0E3i9tmO r6z47EdDeRZ2G5E1VjlSlyOmx95Rau0LJH4FMVMJVQYkzx6aUal6Zewu8i2zsNIBX/ nbtAKN6XlUeEVXM6iXYywIDDfTXI4HmXUFebvpUw= From: "carlos at redhat dot com" To: glibc-bugs@sourceware.org Subject: [Bug build/30157] parallel "make check" randomly skips tests Date: Sat, 25 Feb 2023 19:04:47 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: glibc X-Bugzilla-Component: build X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: carlos at redhat dot com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: P2 X-Bugzilla-Assigned-To: unassigned at sourceware dot org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://sourceware.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 List-Id: https://sourceware.org/bugzilla/show_bug.cgi?id=3D30157 --- Comment #11 from Carlos O'Donell --- (In reply to Vincent Lef=C3=A8vre from comment #9) > (In reply to Carlos O'Donell from comment #8) > > I agree it doesn't answer your question, but it does set the expectation > > that anything you find that is not driven by dependencies is a mistake > > (barring some complexity around tests-container). >=20 > Tests are there to find bugs, and bugs may depend on the run-time > environment in a broad sense (this may be environment variables, but also > the current time, the scheduling, in case of race conditions). So it does > not make much sense to say that if a test gave some result in some run, it > will always give the same result just because the source files have not > changed. Your are correct that the test results may vary depending on the system configuration, but what you're asking for is outside the current assumption= s of the implementation. Describing the expectations highlights that the system = is behaving *as expected*, but what you are asking for is an enhancement to the current build and test system. The build and test system *expects* a correctly configured developer system= and optimizes for that to deduce developer iteration time to look at regression test results. If your system is incorrectly configured you may need to clean the whole bu= ild and start again or `find . -name *.out | xargs rm` or something similar. Could we do better? I expect we could. > > Have you seen tests that were incorrectly executed based on their > > dependencies? >=20 > Yes, for instance, with 3 "make check" without a change of the source fil= es > (thus needing a build of everything only after the first one[*]), but > LD_PRELOAD was unset after the first one, elf/tst-audit23 > * was executed after the first one (obviously) and failed due to the > LD_PRELOAD; > * was not executed after the second one; > * was executed after the third one (and no longer failed as LD_PRELOAD was > no longer set). >=20 > [*] unless there is a bug in the dependencies for the build. (In reply to Vincent Lef=C3=A8vre from comment #10) > I've done another "make check". So now there are tests whose last executi= on > is > * 2023-02-24 (today) > * 2023-02-23 > * 2023-02-22 around 17:43 > * 2023-02-22 around 17:12 (corresponding to the first "make check") >=20 > (No source files have been changed after the initial checkout.) Just to be clear, while you say "No source files have been changed", what is actually important is *dependencies*. In the case of tst-audit23 we have the following: 2283 $(objpfx)tst-audit23.out: $(objpfx)tst-auditmod23.so \ 2284 $(objpfx)tst-audit23mod.so The tests need their output file (the *.out file) and this says that output needs the two audit modules compiled, so they get built before the test is built and run. I would be curious to know if the *.out file was changed, or the *.so files were changed, since they are are of the set of dependencies for the test. --=20 You are receiving this mail because: You are on the CC list for the bug.=