[Bug build/30157] parallel "make check" randomly skips tests

["vincent-srcware at vinc17 dot net" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=30157

--- Comment #9 from Vincent Lefèvre <vincent-srcware at vinc17 dot net> ---
(In reply to Carlos O'Donell from comment #8)
> I agree it doesn't answer your question, but it does set the expectation
> that anything you find that is not driven by dependencies is a mistake
> (barring some complexity around tests-container).

Tests are there to find bugs, and bugs may depend on the run-time environment
in a broad sense (this may be environment variables, but also the current time,
the scheduling, in case of race conditions). So it does not make much sense to
say that if a test gave some result in some run, it will always give the same
result just because the source files have not changed.

> Have you seen tests that were incorrectly executed based on their
> dependencies?

Yes, for instance, with 3 "make check" without a change of the source files
(thus needing a build of everything only after the first one[*]), but
LD_PRELOAD was unset after the first one, elf/tst-audit23
* was executed after the first one (obviously) and failed due to the
LD_PRELOAD;
* was not executed after the second one;
* was executed after the third one (and no longer failed as LD_PRELOAD was no
longer set).

[*] unless there is a bug in the dependencies for the build.

-- 
You are receiving this mail because:
You are on the CC list for the bug.