Re: testsuite: introduce hostedlib effective target

[Jonathan Wakely <jwakely@redhat.com>]

On Tue, 7 Nov 2023 at 10:18, Jonathan Wakely <jwakely@redhat.com> wrote:
>
> On Tue, 7 Nov 2023 at 10:04, Alexandre Oliva <oliva@adacore.com> wrote:
> >
> > [adding libstdc++@]
> >
> > On Nov  5, 2023, Mike Stump <mikestump@comcast.net> wrote:
> >
> > > Ick.
> >
> > Indeed ;-)
> >
> > > I wish there were fewer changed lines and not 1 per test
> > > case. It feels like we've painted ourselves into a corner.
> >
> > The libstdc++ testsuite took a different approach, detecting missing
> > headers (and libraries?) at error pruning time, and xfailing the tests,
> > which seems to be more in line with what you are looking for.
> >
> > That approach, though more expedient, seems more fragile to me, in that
> > an actual bug that caused headers to go missing would cause tests to be
> > silently skipped rather than fail.
>
> I don't think we XFAIL based on missing headers. We XFAIL based on a
> specific #error message in certain headers.
>
> If a header goes missing, we'll still XFAIL.
>
> >
> > I expect the set of headers, and thus of affected tests, won't by very
> > dynamic, so it's kind of a one-shot change.
> >
> > Of course new tests might be added that rely on such headers, and would
> > likely go unnoticed until someone tries them on a non-hosted libstdc++.
>
> Since GCC 13 you don't need to build a non-hosted libstdc++ to test
> it, you can just add -ffreestanding to the runtestflags.
>
> > We could alleviate this if libstdc++ headers that are not installed on
> > hosted systems issued a warning (conditional on some macro defined by
> > the testsuite, say -D_GLIBCXX_WARN_HOSTED_ONLY).
>
> That's exactly what happens (except #error not #warning) when you
> compile with -ffreestanding.
>
> >  For tests aimed
> > exclusively at hosted libstdc++, we'd then use a dg directive that both
> > implied this requirement, and changed the macro definition to suppress
> > the warning.  Then anyone who added a testcase that included hosted
> > headers without indicating its hostedlib requirement would get a fail
> > even when testing with a hosted libstdc++.
>
> I don't think we need to add checks for a new macro and then use that
> when testing, you can just test with -ffreestanding instead. This
> already works today.

Ah, reading back in the thread for  the context I missed, I see that
you're specifically testing a --disable-hosted-libstdcxx build. In
that case some headers really will be absent, not just
present-with-#error. But I am still not concerned about failing to
notice if a header goes unintentionally missing, because the libstdc++
testsuite will still notice that.

We don't prune based on "no such header" errors, so would still get
FAILs for those tests that depend on headers which are supposed to be
present for freestanding.