Re: [PATCH 04/12] sim/testsuite/cris/c: Use -sim3 but only for newlib targets

[Hans-Peter Nilsson <hp@axis.com>]

> Date: Wed, 16 Feb 2022 00:25:07 -0500
> From: Mike Frysinger <vapier@gentoo.org>

> On 15 Feb 2022 00:02, Hans-Peter Nilsson via Gdb-patches wrote:
> > Commit a39487c6685f "sim: cris: use -sim with C tests for cris-elf
> > targets" caused " -sim" to be appended to CFLAGS_FOR_TARGET for
> > cris*-*-elf, where testing had until then relied on
> > "RUNTESTFLAGS=--target_board=cris-sim" being passed when running "make
> > check-sim", adding the right options.  While "-sim" happens to work,
> > the baseboard-file cris-sim.exp uses "-sim3" so for consistency use
> > that instead.
> > 
> > Then commit b42f20d2ac72 "sim: testsuite: drop most specific istarget
> > checks" caused " -sim" to be appended for *all* targets, which just
> > doesn't work.  For example, for crisv32-linux-gnu, that's not a
> > recognized option and will cause a dejagnu error and further testing
> > in c.exp will be aborted.
> > 
> > While cris-sim.exp appends "-static" for *-linux-gnu, further changes
> > in the test-suite have caused "linux"-specific tests to break, so that
> > part will be tended to separately.
> > 
> > But, save and restore CFLAGS_FOR_TARGET around the modification and
> > use where needed, to not have the CRIS-specific modification affect a
> > continuing test-run (possibly for other targets).
> 
> i'm trying to get away from needing dejagnu boards at all.  it brings nothing
> to the table when it comes to testing the sim itself.

I know you're of that opinion, but I'll say this once again (it
was decades ago last time): you're wrong.  This leads you to
discarding half of dejagnu and working *against* it rather than
*with it*.

>  ideally we should have
> a single sim binary that supports all targets simultaneously, and it's only
> runtime options (or dynamic probing) that selects between them.  that's why
> #progos was introduced -- so tests could declare which env they're targeting
> and the test framework can run the simulator with the right settings.
> 
> one can now do:
> $ ./configure --enable-targets=all
> $ make check-sim
> 
> and every architecture will be built & tested.  no need for multiple build
> dirs for diff targets or sep runs with diff runtestflags.

Right, I figured that's your preferred setup.  You can certainly
make dejagnu baseboards work in this scenario: just call *each*
of the one one that fits when testing *each* simulator, and
clear the slate in-between.

> at some point i also want to delete all the custom compile+run logic in the
> testsuite/cris/ tree.  that's why i spent so much time pulling code out and
> into the common one.

The baby went with the bath-water here.  (Or IOW, if you pull
out *all* of the test-suites the testsuite will be really
simple!)

> i even have a poc locally that deletes the dejagnu framework entirely and
> switches to Automake test harness, but i haven't quite figured out how to
> cleanly handle the all_machs multiplex logic in Automake.  i eventually
> pulled out individual cleanups and merged them so at least `make check`
> works in a multi-target build, and isn't nearly as slow as it was.
> -mike

Ouch.  I haven't found the "automake test harness" to be of use
for serious testing.

brgds, H-P