Testing with --target=FOO

Testing with --target=FOO

[Ben Elliston]

When --target= is specified to sid's configure script, the sid
Makefiles build generic components and components related to that
target.  For example, the ARM reference timer is only built when all
targets are enabled or if --target=arm-*.

This causes problems when testing builds with --target specified.
Many of the generic components require a CPU component as part of the
simulation to be tested.  Frequently, we have used the more-dependable
hw-cpu-arm7t.

When the build is configured for other specific targets, the
hw-cpu-arm7t is not built and huge portions of the testsuite fails.
I'd like to discuss solutions to this problem.

Ideas I have had thus far (in increasing order of preference):

 + Mark such tests as "untested".

 + Build a small dummy CPU component that can be compiled
   unconditionally.  This will take time to write and may not be
   adequate for the kinds of tests we need to do.

 + Build the arm7t model unconditionally, regardless of the target.
   Only the cpu is required as far as I can tell.

Thoughts?

Ben

Re: Testing with --target=FOO

[Frank Ch. Eigler]

Hi -

bje wrote:
: When --target= is specified to sid's configure script, the sid
: Makefiles build generic components and components related to that
: target.  [...]
: This causes problems when testing builds with --target specified.
: Many of the generic components require a CPU component as part of the
: simulation to be tested.  [...]

Which ones?  Back when we only had an arm model to work with, we
were expedient and hard-coded tests to use that.  But now, with
many more ports in-house, those old tests should generally be
made cpu-independent, or replicated in multiple cpu-dependent
copies, as appropriate.

: Ideas I have had thus far (in increasing order of preference):
: 
:  + Mark such tests as "untested".

Maybe.

:  + Build a small dummy CPU component that can be compiled
:    unconditionally.  This will take time to write and may not be
:    adequate for the kinds of tests we need to do.

Nah.

:  + Build the arm7t model unconditionally, regardless of the target.
:    Only the cpu is required as far as I can tell.

Can you provide more details on why some canonical cpu is required
for what are presumably non-cpu tests?


or #4:

+  Change the test suite.


- FChE
-- 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE6+8riVZbdDOm/ZT0RAgVPAJ469qO/JPqD0MxAFaEfPg6KglEyoQCcDrrt
9AXKZeZMoFg6gQl/oj8l1vI=
=sCDK
-----END PGP SIGNATURE-----

re: Testing with --target=FOO

[matthew green]

actually, one thing that has annoyed me for a while (infact, ever
since i first realised it was the case) is that all the CPU
components are thrown into the one shared object.   i've often
been annoyed because my newly installed libcgencpu.la, built for
a only the cpu i'm working on, suddenly can't run other cpu's.
also, a full libcgencpu.so can get REALLY large.

so, back to ben's question, i'd prefer it if all cpus were built
always (possibly some configury to disable them?), but that they
each had their own component library.  that should fix the testing
problem ben has raised, as well as my issues above.


.mrg.

Re: Testing with --target=FOO

[Ben Elliston]

>>>>> "Frank" == Frank Ch Eigler <fche@redhat.com> writes:

  Frank> Which ones?  Back when we only had an arm model to work with, we
  Frank> were expedient and hard-coded tests to use that.  But now, with
  Frank> many more ports in-house, those old tests should generally be
  Frank> made cpu-independent, or replicated in multiple cpu-dependent
  Frank> copies, as appropriate.

Right.  I took a moment to do some more investigation:

	find sid/component/testsuite -type f | xargs grep hw-cpu-arm7t

and discovered that the majority of the offenders are in the cfgroot
component testsuite.  For example:

	sid/component/testsuite/sidcomp.cfgroot/multiDso.badconf:new hw-cpu-arm7t cpu

  Frank> Can you provide more details on why some canonical cpu is
  Frank> required for what are presumably non-cpu tests?

Seemingly no good reason.

  Frank> or #4:
  Frank> +  Change the test suite.

This looks like the way to proceed.  I'll submit patches
sometime. Thanks.

Ben

Re: Testing with --target=FOO

[Ben Elliston]

>>>>> "Frank" == Frank Ch Eigler <fche@redhat.com> writes:

  Frank> Which ones?  Back when we only had an arm model to work with, we
  Frank> were expedient and hard-coded tests to use that.  But now, with
  Frank> many more ports in-house, those old tests should generally be
  Frank> made cpu-independent, or replicated in multiple cpu-dependent
  Frank> copies, as appropriate.

I have made a start at making the sidcomp.cfgroot tests more
cpu-independent.  In many cases, these tests just use an arm7t cpu
where any other component would be equally suitable.

I've checked in a few testsuite fixes.  More to come.

Ben