Re: egcs testsuite & dejagnu : A special case?

Re: egcs testsuite & dejagnu : A special case?

[Mike Stump]

> Date: Tue, 30 Jun 1998 09:44:28 -0400 (EDT)
> From: Craig Burley <burley@gnu.org>

> >dejagnu however is not exactly a model of cleanliness.

> This reminds me of a thought I had the other day while not on-line.

> I've gotten the impression (not knowing much about how all this
> testing stuff actually works at this point) that some of the
> problems we've run into are due to attempting to be able to test an
> *uninstalled* compiler as well as an installed one.  Is that true?

:-)  Craig, what a question!  There's not enough information for me to
tell...

For C++ there is a way to run the compiler out of the build tree such
that is is almost indistinguishable from an installed compiler.  For
testing purposes, it is usually indistinguishable.

dejagnu has some of the logic and the Makefiles have some of the
logic.

gcc also runs fairly well I think.

> Certainly it's often the case that I want to run tests of my own
> (using my own primitive test suite, or just try out a submitted test
> case or something), and I've done a variety of things over the years
> to cope with the fact that I want to try out a *built*, but not
> *installed*, version of the compiler.

For a maintainer, this is critical.  I do something like (when last I
did this):

#!/bin/sh
case "$HOSTTYPE" in
    sparc) arch=sol
	;;
    hp9000s700) arch=hppa
	;;
    sgi) arch=sgi
	;;
    alpha) arch=alpha
	;;
    RISC6000) arch=rs6000
	;;
esac

#arch=sol
#arch=tic80
#arch=sun4
#arch=sgi
#arch=alpha
#arch=rs6000
#arch=alpha-dec-osf2.0
#arch=hppa

#arch=i586-unknown-linux
#arch=i586

srcdir=${HOME}/egcs
objdir=${HOME}/$arch

LIBS=
echo " $* " | grep " -[cSE] " >/dev/null
if [ $? = 1 ]; then LIBS="-lstdc++"
fi
exec ${objdir}/gcc/xgcc -B${objdir}/gcc/ ${1+"$@"} -I${objdir}/libio -I${srcdir}/libg++/src -I${srcdir}/libio -I${srcdir}/libstdc++ -I${srcdir}/libstdc++/stl -I${objdir}/gcc/include -L${objdir}/libg++ -L${objdir}/libstdc++ $LIBS

Not pretty, but it is close.

> (On my old 486/33, I had a bunch of shells scripts installed in
> /usr/local/bin that'd invoke the compiler in canonical places in my
> home directory, using the appropriate -B and -L switches, etc.  Now
> that I have a much faster system, I just specify a unique --prefix
> and do "make install" each time it seems appropriate, which takes
> only about a minute, but not everybody has that.  And, I admit it's
> tedious to keeping typing full pathnames like
> `/develop/g77/0.5.23/bin/g77' to specify which version of the
> compiler I want to run.  That's a minor problem compared to back
> when I couldn't really maintain more than two versions, though!)

I install a full tree every now and then, but for the day to day
stuff, I only ever use the build directory.  Also, you can get around
names by having sym links from things like ../local/bin/gcc272 to
../local/gcc-2.7.2.3/bin/gcc have only have local/bin in your path.

Further, I usually g++ -E to get a .ii file, and then I can feed that
stright to cc1plus by hand.  I then just look at the .s file and go
from there.

> It occurred to me that, maybe, we could avoid much of this trouble
> by, instead of working around the differences between installed and
> uninstalled compilers *externally* (to the compiler itself, that
> is), we could define a new interface to the compiler *itself* that,
> when invoked, amounts to saying "I'm invoking the *uninstalled*
> compiler", and leave all the mucking about with pathnames and such
> to the new interface.

A way I like better is to have an etc/{g77,gcc,g++} in the build tree
that invokes out of the build tree, and if you want it, you just use
it.

> For example, what if we had the build process automatically build,
> but not install (obviously), shell scripts

Yes.

> Does this idea have any merit?

Yes.

> Offhand it seems fairly simple to implement, and, once we change the
> pertinent tools (egcs top-level Makefile such as when doing `make
> check') to use them, could really reduce the maintenance headaches
> we seem to be having with trying to do useful work on uninstalled
> compilers.

I din't have any headaches because this is what I already do.  :-)

Re: egcs testsuite & dejagnu : A special case?

[Craig Burley]

>I install a full tree every now and then, but for the day to day
>stuff, I only ever use the build directory.  Also, you can get around
>names by having sym links from things like ../local/bin/gcc272 to
>../local/gcc-2.7.2.3/bin/gcc have only have local/bin in your path.

I'm a bit lost in the parsing of that, but I think I know what you're
saying.  If I'm right, that eases the invocation of a compiler
installed in one of a variety of paths named after the version, to
keep track of things.  (I've been doing that for the past several
months, and boy is it worthwhile, and worth the extra typing, to just
*know* what version of what compiler I'm invoking at any one time.)

What I'm suggesting is a way to "elegantly" export, from the egcs
"system", a single command name that invokes the built command
instead of the installed command.

I recognize that we've already gotten "this" to work, but by "this"
I think is typically meant "exporting of the *internal knowledge*
of how to invoke a built, but perhaps not installed, command in
egcs", in that other products have what *should* be impertinent
knowledge of egcs/gcc internals.  (I'm thinking of the recent
`g77 -v' fiasco, for example.)

Generally, I much prefer the concept of data hiding, when we can get
away with it.  :)

>A way I like better is to have an etc/{g77,gcc,g++} in the build tree
>that invokes out of the build tree, and if you want it, you just use
>it.

IMO, better.  Much, much better.  `etc' is a widely recognized name
for this sort of stuff; if people don't like that, other names
could be `built' and such, but I like `etc' offhand.

>> For example, what if we had the build process automatically build,
>> but not install (obviously), shell scripts
>
>Yes.
>
>> Does this idea have any merit?
>
>Yes.

Glad to hear I'm not obviously crazy.  I think if we'd had this
about 5 years ago in gcc, lots of annoying things would have been
much easier to set up and deal with.  But I don't think it's
too late to be worth doing, especially if it means, down the road,
we can someday rip out internal knowledge of egcs/gcc from other
products that really shouldn't be knowin' that stuff.

Would this be worth trying to put into 1.1?  (Isn't the freeze date
for that coming up, like, this week?)  If so, I'll try and do the
work tout-de-suite myself unless someone (anyone :) with more
scripting knowledge steps forward.  Otherwise, I'll just leave it
on my list and get to it when it percolates to the top (should
be within a couple of months or so, though the fact that I'd find
it personally useful might speed things up).

        tq vm, (burley)

Re: egcs testsuite & dejagnu : A special case?

[Jeffrey A Law]

  > I'm a bit lost in the parsing of that, but I think I know what you're
  > saying.  If I'm right, that eases the invocation of a compiler
  > installed in one of a variety of paths named after the version, to
  > keep track of things.  (I've been doing that for the past several
  > months, and boy is it worthwhile, and worth the extra typing, to just
  > *know* what version of what compiler I'm invoking at any one time.)
  > 
  > What I'm suggesting is a way to "elegantly" export, from the egcs
  > "system", a single command name that invokes the built command
  > instead of the installed command.
Thinking more about this, it basically means putting a large part of
dejagnu's target.exp & libgloss.exp into gcc itself.  That may not
be wise :-) :-)  But if someone wants to experiment and find a good
way to do it, it would be a big help for the developers.

  > Would this be worth trying to put into 1.1?  (Isn't the freeze date
  > for that coming up, like, this week?) 
I doubt it.  I think we've got more important issues to deal with for 1.1.

jeff

Re: egcs testsuite & dejagnu : A special case?

[Carlo Wood]

| Would this be worth trying to put into 1.1?  (Isn't the freeze date
| for that coming up, like, this week?)  If so, I'll try and do the
| work tout-de-suite myself unless someone (anyone :) with more
| scripting knowledge steps forward.  Otherwise, I'll just leave it
| on my list and get to it when it percolates to the top (should
| be within a couple of months or so, though the fact that I'd find
| it personally useful might speed things up).
| 
|         tq vm, (burley)

Would you mind giving an exact description of the gcc invokation
etc that you have in mind, and what it will exactly accomplish?
I am not worried about the patch itself; however it is still unclear
what the `user interface' will become.  I am thinking about a short
alinea that describes the syntax and a few example locations of the
involved files.

 Carlo Wood  <carlo@runaway.xs4all.nl>

Re: egcs testsuite & dejagnu : A special case?

[Craig Burley]

>Would you mind giving an exact description of the gcc invokation
>etc that you have in mind, and what it will exactly accomplish?
>I am not worried about the patch itself; however it is still unclear
>what the `user interface' will become.  I am thinking about a short
>alinea that describes the syntax and a few example locations of the
>involved files.

Essentially, the idea I have is that, after `make' or `make bootstrap'
in gcc (or above in egcs, of course), and given that /home/me/egcs/gcc
is the name of the gcc build directory, the command (shell script)

  /home/me/egcs/gcc/etc/gcc ...

has "exactly" the same behavior as would, the sequence

  make install
  /usr/bin/gcc ...

(assuming --prefix=/usr during configuration).

And, "exactly" is quoted because it's probably not possible to make
it exact, but at least to get the equivalent of using the proper
-B and -L options, plus whatever other options are appropriate.

And, to clarify, /home/me/egcs/gcc/etc/gcc would be copyable to and
runnable from anywhere, but still work the same way.  So it'd contain
absolute pathnames.

For example, the contents of /home/me/egcs/gcc might be (AFAIK):

  #!/bin/sh
  /home/me/egcs/gcc/xgcc -B/home/me/egcs/gcc/ -L/home/me/egcs/gcc $*

It seems pretty straightforward to implement.  The tricks are getting
the options right (which might be, in the strictest sense, impossible,
but if it covers most of "exactly", it's worth doing and worrying
about the rest later) and modifying the build procedures so anything
important vis-a-vis such a command currently done during `make install'
is done during `make [bootstrap]' instead.  (I don't think building
the Info docs is an example of that.  There might be nothing like
this that needs to be done, but it's worth taking a look as part of
this project.)

Repeat the above for any other command/script installed by `make
install' that depends on built-in knowledge of the *installation*
location, and thus needs to be explicitly told to look in the build
directory instead.

So, include g++, g77, and other languages' drivers in that -- there
would therefore be .../etc/g++, ../etc/g77, and perhaps others as
well.  (I'm guessing protoize and unprotoize don't *need* to be
encapsulated this way, but perhaps they should be, or links made using
absolute symbolic, or hard, links, for completeness.  That'd be
kinda nice -- "ls etc" in a gcc build directory would be a quick
way to list all the "exported" commands that are normally installed,
which sysadmins and hackers of all sorts might appreciate.)

The impact of doing this should be that running an arbitrary
test suite, or building anything else that takes an arbitrary
`CC=whatever' definition, can simply be pointed to the .../etc/
pathname that is appropriate (heck, .../etc/gcc could even be
installed as /usr/bin/cc if someone was brave enough) without
having to worry about -B and -L options, or whatever else
a given version of gcc might need to run properly out of the
build directory.

(Of course, once the build directory is blown away, the script
stops working.  The behavior in such a case should be appropriate,
i.e. a not-found message and error status code returned by the
script.)

        tq vm, (burley)

Re: egcs testsuite & dejagnu : A special case?

[Carlo Wood]

Ok, that is a nice design :).
The only thing that is a bit unclear to me is how
you will make it use the correct shared libraries
at run time, of an executable compiled like that.

Essentially this problem is new since egcs is
including shared libraries in the package, and
thus the shared library version becomes a part
of the version of egcs.

The same problem (will) occur when the -V option
is used.  You should at least wait till H.J.
added his patches to rename these shared libraries
to something that includes the version of the
compiler I think.

Apart from that, it would have a logical `feel'
if it was possible to use the -V option for this,
ie:

gcc -V 2.7.2.3 ...
gcc -V 2.8.1 ...
gcc -V 2.91.45 ...
gcc -V built ...

Or something similar, where `gcc' would be just
/usr/bin/gcc (some (old) egcs version).

-- 
 Carlo Wood  <carlo@runaway.xs4all.nl>

export EGCS_BUILD_DIR=/usr/src/egcs/egcs-cvs

Re: egcs testsuite & dejagnu : A special case?

[Craig Burley]

>Ok, that is a nice design :).
>The only thing that is a bit unclear to me is how
>you will make it use the correct shared libraries
>at run time, of an executable compiled like that.

The exact same way that current uses of the just-built compiler do.  That
is, if it's a problem with my scheme, it's a problem with the current
test facilities and stuff.  All I'm suggesting is an encapsulation of
this need in the form of an interface exported by gcc.  If problems
like this arise, it's because external products that already use
built-not-installed versions of gcc have the same problems *now*, or
they solve it somehow, in which case the solutions should be removed
from those products to the implementations of these new interfaces.

I suppose another possibility exists, which we should head off by
documenting that the etc/ interfaces are not perfect and should
not generally be relied upon for any new uses that aren't just copies
of existing situations (e.g. dejagnu, make check, etc.).

>The same problem (will) occur when the -V option
>is used.  You should at least wait till H.J.
>added his patches to rename these shared libraries
>to something that includes the version of the
>compiler I think.

I can't imagine why the two are related.  If -V is used when
invoking built compilers already, then either that would work
fine with my approach (it just gets passed in anyway) or it's
a bug, right?

But getting me to wait isn't a problem, since I'm not planning
on doing this right away, and probably am not the best person
to do it.

>Apart from that, it would have a logical `feel'
>if it was possible to use the -V option for this,
>ie:
>
>gcc -V 2.7.2.3 ...
>gcc -V 2.8.1 ...
>gcc -V 2.91.45 ...
>gcc -V built ...
>
>Or something similar, where `gcc' would be just
>/usr/bin/gcc (some (old) egcs version).

That doesn't really help unless you mean to extend -V to support
an arbitrary pathname (does it do that already??) to a built
tree instead of an installed tree, in which case it'd know how
to do the -B/-L/whatever thing anyway.

There still would be a need, I think, for the ability to export
a single *command* that reliably invokes the built version of gcc,
g77, g++, etc.  Because not all external systems can take both a
command and an option that must be specified immediately after that
command as a substitution for the default `cc' or `gcc'.

        tq vm, (burley)

Re: egcs testsuite & dejagnu : A special case?

[Carlo Wood]

| this need in the form of an interface exported by gcc.  If problems
| like this arise, it's because external products that already use
| built-not-installed versions of gcc have the same problems *now*, or
| they solve it somehow, in which case the solutions should be removed
| from those products to the implementations of these new interfaces.

The way dejagnu `solves' this is as follows:
It picks a foobar.cc file and collects all the gcc parameters it
needs to compile it (the -B, -L etc).  From that it determines
where the libstdc++ resides that apparently is wanted (from a
-L.*stdc++/).  It remembers this path in a variable, compiles the
program and then sets explicitly the LD_LIBRARY_PATH environment
variable to this path before executing the test program.
If you hide the -L paths from dejagnu, it won't be able anymore
to determine where the wanted libstdc++ is and thus it won't be
able to set the LD_LIBRARY_PATH environment variable.


-- 
 Carlo Wood  <carlo@runaway.xs4all.nl>

Re: egcs testsuite & dejagnu : A special case?

[Craig Burley]

>The way dejagnu `solves' this is as follows:
>It picks a foobar.cc file and collects all the gcc parameters it
>needs to compile it (the -B, -L etc).  From that it determines
>where the libstdc++ resides that apparently is wanted (from a
>-L.*stdc++/).  It remembers this path in a variable, compiles the
>program and then sets explicitly the LD_LIBRARY_PATH environment
>variable to this path before executing the test program.
>If you hide the -L paths from dejagnu, it won't be able anymore
>to determine where the wanted libstdc++ is and thus it won't be
>able to set the LD_LIBRARY_PATH environment variable.

Would not another driver option like --print-program-name=whatever
be more appropriate than this approach?  E.g. --print-library-path
or something?

This is yet another point in favor of a weird, old idea I've been
seeing other points in favor of lately.  Namely:

  gcc ("compilers" in general that preprocess/compile/link) should
  also be able to *run* programs!

That'd be pretty cool, and would make for an obvious place to
solve the dynamic-library-path problem.  And it hearkens back
to systems like TOPS-10, which I used to use.  Not sure what
other problems it'd bring, though.

In any case, it sure seems like there's a lot of investment in
the idea that dejagnu, and perhaps other packages, should have
intimate knowledge of gcc command-line options and internal
operations of the driver.  After seeing all the problems this
has caused over and over again for years, I'd have thought it was
a slam-dunk to convince everyone that taking the time and
trouble to define, publish, and implement a clean interface to
the revelant aspects of how gcc does its job would be worthwhile,
and would have saved a lot of trouble had it been done years ago.

But I guess I'm losing my enthusiasm for doing something that
would cause people concern.  Like significant tabs in makefiles,
perhaps we'll just learn to live with the problems that inevitably
result from design mistakes we're unwilling to correct because
of the "huge existing installed base".  :)

In case I haven't made myself clear: it is an outright design
*flaw* that gcc requires any *options* to invoke it properly,
regardless of where it lives.  That is, if you define "built
but not installed" is not invokable, then there's no bug,
except that any other package (or `make check') that tries
to invoke it there is, by definition, broken because it tries
to do so.

But if you say that -B, -L, etc. are required to invoke gcc
so it works as it was intended to (as built), then you're
talking about an outright design flaw.  A command is supposed
to be sufficient to specify the entire data base of whatever
the command needs to properly operate.  Environment variables
and options needed to effect that are simply work-arounds for
a buggy design.  And IMO that extends to dynamic libraries as
well.  So much time wasted because people are willing to live
with obviously bad designs, always resulting in lots of confusion,
subtle programming errors, and the like.  It's understandable
why Microsoft repeats the mistakes of 20 years ago when it
catches up on decades-old technology; I'm not so sure the rest
of us have any excuse for that.

        tq vm, (burley)

Re: egcs testsuite & dejagnu : A special case?

[Mike Stump]

> From: Carlo Wood <carlo@runaway.xs4all.nl>
> Date: Thu, 25 Jun 1998 17:27:34 +0200 (CEST)

> I investigated this new problem and found that it basically is the
> result of the fact that dejagnu is written to test other packages,
> other packages that need a compiler to be compiled, but that
> definitely are not the compiler itself.

Could you rephrase this?

If you meant to say that dejagnu is not written to test the compiler
itself, then I wonder why you say that, as it was.

> The problem, when needing the package itself to compile the package
> (testsuite executables), has not been addressed in dejagnu.

Could you rephrase this?

If you meant to say that dejagnu doesn't address the problem of
testing executables in egcs that are built with the compiler in egcs,
then I wonder why you say that, as it was.

> My feeling is that this is not really a problem of dejagnu: We can't
> expect them

That them is us.  dejagnu exists to test the compiler.  That is why it
was written, and that is it's main purpose in life.

> to hardcode support for all the compiler cases into dejagnu,

We all agree here.  Cleanliness says that we should use generic
interfaces with as little built in knowledge of the compiler specifics
as we can.  dejagnu however is not exactly a model of cleanliness.

> I am more then willing to write the small changes for egcs to make
> it work smoothly, but I'd like to hear a "go" from you before I do
> :).  These patches will be logical, but only if you accept the idea
> that testing a package that is a compiler is a special case.

If your changes make it work more often and don't introduce bugs, then
I'd support them.  The problem is the later one.  If you massively
rearchitect dejagnu and unless you test in many different ways on many
different targets, or are really good, there will be bugs.  What is
painful, is if you break testing that does work today.  This is why, I
think when it comes to dejagnu, I don't like to see a whole lot of
code motion in it, just a little bit every now and then to fix
specific problems in specific ways.

Re: egcs testsuite & dejagnu : A special case?

[Craig Burley]

>We all agree here.  Cleanliness says that we should use generic
>interfaces with as little built in knowledge of the compiler specifics
>as we can.  dejagnu however is not exactly a model of cleanliness.

This reminds me of a thought I had the other day while not on-line.

I've gotten the impression (not knowing much about how all this
testing stuff actually works at this point) that some of the problems
we've run into are due to attempting to be able to test an
*uninstalled* compiler as well as an installed one.  Is that true?

(An uninstalled compiler differs in ways including that the `gcc'
command is named `xgcc' to avoid conflicts when $PATH contains `.',
I assume; the default directory to search for the `specs' file is
the yet-to-be-installed-into directory instead of the same directory
the `xgcc' command is being run from; and other stuff that maybe
even the `-B' and `-L' don't fully ensure makes for identical
behavior running the uninstalled version exactly the same as
running it will be once it's installed.)

Certainly it's often the case that I want to run tests of my own
(using my own primitive test suite, or just try out a submitted
test case or something), and I've done a variety of things over
the years to cope with the fact that I want to try out a *built*,
but not *installed*, version of the compiler.

(On my old 486/33, I had a bunch of shells scripts installed in
/usr/local/bin that'd invoke the compiler in canonical places
in my home directory, using the appropriate -B and -L switches,
etc.  Now that I have a much faster system, I just specify
a unique --prefix and do "make install" each time it seems
appropriate, which takes only about a minute, but not everybody
has that.  And, I admit it's tedious to keeping typing full
pathnames like `/develop/g77/0.5.23/bin/g77' to specify which
version of the compiler I want to run.  That's a minor problem
compared to back when I couldn't really maintain more than two
versions, though!)

It occurred to me that, maybe, we could avoid much of this trouble
by, instead of working around the differences between installed
and uninstalled compilers *externally* (to the compiler itself,
that is), we could define a new interface to the compiler *itself*
that, when invoked, amounts to saying "I'm invoking the *uninstalled*
compiler", and leave all the mucking about with pathnames and such
to the new interface.

So, what I'm suggesting is that we teach the compiler *itself* (as
a complete package) to hide all the details of invoking an
uninstalled version of itself "consistently" within some kind
of published interface that everyone can use, safely.

For example, what if we had the build process automatically build,
but not install (obviously), shell scripts containing the appropriate
options and pathnames, wrapping around invocations of `xgcc', `g77',
`g++', and whatever else people normally invoke as commands in an
installed compiler.  E.g.:

  gcc-ui would be a shell script (command) built to automatically
    do `$(objdir)/xgcc -B$(objdir)/ -L$(objdir) $*', where $(objdir)
    expands to the absolute pathname of the build directory.

  g77-ui would be a similar script but invoke `$(objdir)/g77' instead.

  g++-ui ditto for g++.

And so on.

`-ui' just stands for `uninstalled', but maybe a better naming scheme
can be used.  Or maybe a distinct option, like `-funinstalled', would
be cleaner, though I somehow think that wouldn't fit cleanly into
how most external uses of gcc already operate (they might assume
that the variable meaning "command to invoke gcc", e.g. $(CC), is
really just a single command with no options).

Does this idea have any merit?  Offhand it seems fairly simple to
implement, and, once we change the pertinent tools (egcs top-level
Makefile such as when doing `make check') to use them, could really
reduce the maintenance headaches we seem to be having with trying
to do useful work on uninstalled compilers.

        tq vm, (burley)

Re: egcs testsuite & dejagnu : A special case?

[Jeffrey A Law]

  In message < 199806301344.JAA02773@melange.gnu.org >you write:
  > I've gotten the impression (not knowing much about how all this
  > testing stuff actually works at this point) that some of the problems
  > we've run into are due to attempting to be able to test an
  > *uninstalled* compiler as well as an installed one.  Is that true?
It's certainly a complication :-)  Though from where I sit testing the 
uninstalled compiler actually works quite well.

There's a couple quirks related to a bad choice we made for the runtime
library directory on native platforms ("libraries") that should go away
now that we use the target_alias as a directory name just like we do
for cross builds.

  > Certainly it's often the case that I want to run tests of my own
  > (using my own primitive test suite, or just try out a submitted
  > test case or something), and I've done a variety of things over
  > the years to cope with the fact that I want to try out a *built*,
  > but not *installed*, version of the compiler.
Likewise.  Though most of those scrips and hacks went away once dejagnu
got smart enough to test uninstalled tools.

  > It occurred to me that, maybe, we could avoid much of this trouble
  > by, instead of working around the differences between installed
  > and uninstalled compilers *externally* (to the compiler itself,
  > that is), we could define a new interface to the compiler *itself*
  > that, when invoked, amounts to saying "I'm invoking the *uninstalled*
  > compiler", and leave all the mucking about with pathnames and such
  > to the new interface.
An interesting thought.  Probably a good idea.

jeff

egcs testsuite & dejagnu : A special case?

[Carlo Wood]

Hey all, I love working in this team :).
Quite a change from the seven years I worked
on the undernet ircd project: Here there are
actually others that do something too! :)

But back to serious work:

----

My self-assigned task of checking how well
the egcs testsuite works with dejagnu has
lead to a the following conclusion on which
I'd like to get some feedback from you:

The latest dejagnu (980528) did address the
problem that was temporally fixed with the
lookfor_dir_with_trigger.patch (to be applied
to dejagnu-971222 only thus).

However, as already reported by H.J. Lu,
dejagnu-980528 is broken again (even with
the lookfor_dir_with_trigger.patch because
something else is broken now).

I investigated this new problem and found that
it basically is the result of the fact that
dejagnu is written to test other packages,
other packages that need a compiler to be
compiled, but that definitely are not the
compiler itself.

The problem, when needing the package itself
to compile the package (testsuite executables),
has not been addressed in dejagnu.

My feeling is that this is not really a problem
of dejagnu: We can't expect them to hardcode
support for all the compiler cases into dejagnu,
not unless there is a simple solution that makes
egcs take care of this "special handling".
For example, dejagnu sets $gccpath to the path
of the installed compiler.  I think it is the
responsibility of egcs to explicitly make sure
that it is set to the correct directory of the
package being tested (egcs); and not the
responsibility of dejagnu to detect it is now
testing a compiler package.

I am more then willing to write the small changes
for egcs to make it work smoothly, but I'd like
to hear a "go" from you before I do :).
These patches will be logical, but only if you
accept the idea that testing a package that is
a compiler is a special case.

Comments please?

Carlo Wood

carlo@runaway.xs4all.nl

Re: egcs testsuite & dejagnu : A special case?

[Manfred Hollstein]

On Thu, 25 June 1998, 17:27:34, carlo@runaway.xs4all.nl wrote:

 > Hey all, I love working in this team :).
 > Quite a change from the seven years I worked
 > on the undernet ircd project: Here there are
 > actually others that do something too! :)
 > 
 > But back to serious work:
 > 
 > ----
 > 
 > My self-assigned task of checking how well
 > the egcs testsuite works with dejagnu has
 > lead to a the following conclusion on which
 > I'd like to get some feedback from you:
 > 
 > The latest dejagnu (980528) did address the
 > problem that was temporally fixed with the
 > lookfor_dir_with_trigger.patch (to be applied
 > to dejagnu-971222 only thus).
 > 
 > However, as already reported by H.J. Lu,
 > dejagnu-980528 is broken again (even with
 > the lookfor_dir_with_trigger.patch because
 > something else is broken now).
 > 
 > I investigated this new problem and found that
 > it basically is the result of the fact that
 > dejagnu is written to test other packages,
 > other packages that need a compiler to be
 > compiled, but that definitely are not the
 > compiler itself.
 > 
 > The problem, when needing the package itself
 > to compile the package (testsuite executables),
 > has not been addressed in dejagnu.
 > 
 > My feeling is that this is not really a problem
 > of dejagnu: We can't expect them to hardcode
 > support for all the compiler cases into dejagnu,
 > not unless there is a simple solution that makes
 > egcs take care of this "special handling".
 > For example, dejagnu sets $gccpath to the path
 > of the installed compiler.  I think it is the
 > responsibility of egcs to explicitly make sure
 > that it is set to the correct directory of the
 > package being tested (egcs); and not the
 > responsibility of dejagnu to detect it is now
 > testing a compiler package.
 > 
 > I am more then willing to write the small changes
 > for egcs to make it work smoothly, but I'd like
 > to hear a "go" from you before I do :).
 > These patches will be logical, but only if you
 > accept the idea that testing a package that is
 > a compiler is a special case.
 > 
 > Comments please?

I'd really like the idea, to have _one_ version of dejagnu which can
be used (hopefully) on all platforms. Sometimes in the past I've seen
different test results from different people testing the same compiler 
snapshot. I've even recognized this myself on my Motorola platforms:
the same compiler snapshot produced different failure reports with
dejagnu-971028 and dejagnu-971222.

Yes, each package someone wants to check, should make provisions for
everything behaving different than other packages.

If we can get these testsuite related issues fixed, we'll be able to
compare results people are reporting on a much more accurate basis.

 > 
 > Carlo Wood
 > 
 > carlo@runaway.xs4all.nl
 > 

manfred