Re: Will the consistent failures in EGCS be fixed soon?

Re: Will the consistent failures in EGCS be fixed soon?

[Mike Stump]

> Date: Thu, 25 Jun 1998 00:16:14 +0200
> From: Martin von Loewis <martin@mira.isdn.cs.tu-berlin.de>

> > Does anyone know if these failures will be fixed soon?  

> I suggest that you look closely at the individual failures and decide
> for yourself what kind of problem they constitute.

xfail was invented to solve this problem.  Your suggestion I feel is a
step backwards from just using the technology that already exists.

Re: Will the consistent failures in EGCS be fixed soon?

[Jeffrey A Law]

  In message < 199806292135.OAA10101@kankakee.wrs.com >you write:
  > Yes, they should be.  As Jeff pointed out, they don't have the
  > infrastructure to mark them precisely.  I think the should be marked
  > as precisely as possible.  Or put another way, we should xfail the
  > entire series of sets and have a few extra XPASSes that we can't get
  > rid of, then a few unexpected fails that we can't get ride of.
Or we just fix the bugs.  I'd be game for xfailing anything from before
1998 (ie the loop-* and some of the varargs/stdarg cases).  I'd
prefer to hold off on newer tests since we do have folks working on
fixing them.

jeff

Re: Will the consistent failures in EGCS be fixed soon?

[Mike Stump]

> Date: Thu, 25 Jun 1998 12:53:45 +0100
> From: Richard Earnshaw <rearnsha@arm.com>

> > The loop* things aren't likely to be addressed anytime soon.

> So shouldn't these be marked as XFAILs?  Then I won't waste time looking 
> into the failure in case it is a port problem.

Yes, they should be.  As Jeff pointed out, they don't have the
infrastructure to mark them precisely.  I think the should be marked
as precisely as possible.  Or put another way, we should xfail the
entire series of sets and have a few extra XPASSes that we can't get
rid of, then a few unexpected fails that we can't get ride of.

The reason is, then a random person can look at the results, and just
know...  Jeff, can we do this?

Re: Will the consistent failures in EGCS be fixed soon?

[Richard Earnshaw]

> | Ok, I'll buy that; so how about another document describing known 
> | failures, and the part of the compiler responsible -- this might help to 
> | save hours of wasted work trying to track down the cause of problems that 
> | have already been investigated but not yet fixed.
> | 
> | Richard
> 
> This `database' could grow pretty large.
> I imagine that a sort of E-mail thread on each test case, kept
> on the website would be favourable above including it in the
> tar.gz.
> 

Not necessarily.  Broken tests get fixed, in which case the need to 
document the cause of the ex-failure goes away.  Working tests don't need 
any documentation.  Of course, the test may fail again at some later date, 
but that will need investigation in its own right; I'd guess the chance 
that the same reason would be the cause of a future failure (assuming the 
problem was fixed properly in the first place) would be no more than 50/50.


Richard.

Re: Will the consistent failures in EGCS be fixed soon?

[Carlo Wood]

| Ok, I'll buy that; so how about another document describing known 
| failures, and the part of the compiler responsible -- this might help to 
| save hours of wasted work trying to track down the cause of problems that 
| have already been investigated but not yet fixed.
| 
| Richard

This `database' could grow pretty large.
I imagine that a sort of E-mail thread on each test case, kept
on the website would be favourable above including it in the
tar.gz.

It would be great if some automated interface could be created
that puts mails posted here, with a special trigger in the subject,
into that web database.  For instance Subject: TESTCOMMENT 920520-1.c

Carlo Wood

Re: Will the consistent failures in EGCS be fixed soon?

[Jeffrey A Law]

  In message < 199806251552.QAA14951@sun52.NIS.cambridge >you write:
  > Ok, I'll buy that; so how about another document describing known 
  > failures, and the part of the compiler responsible -- this might help to 
  > save hours of wasted work trying to track down the cause of problems that 
  > have already been investigated but not yet fixed.
This sounds more than reasonable.  Put it on the web and let folks
check in changes as bugs are fixed or new testcases introduced.

In fact, this is one of the pieces of a larger project that we need
to tackle one day -- basicaly we need to find a way to describe how
to interpret test results for non-developers.

jeff

Re: Will the consistent failures in EGCS be fixed soon?

[Richard Earnshaw]

>   > 
>   > So shouldn't these be marked as XFAILs?  Then I won't waste time looking 
>   > into the failure in case it is a port problem.
> It would be nice, but we don't have the capability to xfail a test
> based on certain optimization levels or whatever multilib is in
> effect.
> 
> It's not at all uncommon for those tests to only fail for one or
> two particular optimization levels.
> 

Ok, I'll buy that; so how about another document describing known 
failures, and the part of the compiler responsible -- this might help to 
save hours of wasted work trying to track down the cause of problems that 
have already been investigated but not yet fixed.

Richard

Re: Will the consistent failures in EGCS be fixed soon?

[Jeffrey A Law]

  In message < 199806251153.MAA14147@sun52.NIS.cambridge >you write:
  > > The loop* things aren't likely to be addressed anytime soon.  They
  > > are horribly obscure problems that nobody is likely to ever trigger
  > > unless they go look at the gcc source code and specifically write
  > > code to trigger a specific problem in the loop optimizer.
  > > 
  > > There's also an alias bug in the testsuite that we aren't actively
  > > working on, but we do want to keep the test because we do want to
  > > fix the problem at some point.
  > > 
  > 
  > So shouldn't these be marked as XFAILs?  Then I won't waste time looking 
  > into the failure in case it is a port problem.
It would be nice, but we don't have the capability to xfail a test
based on certain optimization levels or whatever multilib is in
effect.

It's not at all uncommon for those tests to only fail for one or
two particular optimization levels.

The best solution is to fix the compiler :-) :-)

jeffa

Re: Will the consistent failures in EGCS be fixed soon?

[Richard Earnshaw]

> The loop* things aren't likely to be addressed anytime soon.  They
> are horribly obscure problems that nobody is likely to ever trigger
> unless they go look at the gcc source code and specifically write
> code to trigger a specific problem in the loop optimizer.
> 
> There's also an alias bug in the testsuite that we aren't actively
> working on, but we do want to keep the test because we do want to
> fix the problem at some point.
> 

So shouldn't these be marked as XFAILs?  Then I won't waste time looking 
into the failure in case it is a port problem.

Richard.

Re: Will the consistent failures in EGCS be fixed soon?

[Jeffrey A Law]

  > Hi, for about a month now (at least), I haven't been using egcs because it
  > looks rather broken according to the 'make check' results that are being
  > posted.  The 'eb' failures in g++ and the 'execute/loop-2f and
  > execute/9805xx' failures in gcc seem to have been around for a long time
  > now, though there aren't as many as there were before.
We are addressing those which are important.  It can take a while though
since everyone involved is basically volunteers and some of the problems
are rather tricky.

The loop* things aren't likely to be addressed anytime soon.  They
are horribly obscure problems that nobody is likely to ever trigger
unless they go look at the gcc source code and specifically write
code to trigger a specific problem in the loop optimizer.

There's also an alias bug in the testsuite that we aren't actively
working on, but we do want to keep the test because we do want to
fix the problem at some point.

My general feeling is the newer the test, the less I worry about
FAILs :-)  
jeff

Re: Will the consistent failures in EGCS be fixed soon?

[Benjamin Redelings I]

Thanks for all the replies on this subject.  I'll begin using snapshots
again soon :)

-benRI

Re: Will the consistent failures in EGCS be fixed soon?

[Martin von Loewis]

[ suddenly many new failures in eb*, indicating some deeply broken compiler ]
> Is that not true?  

Yes, not true.

> Or perhaps all the failures are just new tests?  

Perhaps :-) If you've been watching closely, or if you had compared
the two subsequent snapshots where the error first appeared, or if you
had used 'cvs log', you'd know that the g++.robertl/eb* test cases are
indeed new to the testsuite, introduced at the same time by going over
the reports of egcs-bugs and selecting 'good' bug reports.

> Does anyone know if these failures will be fixed soon?  

I suggest that you look closely at the individual failures and decide
for yourself what kind of problem they constitute. Please use the
gcc.log and g++.log files produced to see the exact messages produced
when running the testsuite.

For example, bugs in the C++ front end most likely do not harm
compilation of the Linux kernel or glibc.

Regards,
Martin

Re: Will the consistent failures in EGCS be fixed soon?

[Horst von Brand]

Benjamin Redelings I <bredelin@ucsd.edu> said:

[...]

> 	Does anyone know if these failures will be fixed soon?  Or perhaps
> you can mention some programs (linux kernel, glibc,libg++) that compile
> OK with the current egcs, implying that its OK...

I've been compiling linux snapshots (i586 UP, run of the mill clone) with
each egcs snapshot and -O2 -fomit-frame-pointer -march=pentium for quite
some time now... and I _do_ complain loudly when something breaks. In fact,
egcs snapshots is almost all the compiler I use for C. Have very little use
for C++ or FORTRAN, but build and check them anyway, just for kicks.
-- 
Dr. Horst H. von Brand                       mailto:vonbrand@inf.utfsm.cl
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Will the consistent failures in EGCS be fixed soon?

[Benjamin Redelings I]

Hi, for about a month now (at least), I haven't been using egcs because it
looks rather broken according to the 'make check' results that are being
posted.  The 'eb' failures in g++ and the 'execute/loop-2f and
execute/9805xx' failures in gcc seem to have been around for a long time
now, though there aren't as many as there were before.
	As these all seem to have been broken at the same time, I've just assumed
that there are a few breaks from e.g. the gcc merge, that haven't gotten
fixed.  Also, some of the failures seem to flicker in and out of existance,
making me suspect that there is a deeper problem that isn't being
addressed....
	Is that not true?  Or perhaps all the failures are just new tests?  Or
perhaps they wouldn't make a difference in most programs?  H.J. has
mentioned some patches (e.g. to combine.c) that at least make egcs work on
compiling linux kernels...... 
	Does anyone know if these failures will be fixed soon?  Or perhaps you can
mention some programs (linux kernel, glibc,libg++) that compile OK with the
current egcs, implying that its OK...

Thanks,
-Ben "Want to use egcs snapshots again" Redelings