Re: Fortran testsuite?

Re: Fortran testsuite?

[Kate Hedstrom]

In case the g77-alpha people are wondering, it seems like time to
gather our code snippets and produce a Fortran testsuite.  Do you have
anything you would like to contribute?  I have saved some codes people
have posted, but no where near all of them.

>> "call exit(1)" vs. "stop 1"

> Well, the problem is some simulators don't give you an exit status
> from the program.  Thus we have technology to "wrap" the exit
> function to provide the information we need.

I'm using "call exit(1)" until we come to some consensus on this.  It
is true that it isn't portable - I remember using "call exit" which
worked on at least one or two systems that didn't want an argument.
Then it dumped core on the Sun, on which f77 wanted the argument.

> OK.  Then it sounds like we need to get started!  And you can
> push stuff from your testsuite over as time permits.
> 
> Probably the first tests should come from the egcs archives! We
> can branch out from there.

Is there some place I should put draft vesrions?  I have put what I
have so far on gate.gnu.ai.mit.edu in ~fortran/kate.  It unpacks to a
testsuite directory (get dejagnu, "runtest --tool g77").  I hardcoded
the path to /usr/local/bin/g77 and it doesn't get the right answer when
searching for "do" loops.  Naming of tests is quasi-random.  Also, I
get a fail on:

FAIL: g77.tests/execute/alpha1.f execution,  -O0 

I'm not sure if it should behave this way - Craig?
The summary is:

# of expected passes            103
# of unexpected failures        1

Also, this little gem is in my archives from the g77 group:

      CHARACTER*12 FUNC
      WRITE(*,*) FUNC()
      END
C
      CHARACTER*12 FUNCTION FUNC()
      WRITE(2,'(A)') 'HELLO, WORLD'
      FUNC = 'Goodbye, all'
      RETURN
      END

gecko% g77 bad.f
gecko% a.out
I/O recursion: I/O started while already doing I/O
apparent state: unit 6 (unnamed)
last format: list io
lately writing direct formatted external IO
Abort (core dumped)

The IBM xlf produces the same result.

Kate

Re: Fortran testsuite?

[Craig Burley]

>> Well, the problem is some simulators don't give you an exit status
>> from the program.  Thus we have technology to "wrap" the exit
>> function to provide the information we need.
>
>I'm using "call exit(1)" until we come to some consensus on this.  It
>is true that it isn't portable - I remember using "call exit" which
>worked on at least one or two systems that didn't want an argument.
>Then it dumped core on the Sun, on which f77 wanted the argument.

Yup.  The problem with "CALL EXIT(n)" is that it isn't portable and
certainly not universal, but of course we can *make* it so for all
*g77* targets, thus dumping any actual problem on vendors wishing
to use our test suite, whose compilers do not support it.  :)

And, a problem with "STOP n" is that `n' must be a decimal
constant, and that, currently, and probably forever if I'm
guessing right about what Fortran users want, it is implemented
as "print a message, then stop with exit status 0 [success]".

So, even if we decided to change "STOP n" to mean "exit with status
code n", we'd have the problem of n having to be a constant to be
portable anyway -- and different systems have different values for
non-success.

BTW, here's a summary of g77/libf2c braking strategy, assuming
NO_ONEXIT isn't #define'd while building libf2c (and it shouldn't
be for most systems):

main(): Install f_exit() as the routine for exit() to call (via atexit()
  or whatever).  [gcc/f/runtime/libF77/main.c]
f_exit(): Close all open units via f_clos().  [gcc/f/runtime/libI77/close.c]
f_clos(): Close a unit, truncating/deleting file if necessary.  [ditto]
exit_(): Call exit() with status.  Invoked via CALL EXIT(n) by Fortran
  code, though g77's libf2c massages the name for reasons not pertinent
  to this summary, as the same effect is achieved.
  [gcc/f/runtime/libF77/exit.c]
s_stop(): If message not 0 characters long, print "STOP `msg' statement
  executed".  Then call exit(0).  Invoked via STOP.
  [gcc/f/runtime/libF77/s_stop.c]

(If NO_ONEXIT is #define'd, any code in libf2c that calls exit() first
calls f_exit().  So, the same effect is achieved, except any *other*
paths the program has to exit() won't call f_exit() -- e.g. if a library
or C code or something calls exit(), open units won't get closed,
which could mean things like partially overwritten files don't
get truncated.)

>FAIL: g77.tests/execute/alpha1.f execution,  -O0 
>
>I'm not sure if it should behave this way - Craig?

I'm not sure what that is -- I haven't downloaded any egcs stuff yet,
and probably won't get to it for another couple of weeks or so, due
to working on this new project pretty intensely.  (Which I should
really be doing now, as something *was* literally due yesterday.  :)

>Also, this little gem is in my archives from the g77 group:
>
>      CHARACTER*12 FUNC
>      WRITE(*,*) FUNC()
>      END
>C
>      CHARACTER*12 FUNCTION FUNC()
>      WRITE(2,'(A)') 'HELLO, WORLD'
>      FUNC = 'Goodbye, all'
>      RETURN
>      END
>
>gecko% g77 bad.f
>gecko% a.out
>I/O recursion: I/O started while already doing I/O
>apparent state: unit 6 (unnamed)
>last format: list io
>lately writing direct formatted external IO
>Abort (core dumped)
>
>The IBM xlf produces the same result.

Yup, that's exactly right.  Someday we'll (or dmg, libf2c maintainer,
will) probably teach libI77 to cope with recursive I/O on different
I/O units, but at least we give a nicer diagnostic in the 0.5.21
version of g77 than some obtuse crash, or worse.

        tq vm, (burley)

Re: Fortran testsuite?

[Jeffrey A Law]

  In message < 199709170143.VAA00338@ahab.rutgers.edu >you write:
  > In case the g77-alpha people are wondering, it seems like time to
  > gather our code snippets and produce a Fortran testsuite.
Sounds great.

  > Do you have
  > anything you would like to contribute?  I have saved some codes people
  > have posted, but no where near all of them.
I would encourage folks to send mail to Kate when the report bugs which
includes something like

Subject: New Fortran test


So that she can quickly identify new tests for the testsuite.  That is
how things are (supposed) to work with tege & c-torture.

  > I'm using "call exit(1)" until we come to some consensus on this.  It
  > is true that it isn't portable - I remember using "call exit" which
  > worked on at least one or two systems that didn't want an argument.
  > Then it dumped core on the Sun, on which f77 wanted the argument.
Sounds reasonable.

  > Is there some place I should put draft vesrions?  I have put what I
  > have so far on gate.gnu.ai.mit.edu in ~fortran/kate.
Sounds good for now.

  > It unpacks to a
  > testsuite directory (get dejagnu, "runtest --tool g77").  I hardcoded
  > the path to /usr/local/bin/g77 and it doesn't get the right answer when
  > searching for "do" loops.
It's still good 'nuff for us to start hacking on.  I can probably help
with the stuff like having the test harness find g77 and the like.

I don't know enough fortran to provide regexps to find loops in the
code.  If it's not easy 'nuff, then we can just have the loop unrolling
tests run regardless of what the source looks like.

I'm not sure if we should be working with something like f-torture.exp or
trying to make the fortran tests use c-torture.exp.  I'm tempted to go
with the first until we have a better idea how to break out any language
dependent stuff from c-torture.exp.


jeff

Re: Fortran testsuite?

[Jeffrey A Law]

  In message <199709170143.VAA00338@ahab.rutgers.edu>you write:
  > Is there some place I should put draft vesrions?  I have put what I
  > have so far on gate.gnu.ai.mit.edu in ~fortran/kate.  It unpacks to a
  > testsuite directory (get dejagnu, "runtest --tool g77").  I hardcoded
  > the path to /usr/local/bin/g77 and it doesn't get the right answer when
  > searching for "do" loops.  Naming of tests is quasi-random.  Also, I
  > get a fail on:
Well, it's a start.

The one question I currently have is what happens for an execute test
that looks like this:

      parameter (nmax=165000)
      double precision x(nmax)
      end

Does that have a well defined exit status?

Any objection to calling the directory something like g77.f-torture since
it's obviously based on c-torture :-)

Other g77 tests that don't use the same framework could go into a directory
like g77.foobar.

I'm going to play with this some more after I get back from vacation
(or maybe a litte tomorrow if I have time).   Seems like the biggest
issue is getting a "find_g77" proc into libgloss.  That should be
pretty simple.  Then we can start adding more tests!

Jeff

Re: Fortran testsuite?

[Thomas Koenig]

Jeffrey A Law wrote:

>      parameter (nmax=165000)
>      double precision x(nmax)
>      end
>
>Does that have a well defined exit status?

Yes, it has exit status 0.
-- 
Thomas Koenig, Thomas.Koenig@ciw.uni-karlsruhe.de, ig25@dkauni2.bitnet.
The joy of engineering is to find a straight line on a double
logarithmic diagram.

Re: Fortran testsuite?

[Jeffrey A Law]

  In message < 199710020026.UAA04477@nemo.rutgers.edu >you write:
  > > I'm going to play with this some more after I get back from vacation
  > > (or maybe a litte tomorrow if I have time).   Seems like the biggest
  > > issue is getting a "find_g77" proc into libgloss.  That should be
  > > pretty simple.  Then we can start adding more tests!
  > 
  > I've added a few tests to my home version, but haven't changed anything
  > else.  I'll update what's in ~fortran this weekend.
Cool.  I can't get to the FSF machines right now; I assume this is
due to the recent reorg of their network.

I'm going to go ahead and import the current g77 testsuite into
egcs.  When I can get to the FSF machines I'll pick up any new stuff

jeff

Re: Fortran testsuite?

[Kate Hedstrom]

> From law@hurl.cygnus.com Wed Oct  1 04:16 EDT 1997
> To: Kate Hedstrom <kate@ahab.rutgers.edu>
> cc: burley@gnu.ai.mit.edu, egcs@cygnus.com, g77-alpha@gnu.ai.mit.edu
> Subject: Re: Fortran testsuite? 
> Reply-To: law@cygnus.com
> Content-Type: text
> Content-Length: 1153
> 
> 
> Any objection to calling the directory something like g77.f-torture since
> it's obviously based on c-torture :-)

None at all.

> I'm going to play with this some more after I get back from vacation
> (or maybe a litte tomorrow if I have time).   Seems like the biggest
> issue is getting a "find_g77" proc into libgloss.  That should be
> pretty simple.  Then we can start adding more tests!

I've added a few tests to my home version, but haven't changed anything
else.  I'll update what's in ~fortran this weekend.

Kate

Re: Fortran testsuite?

[Jeffrey A Law]

  In message < 199709150711.DAA18207@churchy.gnu.ai.mit.edu >you write:
  > Yes, the main problem being that I can't just dump what I have to
  > somebody else without going through it first...and I don't have time
  > to do that right now.  But if something is put together, I can
  > probably find time to add dribs and drabs of tests as I find they're
  > okay, down the road.
OK.  That sounds reasonable.


  > >Most of the time fortran is only used on native systems, and you can
  > >probably assume that you'll have suitable libraries since they're
  > >build at the same time as g77.
  > 
  > Note that g77 implements STOP as a call to the library anyway.  Though,
  > this is to a different function -- f_exit() instead of exit_(), or
  > some such thing.
Though f_exit eventually calls exit doesn't it?  So long as there's
a way to tell the difference between success and failure STOP is
fine.


  > I would think this would work for STOP anyway, but generally I'd
  > prefer nonzero return codes whenever it's just as easy to do it
  > that way.  OTOH, a "standard Fortran test suite" would use "STOP n"
  > throughout, I guess, not "CALL EXIT(n)".  Hmm.
Well, the problem is some simulators don't give you an exit status
from the program.  Thus we have technology to "wrap" the exit
function to provide the information we need.

  > Kate's already talked to me.  I think it's great if she and others
  > work on putting together an infrastructure and add whatever tests
  > they have handy and know are fine, copying-wise.
OK.  Then it sounds like we need to get started!  And you can
push stuff from your testsuite over as time permits.

Probably the first tests should come from the egcs archives! We
can branch out from there.

jeff

Re: Fortran testsuite?

[Craig Burley]

>I think Craig Burley has something along these lines, except that it
>may contain code that shouldn't be released to the public.

Yes, the main problem being that I can't just dump what I have to
somebody else without going through it first...and I don't have time
to do that right now.  But if something is put together, I can
probably find time to add dribs and drabs of tests as I find they're
okay, down the road.

>If my understanding is correct, someone would need to pour over the
>code & bug reports and extract just those testcase which are suitable
>for a public audience.
>
>I'm sure Craig will correct me if I'm wrong.

Yes, though I'd rather they "pore" over the tests than "pour" over them. :)

>Most of the time fortran is only used on native systems, and you can
>probably assume that you'll have suitable libraries since they're
>build at the same time as g77.

Note that g77 implements STOP as a call to the library anyway.  Though,
this is to a different function -- f_exit() instead of exit_(), or
some such thing.

>The area where you run into trouble is simulator and target board testing.
>Luckily we have some pretty neat features in ld & the testing harnes which
>will allows us to redefine exit to print a pass/fail message (or whatever we
>want) and the testing harness can look for those messages.

I would think this would work for STOP anyway, but generally I'd
prefer nonzero return codes whenever it's just as easy to do it
that way.  OTOH, a "standard Fortran test suite" would use "STOP n"
throughout, I guess, not "CALL EXIT(n)".  Hmm.

>Craig is probably the man to talk to about putting together a fortan
>testsuite.  It's something I'd certainly like to see us start working on.

Kate's already talked to me.  I think it's great if she and others
work on putting together an infrastructure and add whatever tests
they have handy and know are fine, copying-wise.  E.g. there's lots
of PD stuff on the net I don't know much about.  So, feel free to
help her out on the g77 front, and don't worry about (and especially
don't wait for) my input on this -- I'll help out when I can, but
am too busy (and, anyway, not really knowledgable enough about
Fortran testing).

        tq vm, (burley)

Re: Fortran testsuite?

[Jeffrey A Law]

  In message < 199709140207.WAA28547@nemo.rutgers.edu >you write:
  > It seems to me that one thing that is missing in egcs is a Fortran
  > testsuite.  Is anyone working on one?  I looked a little at the 
  > way it is done for gcc and g++ and I have several questions about
  > it:
I think Craig Burley has something along these lines, except that it
may contain code that shouldn't be released to the public.

If my understanding is correct, someone would need to pour over the
code & bug reports and extract just those testcase which are suitable
for a public audience.

I'm sure Craig will correct me if I'm wrong.

  > 1. It seems like the gcc compile/execute split is a good way to go
  > for Fortran.  Right?  Will we need other directories for other
  > compiler flags (like one for free format code)?
It's a reasonable start.  There's other stuff that's more complicated;
for example one could use specfp92 (all the programs are publicly
avaiable, you just need reference inputs and outputs).  Similarly for
other benchmarking code which contains sample input and output.


  > 2. Here is an example of something that once failed in g77:
  > 
  >       program complex_1
  >       complex      z0, z1, z2
  > 
  >       z0 = cmplx(0.,.5)
  >       z1 = 1./z0
  >       if (z1 .ne. cmplx(0.,-2)) stop 1
  > 
  >       z0 = 10.*z0
  >       if (z0 .ne. cmplx(0.,5.)) stop 2
  > 
  >       z2 = cmplx(1.,2.)
  >       z1 = z0/z2
  >       if (z1 .ne. cmplx(2.,1.)) stop 3
  > 
  >       z1 = z0*z2
  >       if (z1 .ne. cmplx(-10.,5.)) stop 4
  >       end
  > 
  > If it fails, it will print "STOP 1 statement reached", but the return
  > code to the OS is zero.  An alternate is to replace "stop 1" with
  > "call exit(1)" but this uses the library and will fail if the library
  > isn't built right.  Any comment?
Well, my tempation would be to go ahead and call exit (1).

Most of the time fortran is only used on native systems, and you can
probably assume that you'll have suitable libraries since they're
build at the same time as g77.

The area where you run into trouble is simulator and target board testing.
Luckily we have some pretty neat features in ld & the testing harnes which
will allows us to redefine exit to print a pass/fail message (or whatever we
want) and the testing harness can look for those messages.

  > 3. I copied lib/gcc.exp to lib/g77.exp and changed gcc to g77, but
  > libgloss doesn't have a find_g77 in it.  I also looked at g++.exp for
  > how to find the compiler, but I'm still stuck.
I'd avoid g++.exp since it works in an entirely different fashion.
gcc/testsuite/lib/c-torture.exp is probably the thing you'd want to look at.

  > I'm willing to put some time into this, but I'll need some guidance in
  > order to make it fit in with the existing structure.  I'm rather a
  > novice at tcl/expect.
Craig is probably the man to talk to about putting together a fortan
testsuite.  It's something I'd certainly like to see us start working on.

jeff

Fortran testsuite?

[Kate Hedstrom]

It seems to me that one thing that is missing in egcs is a Fortran
testsuite.  Is anyone working on one?  I looked a little at the 
way it is done for gcc and g++ and I have several questions about
it:

1. It seems like the gcc compile/execute split is a good way to go
for Fortran.  Right?  Will we need other directories for other
compiler flags (like one for free format code)?

2. Here is an example of something that once failed in g77:

      program complex_1
      complex      z0, z1, z2

      z0 = cmplx(0.,.5)
      z1 = 1./z0
      if (z1 .ne. cmplx(0.,-2)) stop 1

      z0 = 10.*z0
      if (z0 .ne. cmplx(0.,5.)) stop 2

      z2 = cmplx(1.,2.)
      z1 = z0/z2
      if (z1 .ne. cmplx(2.,1.)) stop 3

      z1 = z0*z2
      if (z1 .ne. cmplx(-10.,5.)) stop 4
      end

If it fails, it will print "STOP 1 statement reached", but the return
code to the OS is zero.  An alternate is to replace "stop 1" with
"call exit(1)" but this uses the library and will fail if the library
isn't built right.  Any comment?

3. I copied lib/gcc.exp to lib/g77.exp and changed gcc to g77, but
libgloss doesn't have a find_g77 in it.  I also looked at g++.exp for
how to find the compiler, but I'm still stuck.

I'm willing to put some time into this, but I'll need some guidance in
order to make it fit in with the existing structure.  I'm rather a
novice at tcl/expect.

Kate