Re: dg-extract-results broken since rev 268511, was Re: Status of 9.0.1 20190415 [trunk revision 270358] on x86_64-w64-mingw32

[Christophe Lyon <christophe.lyon@linaro.org>]

On Tue, 16 Apr 2019 at 17:36, Jakub Jelinek <jakub@redhat.com> wrote:
>
> On Tue, Apr 16, 2019 at 03:44:44PM +0200, Jakub Jelinek wrote:
> > I can't reproduce this on my Fedora 29 x86_64-linux bootstrap box though,
> > the *.log files are complete there.
> >
> > And I have no idea if it was introduced with your change or earlier.
>
> Actually, I managed to reproduce in a Fedora 31 chroot, in which I don't
> have /usr/bin/python installed (I think in Fedora 30+ there is
> /usr/bin/python2 and /usr/bin/python3 but not /usr/bin/python, at least not
> in the default buildroot).
>
> The changes to contrib/dg-extract-results.sh look wrong to me:
> --- contrib/dg-extract-results.sh       2018-04-25 09:40:40.139659386 +0200
> +++ contrib/dg-extract-results.sh       2019-03-05 21:49:34.471573434 +0100
> @@ -298,6 +298,8 @@ BEGIN {
>    cnt=0
>    print_using=0
>    need_close=0
> +  has_timeout=0
> +  timeout_cnt=0
>  }
>  /^EXPFILE: / {
>    expfiles[expfileno] = \$2
> @@ -329,16 +331,37 @@ BEGIN {
>    # Ugly hack for gfortran.dg/dg.exp
>    if ("$TOOL" == "gfortran" && testname ~ /^gfortran.dg\/g77\//)
>      testname="h"testname
> +  if (\$1 == "WARNING:" && \$2 == "program" && \$3 == "timed" && (\$4 == "out" || \$4 == "out.")) {
> +        has_timeout=1
> +        timeout_cnt=cnt
> +  } else {
> +  # Prepare timeout replacement message in case it's needed
> +    timeout_msg=\$0
> +    sub(\$1, "WARNING:", timeout_msg)
> +  }
>  }
>  /^$/ { if ("$MODE" == "sum") next }
>  { if (variant == curvar && curfile) {
>      if ("$MODE" == "sum") {
> -      printf "%s %08d|", testname, cnt >> curfile
> -      cnt = cnt + 1
> +      # Do not print anything if the current line is a timeout
> +      if (has_timeout == 0) {
> +        # If the previous line was a timeout,
> +        # insert the full current message without keyword
> +        if (timeout_cnt != 0) {
> +          printf "%s %08d|%s program timed out.\n", testname, timeout_cnt, timeout_msg >> curfile
> +          timeout_cnt = 0
> +          cnt = cnt + 1
> +        }
> +        printf "%s %08d|", testname, cnt >> curfile
> +        cnt = cnt + 1
> +        filewritten[curfile]=1
> +        need_close=1
> +        if (timeout_cnt == 0)
> +          print >> curfile
> +      }
> +
> +      has_timeout=0
>      }
> -    filewritten[curfile]=1
> -    need_close=1
> -    print >> curfile
>    } else
>      next
>  }
> First of all, I don't see why the WARNING: program timed out
> stuff should be handled in any way specially in -L mode, there is no sorting
> at all and all the lines go together.  But more importantly, the above

The "WARNING: program timed out" stuff needs to be handled specially
in non-L mode (when handling .sum), because in that case we are using
"sort", which used to put all "WARNING:" lines together before most of
the report.

> changes broke completely the -L mode, previously the filewritten, need_close
> and print lines were done for both sum and log modes, but now they are done
> only in the sum mode (and in that case only if has_timeout is 0, which is
> desirable).
>
I did check my patch against .sum and .log files, but it looks like my
tests were incomplete, sorry for that.

> I believe the following patch should fix it, but I don't actually have any
> WARNING: program timed out
> lines in my *.sep files in any of the last 12 bootstraps I have around.

You can just insert one such line in your .sum/.log manually, and
possibly replace a PASS with a FAIL to check that the WARNING and FAIL
are kept next to each other in the .sum (that was my original
intention).

Christophe

>
> Additionally, perhaps we should change dg-extract-results.sh, so that it
> doesn't try just python, but also python3?  I think in some distros
> /usr/bin/python even warns users that they should decide if they mean
> python2 or python3.
>
> 2019-04-16  Jakub Jelinek  <jakub@redhat.com>
>
>         * dg-extract-results.sh: Only handle WARNING: program timed out
>         lines specially in "$MODE" == "sum".  Restore previous behavior
>         for "$MODE" != "sum".  Clear has_timeout and timeout_cnt if in
>         a different variant or curfile is empty.
>         * dg-extract-results.py: Fix a typo.
>
> --- contrib/dg-extract-results.sh.jj    2019-03-05 21:49:34.471573434 +0100
> +++ contrib/dg-extract-results.sh       2019-04-16 17:09:02.710004553 +0200
> @@ -331,13 +331,15 @@ BEGIN {
>    # Ugly hack for gfortran.dg/dg.exp
>    if ("$TOOL" == "gfortran" && testname ~ /^gfortran.dg\/g77\//)
>      testname="h"testname
> -  if (\$1 == "WARNING:" && \$2 == "program" && \$3 == "timed" && (\$4 == "out" || \$4 == "out.")) {
> -        has_timeout=1
> -        timeout_cnt=cnt
> -  } else {
> -  # Prepare timeout replacement message in case it's needed
> -    timeout_msg=\$0
> -    sub(\$1, "WARNING:", timeout_msg)
> +  if ("$MODE" == "sum") {
> +    if (\$0 ^ /^WARNING: program timed out/) {
> +      has_timeout=1
> +      timeout_cnt=cnt
> +    } else {
> +      # Prepare timeout replacement message in case it's needed
> +      timeout_msg=\$0
> +      sub(\$1, "WARNING:", timeout_msg)
> +    }
>    }
>  }
>  /^$/ { if ("$MODE" == "sum") next }
> @@ -345,25 +347,30 @@ BEGIN {
>      if ("$MODE" == "sum") {
>        # Do not print anything if the current line is a timeout
>        if (has_timeout == 0) {
> -        # If the previous line was a timeout,
> -        # insert the full current message without keyword
> -        if (timeout_cnt != 0) {
> -          printf "%s %08d|%s program timed out.\n", testname, timeout_cnt, timeout_msg >> curfile
> -          timeout_cnt = 0
> -          cnt = cnt + 1
> -        }
> -        printf "%s %08d|", testname, cnt >> curfile
> -        cnt = cnt + 1
> -        filewritten[curfile]=1
> -        need_close=1
> -        if (timeout_cnt == 0)
> -          print >> curfile
> +       # If the previous line was a timeout,
> +       # insert the full current message without keyword
> +       if (timeout_cnt != 0) {
> +         printf "%s %08d|%s program timed out.\n", testname, timeout_cnt, timeout_msg >> curfile
> +         timeout_cnt = 0
> +         cnt = cnt + 1
> +       }
> +       printf "%s %08d|", testname, cnt >> curfile
> +       cnt = cnt + 1
> +       filewritten[curfile]=1
> +       need_close=1
> +       print >> curfile
>        }
> -
>        has_timeout=0
> +    } else {
> +      filewritten[curfile]=1
> +      need_close=1
> +      print >> curfile
>      }
> -  } else
> +  } else {
> +    has_timeout=0
> +    timeout_cnt=0
>      next
> +  }
>  }
>  END {
>    n=1
> --- contrib/dg-extract-results.py.jj    2019-03-05 21:49:34.471573434 +0100
> +++ contrib/dg-extract-results.py       2019-04-16 17:14:54.447248209 +0200
> @@ -296,7 +296,7 @@ class Prog:
>                  # If we have a time out warning, make sure it appears
>                  # before the following testcase diagnostic: we insert
>                  # the testname before 'program' so that sort faces a
> -                # list of testhanes.
> +                # list of testnames.
>                  if line.startswith ('WARNING: program timed out'):
>                    has_warning = 1
>                  else:
>
>
>         Jakub