[Bug testsuite/18067] New: many failures in testsuite

[Bug testsuite/18067] New: many failures in testsuite

[yuri at tsoft dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18067

            Bug ID: 18067
           Summary: many failures in testsuite
           Product: gdb
           Version: 7.8
            Status: NEW
          Severity: normal
          Priority: P2
         Component: testsuite
          Assignee: unassigned at sourceware dot org
          Reporter: yuri at tsoft dot com

Created attachment 8158
  --> https://sourceware.org/bugzilla/attachment.cgi?id=8158&action=edit
check log

I ran it for 7.8.2 on Arch linux, and results aren't assuring gdb is ok.

Problematic results:
# of unexpected failures        616
# of unexpected successes       1

It appears gdb.threads, gdb.python are broken, among others.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/18067] many failures in testsuite

[keiths at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18067

Keith Seitz <keiths at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |keiths at redhat dot com

--- Comment #1 from Keith Seitz <keiths at redhat dot com> ---
The real test log would probably be more enlightening than the .sum file, but
from what I can see in the .sum you supplied, there appears to be something
quite suspicious about your build/test environment.

Can you attach the gdb.log of:
$ make check RUNTESTFLAGS=dw2-basic.exp

This is showing:
ERROR: remote_download to host of ./gdb.dwarf2/file1.txt to
/home/yuri/gdb-7.8.2/gdb/testsuite/gdb.dwarf2/file.txt: cp: cannot stat
'./gdb.dwarf2/file1.txt': No such file or directory

This same error appears in several places, along with:
FAIL: gdb.dwarf2/gdb-index.exp: touch binary

This is all very odd. What directory are you trying to run tests in? Is it
read-only?

A huge chunk of failures is in attach-into-signal.exp. That has, in the past,
been one of a handful of racy tests. Try running it alone and see if it passes.

We'll worry about the other big problem area, python, when we get the
remote_download issues resolved.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/18067] many failures in testsuite

[ciro.santilli at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18067

Ciro Santilli <ciro.santilli at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ciro.santilli at gmail dot com

--- Comment #2 from Ciro Santilli <ciro.santilli at gmail dot com> ---
I was getting 388 failures on tag binutils-2_25, Ubuntu 14.04 GCC 4.8 the large
majority related to attach.

Then I run with:

    sudo make check RUNTESTFLAGS="--ignore bigcore.exp"

and it fell down to 36 only!

I ignore bigcore.exp because it slows my machine too much:
https://sourceware.org/bugzilla/show_bug.cgi?id=18704

So it seems that the inability to attach without sudo was the cause of most
failures.

We should really document that somewhere.

@Keith:

- why don't we mark failing tests with XFAIL so as to unclutter the tests?
- shouldn't we document about the sudo on the testsuite/README ?
- I am unable to find the buildbot results for a given commit to compare, e.g.
binutils-2_25... is that possible? Not built-in it seems:
https://groups.google.com/a/chromium.org/forum/#!topic/infra-dev/T_7S9HXLWlo

Attaching .sum and .log.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/18067] many failures in testsuite

[ciro.santilli at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18067

--- Comment #3 from Ciro Santilli <ciro.santilli at gmail dot com> ---
Created attachment 8448
  --> https://sourceware.org/bugzilla/attachment.cgi?id=8448&action=edit
sudo Ubuntu 14.04

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/18067] many failures in testsuite

[palves at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18067

Pedro Alves <palves at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |palves at redhat dot com

--- Comment #5 from Pedro Alves <palves at redhat dot com> ---
> why don't we mark failing tests with XFAIL so as to unclutter the tests?

We do - though it's KFAIL for known gdb failures - but it takes someone to
actually do the leg work.  All help appreciated.

> - shouldn't we document about the sudo on the testsuite/README ?

I'd welcome a patch.  As you found out, it's the ptrace_scope thing that's at
play.  Running gdb under sudo isn't a good idea.

> - I am unable to find the buildbot results for a given commit to compare, e.g. binutils-2_25... is that possible? Not built-in it seems: 

Find the git repo for results of the builder you want the results for, then
look for a results commit whose subject indicates it tested the gdb commit hash
you wanted.  Note that most builders are testing master, some the 7.10 branch,
so you naturally won't find results for hashes only on other branches.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/18067] many failures in testsuite

[ciro.santilli at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=18067

--- Comment #6 from Ciro Santilli <ciro.santilli at gmail dot com> ---
(In reply to Pedro Alves from comment #5)
> > why don't we mark failing tests with XFAIL so as to unclutter the tests?
> 
> We do - though it's KFAIL for known gdb failures - but it takes someone to
> actually do the leg work.  All help appreciated.
> 

OK!

> > - shouldn't we document about the sudo on the testsuite/README ?
> 
> I'd welcome a patch.  As you found out, it's the ptrace_scope thing that's
> at play.  Running gdb under sudo isn't a good idea.
> 

Patch at: https://sourceware.org/ml/gdb-patches/2015-07/msg00932.html

> > - I am unable to find the buildbot results for a given commit to compare, e.g. binutils-2_25... is that possible? Not built-in it seems: 
> 
> Find the git repo for results of the builder you want the results for, then
> look for a results commit whose subject indicates it tested the gdb commit
> hash you wanted.  Note that most builders are testing master, some the 7.10
> branch, so you naturally won't find results for hashes only on other
> branches.

Ah, after reading the wiki a bit more I've found it, thanks:
http://gdb-build.sergiodj.net/cgit

-- 
You are receiving this mail because:
You are on the CC list for the bug.