[Bug external/31410] New: [gdb/Linaro CI progression] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[Bug external/31410] New: [gdb/Linaro CI progression] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

            Bug ID: 31410
           Summary: [gdb/Linaro CI progression] FAIL:
                    gdb.base/startup-with-shell.exp: startup_with_shell =
                    off; run_args = *.unique-extension: first argument not
                    expanded
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: external
          Assignee: unassigned at sourceware dot org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

The linaro CI consistently reports a progression on test-case
gdb.base/startup-with-shell.exp.

AFAICU, this happens because:
- the FAIL is listed in an xfails.xfail file, and
- the FAIL was fixed.

I think the FAIL was fixed by:
...
commit 52498004a34dfa4bc55fbf791a4ed4e81599a436
Author: Andrew Burgess <aburgess@redhat.com>
Date:   Wed Jan 17 09:53:16 2024 +0000

    gdb/testsuite: handle long filenames in gdb.base/startup-with-shell.exp

    I got a report of a failure from Linaro's CI testing for the test
    gdb.base/startup-with-shell.exp.
...

Can we update the xfails.xfail file?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug external/31410] [gdb/Linaro CI progression] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |thiago.bauermann at linaro dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug external/31410] [gdb/Linaro CI progression] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[maxim.kuvyrkov at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

Maxim Kuvyrkov <maxim.kuvyrkov at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |maxim.kuvyrkov at gmail dot com

--- Comment #1 from Maxim Kuvyrkov <maxim.kuvyrkov at gmail dot com> ---
Hi Tom,

Yes, this annoys me as well, but the problem is a bit more complicated.  Our CI
automatically updates xfails file, which includes both failed test in the
baseline and flaky tests.

The problem with this test, is that it consistently fails in our post-commit CI
jobs, but tends to pass in the pre-commit jobs.  Since we are using baseline
results from the post-commit CI to compare pre-commit CI results against -- we
get this "progression".

We have it on our TODO list to fix this problem, but it requires digging into
the GDB testsuite logic to understand where the different behavior is coming
from.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/31410] [gdb/testsuite] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|[gdb/Linaro CI progression] |[gdb/testsuite] FAIL:
                   |FAIL:                       |gdb.base/startup-with-shell
                   |gdb.base/startup-with-shell |.exp: startup_with_shell =
                   |.exp: startup_with_shell =  |off; run_args =
                   |off; run_args =             |*.unique-extension: first
                   |*.unique-extension: first   |argument not expanded
                   |argument not expanded       |
          Component|external                    |testsuite

--- Comment #2 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Maxim Kuvyrkov from comment #1)
> Hi Tom,
> 
> Yes, this annoys me as well, but the problem is a bit more complicated.  Our
> CI automatically updates xfails file, which includes both failed test in the
> baseline and flaky tests.
> 
> The problem with this test, is that it consistently fails in our post-commit
> CI jobs, but tends to pass in the pre-commit jobs.  Since we are using
> baseline results from the post-commit CI to compare pre-commit CI results
> against -- we get this "progression".
> 

Hi Maxim,

I see, interesting.

> We have it on our TODO list to fix this problem, but it requires digging
> into the GDB testsuite logic to understand where the different behavior is
> coming from.

Understood.

I managed to find the failure mode:
...
(gdb) PASS: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args
= *.unique-extension: inferior started
print argv[1]
$1 = 0xfffeda96
"/home/tcwg-buildslave/workspace/tcwg_gnu_4/abe/builds/armv8l-unknown-linux-gnueabihf/armv8l-unknown-linux-gnueabihf/gdb-gdb.git~master/gdb/testsuite/outputs/gdb.base/startup-with-shell/*.unique-extens"...
(gdb) FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args
= *.unique-extension: first argument not expanded
...
at
https://ci.linaro.org/job/tcwg_gdb_check--master-arm-build/lastSuccessfulBuild/artifact/artifacts/00-sumfiles/gdb.log.xz
.

I'm changing the component from external to testsuite.

I think the fix should be the same as in the commit mentioned in comment 0:
...
diff --git a/gdb/testsuite/gdb.base/startup-with-shell.exp
b/gdb/testsuite/gdb.base/startup-with-shell.exp
index d5ec503dceb..670fab6a543 100644
--- a/gdb/testsuite/gdb.base/startup-with-shell.exp
+++ b/gdb/testsuite/gdb.base/startup-with-shell.exp
@@ -66,6 +66,7 @@ with_test_prefix "startup_with_shell = on; run_args =
*.unique-extension" {

 with_test_prefix "startup_with_shell = off; run_args = *.unique-extension" {
     initial_setup_simple "off" "$unique_file_dir/*.unique-extension"
+    gdb_test_no_output "set print characters unlimited"
     gdb_test "print argv\[1\]" "\\\$$decimal = $hex
\"$unique_file_dir/\\\*\.unique-extension\"" \
        "first argument not expanded"
 }
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/31410] [gdb/testsuite] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/31410] [gdb/testsuite] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

--- Comment #3 from Tom de Vries <vries at gcc dot gnu.org> ---
https://sourceware.org/pipermail/gdb-patches/2024-February/206826.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/31410] [gdb/testsuite] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

--- Comment #4 from Sourceware Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom de Vries <vries@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=d82ede20df36eb8acfc9e67969e396e3a8fe1d45

commit d82ede20df36eb8acfc9e67969e396e3a8fe1d45
Author: Tom de Vries <tdevries@suse.de>
Date:   Mon Feb 26 15:59:47 2024 +0100

    [gdb/testsuite] Further handle long filenames in
gdb.base/startup-with-shell.exp

    In commit 52498004a34 ("gdb/testsuite: handle long filenames in
    gdb.base/startup-with-shell.exp") we fixed a FAIL reported by the Linaro
CI:
    ...
    (gdb) print argv[1]
    $1 = 0xfffed978 "<snip>/startup-with-shell/unique-file.unique-e"...
    (gdb) FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = on; \
      run_args = *.unique-extension: first argument expanded
    ...

    PR testsuite/31410 reports a similar failure:
    ...
    (gdb) print argv[1]
    $1 = 0xfffeda96 "<snip>/startup-with-shell/*.unique-extens"...
    (gdb) FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; \
      run_args = *.unique-extension: first argument not expanded
    ...

    Fix this in the same way, using "set print characters unlimited".

    Tested on x86_64-linux.

    Approved-By: Tom Tromey <tom@tromey.com>

    Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31410

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/31410] [gdb/testsuite] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |15.1
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #5 from Tom de Vries <vries at gcc dot gnu.org> ---
Fixed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug testsuite/31410] [gdb/testsuite] FAIL: gdb.base/startup-with-shell.exp: startup_with_shell = off; run_args = *.unique-extension: first argument not expanded

[thiago.bauermann at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=31410

--- Comment #6 from Thiago Jung Bauermann <thiago.bauermann at linaro dot org> ---
Thank you!

-- 
You are receiving this mail because:
You are on the CC list for the bug.