[Bug sim/29596] New: "make check-sim" is broken (unexpectedly passes without actually testing)

["research_trasio at irq dot a4lg.com" <sourceware-bugzilla@sourceware.org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=29596

            Bug ID: 29596
           Summary: "make check-sim" is broken (unexpectedly passes
                    without actually testing)
           Product: gdb
           Version: HEAD
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: sim
          Assignee: unassigned at sourceware dot org
          Reporter: research_trasio at irq dot a4lg.com
                CC: vapier at gentoo dot org
  Target Milestone: ---

Created attachment 14344
  --> https://sourceware.org/bugzilla/attachment.cgi?id=14344&action=edit
testrun.1.log: AArch64 "make check-sim" log before the patch

While I'm testing whether PR29595 is fixed, I found another bug.  "make
check-sim" doesn't work because it cannot detect a suitable assembler/compiler.

This is what I'm repeatedly pinging but there's no response.  At least, it's
worth it to track here.

Patch:
<https://sourceware.org/pipermail/gdb-patches/2022-August/191564.html>
<https://sourceware.org/pipermail/gdb-patches/2022-September/191843.html> (ping
2)
<https://sourceware.org/pipermail/gdb-patches/2022-September/191997.html> (ping
3)

I first reproduced this on riscv64-unknown-elf but talking with
aarch64-unknown-elf will be better since AArch64 has decent number of tests (in
contrast to 1 from RISC-V).


[How to reproduce]

1.  Configure Binutils with aarch64-unknown-elf and build it
    /src/binutils/configure --target=aarch64-unknown-elf && make
2.  Run `make check-sim' and confirmed that the simulator tests "pass"
3.  Intentionally try to fail the test by modifying
    `sim/testsuite/aarch64/pass.s'
    (replace the last line from "pass" to "fail")
4.  Run `make check-sim' and "confirmed" that the simulator tests "pass"
    (it should fail!)


[Analysis]

... Yes, something is going wrong.
After the test, you can see the test log in
`sim/testsuite/aarch64/allinsn/testrun.log' (example: attached testrun.1.log).
We clearly find that the test runner didn't find recently built assembler
(gas/as-new).
That assembler (and the linker) is supposed to be used because the simulator
itself (sim/Makefile) sets its configuration (see attached site.exp, generated
by sim/Makefile).

At last, we find `sim/testsuite/lib/sim-defs.exp'.
In the `sim_init_toolchain' function, it extracts {AS,LD,CC}_FOR_TARGET_AARCH64
and sets proper {AS,LD,CC}_FOR_TARGET.
At least, it is supposed to do so.

However, this block doesn't work because the `arch' variable returned by the
`sim_arch' function is "./aarch64".  That is supposed to be "aarch64".


[Fix: Cause and Patch Details]

`sim_arch' function is simple.  Until the dirname part of the "arch" is not
"." (current directory), it trims the filename part.

For instance, if subdir is "A/B/C/D", this function returns "A".
However, if subdir is "./A/B/C/D", this function returns "./A", not "A".
In fact, actual subdir value here is "./aarch64" and we will get "./aarch64"
for [sim_arch].
As a result, it fails to extract proper assembler/linker/compiler for given
target.

To deal with it, I added another "file tail" function call before returning.

After this patch, `sim/testsuite/aarch64/allinsn/testrun.log' will look like
attached testrun.2.log.
We can confirm that the assembler is detected and "expectedly" fail (because we
intentionally broke a testcase).

Of course, fixing a testcase will make a success (see attached testrun.3.log).


[My status]

I completed copyright assignment of GDB to FSF in August 2022 and I'm now even
a "write after approval" committer.  If a maintainer gives me an approval, I
can commit it in a day.

-- 
You are receiving this mail because:
You are on the CC list for the bug.