[committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[Tom de Vries]

Hi,

When running test-case gdb.base/jit-elf.exp with target board
cc-with-gdb-index, we get:
...
spawn -ignore SIGHUP gdb/contrib/cc-with-tweaks.sh -i gcc \
  -fno-stack-protector src/gdb/testsuite/gdb.base/jit-elf-main.c \
  -fdiagnostics-color=never -DATTACH=1 -DLOAD_ADDRESS=0x7000000 \
  -DLOAD_INCREMENT=0x1000000 -g -lm \
  -o outputs/gdb.base/jit-elf/jit-elf-main"-attach"^M
outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
  No such file or directory.^M
output is:
outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
  No such file or directory.^M

gdb compile failed, outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
  No such file or directory.
UNTESTED: gdb.base/jit-elf.exp: failed to compile jit-elf-main"-attach"
...

The problem is a string concat in jit-elf.exp:
...
  ${main_binfile}"-attach"
...
which is intended to generate string 'jit-elf-main-attach' but instead
generates string 'jit-elf-main"-attach"'.

Fix this by using "${main_binfile}-attach" instead.

Tested on x86_64-linux.

Committed to trunk.

Thanks,
- Tom

[gdb/testsuite] Fix incorrect string concat in jit-elf.exp

gdb/testsuite/ChangeLog:

2020-05-12  Tom de Vries  <tdevries@suse.de>

	* gdb.base/jit-elf.exp: Fix string concat.

---
 gdb/testsuite/gdb.base/jit-elf.exp | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gdb/testsuite/gdb.base/jit-elf.exp b/gdb/testsuite/gdb.base/jit-elf.exp
index 68196a0a88..78a19c2c9a 100644
--- a/gdb/testsuite/gdb.base/jit-elf.exp
+++ b/gdb/testsuite/gdb.base/jit-elf.exp
@@ -147,7 +147,7 @@ if { [compile_jit_main ${main_srcfile} ${main_binfile} {}] == 0 } {
 # registered.  We reuse the normal test, and detach/reattach at
 # specific interesting points.
 if {[can_spawn_for_attach]} {
-    if { [compile_jit_main ${main_srcfile} ${main_binfile}"-attach" \
+    if { [compile_jit_main ${main_srcfile} "${main_binfile}-attach" \
 	    {additional_flags=-DATTACH=1}] == 0 } {
 	with_test_prefix attach {
 	    one_jit_test $jit_solibs_target "${hex}  jit_function_0001\[\r\n\]+${hex}  jit_function_0002" 1
@@ -155,7 +155,8 @@ if {[can_spawn_for_attach]} {
     }
 }
 
-if { [compile_jit_main ${main_srcfile} ${main_binfile}"-pie" {additional_flags=-fPIE ldflags=-pie}] == 0 } {
+if { [compile_jit_main ${main_srcfile} "${main_binfile}-pie" \
+	  {additional_flags=-fPIE ldflags=-pie}] == 0 } {
     with_test_prefix PIE {
 	one_jit_test [lindex $jit_solibs_target 0] "${hex}  jit_function_0001" 0
     }

Re: [committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[Simon Marchi]

On 2020-05-12 9:00 a.m., Tom de Vries wrote:
> Hi,
> 
> When running test-case gdb.base/jit-elf.exp with target board
> cc-with-gdb-index, we get:
> ...
> spawn -ignore SIGHUP gdb/contrib/cc-with-tweaks.sh -i gcc \
>   -fno-stack-protector src/gdb/testsuite/gdb.base/jit-elf-main.c \
>   -fdiagnostics-color=never -DATTACH=1 -DLOAD_ADDRESS=0x7000000 \
>   -DLOAD_INCREMENT=0x1000000 -g -lm \
>   -o outputs/gdb.base/jit-elf/jit-elf-main"-attach"^M
> outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>   No such file or directory.^M
> output is:
> outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>   No such file or directory.^M
> 
> gdb compile failed, outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>   No such file or directory.
> UNTESTED: gdb.base/jit-elf.exp: failed to compile jit-elf-main"-attach"
> ...
> 
> The problem is a string concat in jit-elf.exp:
> ...
>   ${main_binfile}"-attach"
> ...
> which is intended to generate string 'jit-elf-main-attach' but instead
> generates string 'jit-elf-main"-attach"'.
> 
> Fix this by using "${main_binfile}-attach" instead.
> 
> Tested on x86_64-linux.
> 
> Committed to trunk.
> 
> Thanks,
> - Tom

Thanks for the fixes to the JIT tests Tom.  I did not test with non-default
boards, I'll try to do it more diligently in the future.

Simon

RE: [committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[Strasuns, Mihails]

On that topic - what is the recommended set of boards to check locally for a generic gdb patchset?

Mihails

> -----Original Message-----
> From: Simon Marchi <simon.marchi@polymtl.ca>
> Sent: Tuesday, May 12, 2020 4:15 PM
> To: Tom de Vries <tdevries@suse.de>; gdb-patches@sourceware.org
> Cc: Strasuns, Mihails <mihails.strasuns@intel.com>
> Subject: Re: [committed][gdb/testsuite] Fix incorrect string concat in jit-
> elf.exp
> 
> On 2020-05-12 9:00 a.m., Tom de Vries wrote:
> > Hi,
> >
> > When running test-case gdb.base/jit-elf.exp with target board
> > cc-with-gdb-index, we get:
> > ...
> > spawn -ignore SIGHUP gdb/contrib/cc-with-tweaks.sh -i gcc \
> >   -fno-stack-protector src/gdb/testsuite/gdb.base/jit-elf-main.c \
> >   -fdiagnostics-color=never -DATTACH=1 -DLOAD_ADDRESS=0x7000000 \
> >   -DLOAD_INCREMENT=0x1000000 -g -lm \
> >   -o outputs/gdb.base/jit-elf/jit-elf-main"-attach"^M
> > outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
> >   No such file or directory.^M
> > output is:
> > outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
> >   No such file or directory.^M
> >
> > gdb compile failed, outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
> >   No such file or directory.
> > UNTESTED: gdb.base/jit-elf.exp: failed to compile jit-elf-main"-attach"
> > ...
> >
> > The problem is a string concat in jit-elf.exp:
> > ...
> >   ${main_binfile}"-attach"
> > ...
> > which is intended to generate string 'jit-elf-main-attach' but instead
> > generates string 'jit-elf-main"-attach"'.
> >
> > Fix this by using "${main_binfile}-attach" instead.
> >
> > Tested on x86_64-linux.
> >
> > Committed to trunk.
> >
> > Thanks,
> > - Tom
> 
> Thanks for the fixes to the JIT tests Tom.  I did not test with non-default
> boards, I'll try to do it more diligently in the future.
> 
> Simon
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Gary Kershaw
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: [committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[Tom de Vries]

On 12-05-2020 16:14, Simon Marchi wrote:
> On 2020-05-12 9:00 a.m., Tom de Vries wrote:
>> Hi,
>>
>> When running test-case gdb.base/jit-elf.exp with target board
>> cc-with-gdb-index, we get:
>> ...
>> spawn -ignore SIGHUP gdb/contrib/cc-with-tweaks.sh -i gcc \
>>   -fno-stack-protector src/gdb/testsuite/gdb.base/jit-elf-main.c \
>>   -fdiagnostics-color=never -DATTACH=1 -DLOAD_ADDRESS=0x7000000 \
>>   -DLOAD_INCREMENT=0x1000000 -g -lm \
>>   -o outputs/gdb.base/jit-elf/jit-elf-main"-attach"^M
>> outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>>   No such file or directory.^M
>> output is:
>> outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>>   No such file or directory.^M
>>
>> gdb compile failed, outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>>   No such file or directory.
>> UNTESTED: gdb.base/jit-elf.exp: failed to compile jit-elf-main"-attach"
>> ...
>>
>> The problem is a string concat in jit-elf.exp:
>> ...
>>   ${main_binfile}"-attach"
>> ...
>> which is intended to generate string 'jit-elf-main-attach' but instead
>> generates string 'jit-elf-main"-attach"'.
>>
>> Fix this by using "${main_binfile}-attach" instead.
>>
>> Tested on x86_64-linux.
>>
>> Committed to trunk.
>>
>> Thanks,
>> - Tom
> 
> Thanks for the fixes to the JIT tests Tom.  I did not test with non-default
> boards, I'll try to do it more diligently in the future.

I usually only test with the symtabs-related boards if I do a
symtab-related fix, so I don't think I would have found this either
pre-commit :)

[ And this could have been found by the buildbot, were it not two weeks
behind :( . I realize I'm guilty there a bit for doing lots of small
commits, which doesn't help. ]

Thanks,
- Tom

Re: [committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[Tom de Vries]

Hi,

I don't know if such a generic list exists.  I'd say what makes sense
depends on the patch.

For symtab-related patches, use readnow, cc-with-gdb-index,
cc-with-debug-names.

If the test contains nonstandard handling of inferiors, try some
gdbserver boards (I'm not sure which).

If the test contains explicit file handling, try some of the remote
boards (I'm not sure which).

Thanks,
- Tom

On 12-05-2020 16:37, Strasuns, Mihails wrote:
> On that topic - what is the recommended set of boards to check locally for a generic gdb patchset?
> 
> Mihails
> 
>> -----Original Message-----
>> From: Simon Marchi <simon.marchi@polymtl.ca>
>> Sent: Tuesday, May 12, 2020 4:15 PM
>> To: Tom de Vries <tdevries@suse.de>; gdb-patches@sourceware.org
>> Cc: Strasuns, Mihails <mihails.strasuns@intel.com>
>> Subject: Re: [committed][gdb/testsuite] Fix incorrect string concat in jit-
>> elf.exp
>>
>> On 2020-05-12 9:00 a.m., Tom de Vries wrote:
>>> Hi,
>>>
>>> When running test-case gdb.base/jit-elf.exp with target board
>>> cc-with-gdb-index, we get:
>>> ...
>>> spawn -ignore SIGHUP gdb/contrib/cc-with-tweaks.sh -i gcc \
>>>   -fno-stack-protector src/gdb/testsuite/gdb.base/jit-elf-main.c \
>>>   -fdiagnostics-color=never -DATTACH=1 -DLOAD_ADDRESS=0x7000000 \
>>>   -DLOAD_INCREMENT=0x1000000 -g -lm \
>>>   -o outputs/gdb.base/jit-elf/jit-elf-main"-attach"^M
>>> outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>>>   No such file or directory.^M
>>> output is:
>>> outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>>>   No such file or directory.^M
>>>
>>> gdb compile failed, outputs/gdb.base/jit-elf/.tmp/jit-elf-main-attach: \
>>>   No such file or directory.
>>> UNTESTED: gdb.base/jit-elf.exp: failed to compile jit-elf-main"-attach"
>>> ...
>>>
>>> The problem is a string concat in jit-elf.exp:
>>> ...
>>>   ${main_binfile}"-attach"
>>> ...
>>> which is intended to generate string 'jit-elf-main-attach' but instead
>>> generates string 'jit-elf-main"-attach"'.
>>>
>>> Fix this by using "${main_binfile}-attach" instead.
>>>
>>> Tested on x86_64-linux.
>>>
>>> Committed to trunk.
>>>
>>> Thanks,
>>> - Tom
>>
>> Thanks for the fixes to the JIT tests Tom.  I did not test with non-default
>> boards, I'll try to do it more diligently in the future.
>>
>> Simon
> Intel Deutschland GmbH
> Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
> Tel: +49 89 99 8853-0, www.intel.de
> Managing Directors: Christin Eisenschmid, Gary Kershaw
> Chairperson of the Supervisory Board: Nicole Lau
> Registered Office: Munich
> Commercial Register: Amtsgericht Muenchen HRB 186928
> 

Re: [committed][gdb/testsuite] Fix incorrect string concat in jit-elf.exp

[Simon Marchi]

On 2020-05-13 4:52 a.m., Tom de Vries wrote:
> Hi,
> 
> I don't know if such a generic list exists.  I'd say what makes sense
> depends on the patch.
> 
> For symtab-related patches, use readnow, cc-with-gdb-index,
> cc-with-debug-names.
> 
> If the test contains nonstandard handling of inferiors, try some
> gdbserver boards (I'm not sure which).
> 
> If the test contains explicit file handling, try some of the remote
> boards (I'm not sure which).
> 
> Thanks,
> - Tom

FWIW, I just learned it's possible to pass a list of target boards to --target_board:

  --target_board='unix native-gdbserver native-extended-gdbserver'

And it gives a combined result at the end.  So I'll make myself an alias to test on all
the commonly used boards, probably:

- cc-with-debug-names.exp
- cc-with-dwz.exp
- cc-with-dwz-m.exp
- cc-with-gdb-index.exp
- debug-types.exp
- dwarf4-gdb-index.exp
- fission-dwp.exp
- fission.exp
- native-extended-gdbserver.exp
- native-gdbserver.exp
- native-stdio-gdbserver.exp
- readnow.exp

When working on a single test, it's usually not to long to run them all.  Of course, it's
not really practical to run the complete testsuite twice (before and after) for each board
for every change we do...

Simon

How to run multiple target boards

[Tom de Vries]

[ was: Re: [committed][gdb/testsuite] Fix incorrect string concat in
jit-elf.exp ]

On 13-05-2020 15:45, Simon Marchi wrote:
> FWIW, I just learned it's possible to pass a list of target boards to --target_board:
> 
>   --target_board='unix native-gdbserver native-extended-gdbserver'
> 
> And it gives a combined result at the end.  So I'll make myself an alias to test on all
> the commonly used boards, probably:
> 
> - cc-with-debug-names.exp
> - cc-with-dwz.exp
> - cc-with-dwz-m.exp
> - cc-with-gdb-index.exp
> - debug-types.exp
> - dwarf4-gdb-index.exp
> - fission-dwp.exp
> - fission.exp
> - native-extended-gdbserver.exp
> - native-gdbserver.exp
> - native-stdio-gdbserver.exp
> - readnow.exp
> 
> When working on a single test, it's usually not to long to run them all.  Of course, it's
> not really practical to run the complete testsuite twice (before and after) for each board
> for every change we do...

Hi Simon,

I just tried this out.

My test scripts use make check, but I didn't manage to make that work
yet, so I tried out using runtest directly (and I may be missing
something obvious, given that I haven't used this before):
...
$ cd build/gdb/testsuite
$ runtest gdb.base/gold-gdb-index.exp --target_board='cc-with-gdb-index
 unix'
...

It seems however that the global settings (CC_FOR_TARGET etc) from the
first board stay active in the unix board:
...
Schedule of variations:
    cc-with-gdb-index
    unix

Running target cc-with-gdb-index
Using
/data/gdb_versions/devel/src/gdb/testsuite/boards/../boards/cc-with-gdb-index.exp
as board description file for target.
Using
/data/gdb_versions/devel/src/gdb/testsuite/boards/../boards/cc-with-tweaks.exp
as board description file for target.
Using
/data/gdb_versions/devel/src/gdb/testsuite/boards/../boards/local-board.exp
as board description file for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for
target.
Using /data/gdb_versions/devel/src/gdb/testsuite/config/unix.exp as
tool-and-target-specific interface file.
Running
/data/gdb_versions/devel/src/gdb/testsuite/gdb.base/gold-gdb-index.exp ...
gdb compile failed, Error while writing index for
`/data/gdb_versions/devel/build/gdb/testsuite/outputs/gdb.base/gold-gdb-index/.tmp/gold-gdb-index':
Cannot use an index to create the index

                === gdb Summary for cc-with-gdb-index ===

# of untested testcases         1
Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file
for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for
target.
Using /data/gdb_versions/devel/src/gdb/testsuite/config/unix.exp as
tool-and-target-specific interface file.
Running
/data/gdb_versions/devel/src/gdb/testsuite/gdb.base/gold-gdb-index.exp ...
gdb compile failed, Error while writing index for
`/data/gdb_versions/devel/build/gdb/testsuite/outputs/gdb.base/gold-gdb-index/.tmp/gold-gdb-index':
Cannot use an index to create the index

                === gdb Summary for unix ===

# of untested testcases         1

                === gdb Summary ===

# of untested testcases         2
/data/gdb_versions/devel/build/gdb/gdb version  10.0.50.20200525-git -nw
-nx -data-directory
/data/gdb_versions/devel/build/gdb/testsuite/../data-directory  -ex "set
sysroot"
...

I managed to make this work by unsetting some vars in base-config.exp:
...
$ cat gdb/testsuite/config/base-config.exp
if { [info exists CC_FOR_TARGET] } {
    unset CC_FOR_TARGET
}
if { [info exists CXX_FOR_TARGET] } {
    unset CXX_FOR_TARGET
}
...
after which I get:
...
                === gdb Summary ===

# of expected passes            4
# of untested testcases         1
...

I'm not sure yet if this is the proper way to fix this.

Thanks,
- Tom

Re: How to run multiple target boards

[Pedro Alves]

On 5/26/20 9:49 AM, Tom de Vries wrote:
> [ was: Re: [committed][gdb/testsuite] Fix incorrect string concat in
> jit-elf.exp ]
> 
> On 13-05-2020 15:45, Simon Marchi wrote:
>> FWIW, I just learned it's possible to pass a list of target boards to --target_board:
>>
>>   --target_board='unix native-gdbserver native-extended-gdbserver'
>>
>> And it gives a combined result at the end.  So I'll make myself an alias to test on all
>> the commonly used boards, probably:
>>
>> - cc-with-debug-names.exp
>> - cc-with-dwz.exp
>> - cc-with-dwz-m.exp
>> - cc-with-gdb-index.exp
>> - debug-types.exp
>> - dwarf4-gdb-index.exp
>> - fission-dwp.exp
>> - fission.exp
>> - native-extended-gdbserver.exp
>> - native-gdbserver.exp
>> - native-stdio-gdbserver.exp
>> - readnow.exp

What are the advantages of doing this compared to a script that runs the
testsuite for each of the boards, separately?

>>
>> When working on a single test, it's usually not to long to run them all.  Of course, it's
>> not really practical to run the complete testsuite twice (before and after) for each board
>> for every change we do...
> 
> Hi Simon,
> 
> I just tried this out.
> 
> My test scripts use make check, but I didn't manage to make that work
> yet, so I tried out using runtest directly (and I may be missing
> something obvious, given that I haven't used this before):
> ...
> $ cd build/gdb/testsuite
> $ runtest gdb.base/gold-gdb-index.exp --target_board='cc-with-gdb-index
>  unix'
> ...

One issue this with that I've run into in the past, is with global
variables leaking from one board to the other.

And I've just tried a quick test, and lucky me I seem to have run into
something like that immediately:

 $ make check RUNTESTFLAGS="--target_board='native-gdbserver unix' break.exp"
 ...
 Schedule of variations:
     native-gdbserver
     unix
 ...
 FAIL: gdb.base/break.exp: run until function breakpoint
 FAIL: gdb.base/break.exp: run until breakpoint set at a line number (the program is no longer running)
 FAIL: gdb.base/break.exp: run until file:function(6) breakpoint (the program is no longer running)
 FAIL: gdb.base/break.exp: run until file:function(5) breakpoint (the program is no longer running)
 (snip a bunch more)

(Running break.exp for each board in isolation passes cleanly, of course.)

Another thing is that the test messages in gdb.sum don't indicate which
board issued the PASS/FAIL (on each result), making it more difficult
to analyze.

> 
> It seems however that the global settings (CC_FOR_TARGET etc) from the
> first board stay active in the unix board:

Yeah, things like that.

Also, the tests for all boards end up under the same testsuite/outputs/ directory,
so whatever the last board was, wins.  I'm not even sure how the testsuite
caching works when you run more than one board in parallel mode.

It would seem to me that if you're looking at running tests against different
boards, that you would want the output directories to be separate, so you can
analyze the results afterwards, avoid cache issues, etc.

Maybe it would be better to come up with our own way to do this, without
relying on dejagnu directly.  Much like we have the TESTS variable,
we could have a BOARDS variable:

 make check TESTS="gdb.base/break.exp" BOARDS="native-gdbserver unix"

and then the outputs foreach board would be in
"testsuite/outputs.native-gdbserver/" and "testsuite/outputs.unix/"
respectively.

Thanks,
Pedro Alves

Re: How to run multiple target boards

[Tom de Vries]

On 26-05-2020 12:39, Pedro Alves wrote:
> On 5/26/20 9:49 AM, Tom de Vries wrote:
>> [ was: Re: [committed][gdb/testsuite] Fix incorrect string concat in
>> jit-elf.exp ]
>>
>> On 13-05-2020 15:45, Simon Marchi wrote:
>>> FWIW, I just learned it's possible to pass a list of target boards to --target_board:
>>>
>>>   --target_board='unix native-gdbserver native-extended-gdbserver'
>>>
>>> And it gives a combined result at the end.  So I'll make myself an alias to test on all
>>> the commonly used boards, probably:
>>>
>>> - cc-with-debug-names.exp
>>> - cc-with-dwz.exp
>>> - cc-with-dwz-m.exp
>>> - cc-with-gdb-index.exp
>>> - debug-types.exp
>>> - dwarf4-gdb-index.exp
>>> - fission-dwp.exp
>>> - fission.exp
>>> - native-extended-gdbserver.exp
>>> - native-gdbserver.exp
>>> - native-stdio-gdbserver.exp
>>> - readnow.exp
> 
> What are the advantages of doing this compared to a script that runs the
> testsuite for each of the boards, separately?
> 

I'd say:
- the target boards not overwriting the .log and .sum files of other
  target boards
- the joint summary.

Thanks,
- Tom

>>>
>>> When working on a single test, it's usually not to long to run them all.  Of course, it's
>>> not really practical to run the complete testsuite twice (before and after) for each board
>>> for every change we do...
>>
>> Hi Simon,
>>
>> I just tried this out.
>>
>> My test scripts use make check, but I didn't manage to make that work
>> yet, so I tried out using runtest directly (and I may be missing
>> something obvious, given that I haven't used this before):
>> ...
>> $ cd build/gdb/testsuite
>> $ runtest gdb.base/gold-gdb-index.exp --target_board='cc-with-gdb-index
>>  unix'
>> ...
> 
> One issue this with that I've run into in the past, is with global
> variables leaking from one board to the other.
> 
> And I've just tried a quick test, and lucky me I seem to have run into
> something like that immediately:
> 
>  $ make check RUNTESTFLAGS="--target_board='native-gdbserver unix' break.exp"
>  ...
>  Schedule of variations:
>      native-gdbserver
>      unix
>  ...
>  FAIL: gdb.base/break.exp: run until function breakpoint
>  FAIL: gdb.base/break.exp: run until breakpoint set at a line number (the program is no longer running)
>  FAIL: gdb.base/break.exp: run until file:function(6) breakpoint (the program is no longer running)
>  FAIL: gdb.base/break.exp: run until file:function(5) breakpoint (the program is no longer running)
>  (snip a bunch more)
> 
> (Running break.exp for each board in isolation passes cleanly, of course.)
> 
> Another thing is that the test messages in gdb.sum don't indicate which
> board issued the PASS/FAIL (on each result), making it more difficult
> to analyze.
> 
>>
>> It seems however that the global settings (CC_FOR_TARGET etc) from the
>> first board stay active in the unix board:
> 
> Yeah, things like that.
> 
> Also, the tests for all boards end up under the same testsuite/outputs/ directory,
> so whatever the last board was, wins.  I'm not even sure how the testsuite
> caching works when you run more than one board in parallel mode.
> 
> It would seem to me that if you're looking at running tests against different
> boards, that you would want the output directories to be separate, so you can
> analyze the results afterwards, avoid cache issues, etc.
> 
> Maybe it would be better to come up with our own way to do this, without
> relying on dejagnu directly.  Much like we have the TESTS variable,
> we could have a BOARDS variable:
> 
>  make check TESTS="gdb.base/break.exp" BOARDS="native-gdbserver unix"
> 
> and then the outputs foreach board would be in
> "testsuite/outputs.native-gdbserver/" and "testsuite/outputs.unix/"
> respectively.
> 
> Thanks,
> Pedro Alves
> 

Re: How to run multiple target boards

[Simon Marchi]

On 2020-05-26 6:56 a.m., Tom de Vries wrote:
> On 26-05-2020 12:39, Pedro Alves wrote:
>> On 5/26/20 9:49 AM, Tom de Vries wrote:
>>> [ was: Re: [committed][gdb/testsuite] Fix incorrect string concat in
>>> jit-elf.exp ]
>>>
>>> On 13-05-2020 15:45, Simon Marchi wrote:
>>>> FWIW, I just learned it's possible to pass a list of target boards to --target_board:
>>>>
>>>>   --target_board='unix native-gdbserver native-extended-gdbserver'
>>>>
>>>> And it gives a combined result at the end.  So I'll make myself an alias to test on all
>>>> the commonly used boards, probably:
>>>>
>>>> - cc-with-debug-names.exp
>>>> - cc-with-dwz.exp
>>>> - cc-with-dwz-m.exp
>>>> - cc-with-gdb-index.exp
>>>> - debug-types.exp
>>>> - dwarf4-gdb-index.exp
>>>> - fission-dwp.exp
>>>> - fission.exp
>>>> - native-extended-gdbserver.exp
>>>> - native-gdbserver.exp
>>>> - native-stdio-gdbserver.exp
>>>> - readnow.exp
>>
>> What are the advantages of doing this compared to a script that runs the
>> testsuite for each of the boards, separately?
>>
> 
> I'd say:
> - the target boards not overwriting the .log and .sum files of other
>   target boards
> - the joint summary.
> 
> Thanks,
> - Tom
> 
>>>>
>>>> When working on a single test, it's usually not to long to run them all.  Of course, it's
>>>> not really practical to run the complete testsuite twice (before and after) for each board
>>>> for every change we do...
>>>
>>> Hi Simon,
>>>
>>> I just tried this out.
>>>
>>> My test scripts use make check, but I didn't manage to make that work
>>> yet, so I tried out using runtest directly (and I may be missing
>>> something obvious, given that I haven't used this before):
>>> ...
>>> $ cd build/gdb/testsuite
>>> $ runtest gdb.base/gold-gdb-index.exp --target_board='cc-with-gdb-index
>>>  unix'
>>> ...
>>
>> One issue this with that I've run into in the past, is with global
>> variables leaking from one board to the other.
>>
>> And I've just tried a quick test, and lucky me I seem to have run into
>> something like that immediately:
>>
>>  $ make check RUNTESTFLAGS="--target_board='native-gdbserver unix' break.exp"
>>  ...
>>  Schedule of variations:
>>      native-gdbserver
>>      unix
>>  ...
>>  FAIL: gdb.base/break.exp: run until function breakpoint
>>  FAIL: gdb.base/break.exp: run until breakpoint set at a line number (the program is no longer running)
>>  FAIL: gdb.base/break.exp: run until file:function(6) breakpoint (the program is no longer running)
>>  FAIL: gdb.base/break.exp: run until file:function(5) breakpoint (the program is no longer running)
>>  (snip a bunch more)
>>
>> (Running break.exp for each board in isolation passes cleanly, of course.)
>>
>> Another thing is that the test messages in gdb.sum don't indicate which
>> board issued the PASS/FAIL (on each result), making it more difficult
>> to analyze.

Ok, so in retrospect, not such a good idea :(.

Simon