remote host testing expectations re multiple simultaneous connections

remote host testing expectations re multiple simultaneous connections

[Sandra Loosemore]

I've noticed that there are several gdb.mi tests that start GDB before 
building the test program, like this bit of code from mi-cli.exp:

gdb_exit
if [mi_gdb_start] {
     continue
}

standard_testfile basics.c

if  { [gdb_compile "${srcdir}/${subdir}/${srcfile}" "${binfile}" 
executable {debug}] != "" } {
      untested "failed to compile"
      return -1
}

mi_gdb_test "-interpreter-exec" \
   {\^error,msg="-interpreter-exec: Usage: -interpreter-exec interp 
command"} \
   "-interpreter-exec with no arguments"

E.g. there's an implicit expectation that when testing on a remote host, 
it can start GDB and leave it running and waiting to accept GDB commands 
while running some other shell commands to build the test program on a 
different channel to the same remote host.  Is this a reasonable 
expectation that is documented somewhere?  Or are the test cases that 
are structured like this just broken?  :-S  If the latter I will submit 
a patch to fix them.

-Sandra

Re: remote host testing expectations re multiple simultaneous connections

[Simon Marchi]

On 2018-10-26 15:21, Sandra Loosemore wrote:
> I've noticed that there are several gdb.mi tests that start GDB before
> building the test program, like this bit of code from mi-cli.exp:
> 
> gdb_exit
> if [mi_gdb_start] {
>     continue
> }
> 
> standard_testfile basics.c
> 
> if  { [gdb_compile "${srcdir}/${subdir}/${srcfile}" "${binfile}"
> executable {debug}] != "" } {
>      untested "failed to compile"
>      return -1
> }
> 
> mi_gdb_test "-interpreter-exec" \
>   {\^error,msg="-interpreter-exec: Usage: -interpreter-exec interp 
> command"} \
>   "-interpreter-exec with no arguments"
> 
> E.g. there's an implicit expectation that when testing on a remote
> host, it can start GDB and leave it running and waiting to accept GDB
> commands while running some other shell commands to build the test
> program on a different channel to the same remote host.  Is this a
> reasonable expectation that is documented somewhere?  Or are the test
> cases that are structured like this just broken?  :-S  If the latter I
> will submit a patch to fix them.

I haven't tried remote host testing in a long time, but I would expect 
the framework to be able to run multiple things concurrently.  Just like 
for remote target testing, we are able to spawn multiple programs, spawn 
gdbserver, and attach to them.

What happens currently when you try it?

Simon

Re: remote host testing expectations re multiple simultaneous connections

[Sandra Loosemore]

On 10/28/2018 11:04 AM, Simon Marchi wrote:
> On 2018-10-26 15:21, Sandra Loosemore wrote:
>> I've noticed that there are several gdb.mi tests that start GDB before
>> building the test program, like this bit of code from mi-cli.exp:
>>
>> gdb_exit
>> if [mi_gdb_start] {
>>     continue
>> }
>>
>> standard_testfile basics.c
>>
>> if  { [gdb_compile "${srcdir}/${subdir}/${srcfile}" "${binfile}"
>> executable {debug}] != "" } {
>>      untested "failed to compile"
>>      return -1
>> }
>>
>> mi_gdb_test "-interpreter-exec" \
>>   {\^error,msg="-interpreter-exec: Usage: -interpreter-exec interp 
>> command"} \
>>   "-interpreter-exec with no arguments"
>>
>> E.g. there's an implicit expectation that when testing on a remote
>> host, it can start GDB and leave it running and waiting to accept GDB
>> commands while running some other shell commands to build the test
>> program on a different channel to the same remote host.  Is this a
>> reasonable expectation that is documented somewhere?  Or are the test
>> cases that are structured like this just broken?  :-S  If the latter I
>> will submit a patch to fix them.
> 
> I haven't tried remote host testing in a long time, but I would expect 
> the framework to be able to run multiple things concurrently.  Just like 
> for remote target testing, we are able to spawn multiple programs, spawn 
> gdbserver, and attach to them.
> 
> What happens currently when you try it?

Our local remote-host DejaGnu test support (for testing on Windows host, 
actually) maintains some global state about the spawned GDB connection 
that is getting clobbered by opening another connection to the same host 
to run shell commands, resulting in GDB getting killed (so all these 
tests fail).  I've been struggling to find exactly what is going wrong, 
but wanted to make sure that this is actually a test harness bug and not 
broken testcases before spending any more time on it.

It does seem like requiring support for multiple simultaneous 
connections to either a host or target would preclude testing on 
anything where all you have is a single serial connection to talk to it, 
and not (say) multiple ssh connections.

-Sandra

Re: remote host testing expectations re multiple simultaneous connections

[Simon Marchi]

On 2018-10-28 13:43, Sandra Loosemore wrote:
> Our local remote-host DejaGnu test support (for testing on Windows
> host, actually) maintains some global state about the spawned GDB
> connection that is getting clobbered by opening another connection to
> the same host to run shell commands, resulting in GDB getting killed
> (so all these tests fail).  I've been struggling to find exactly what
> is going wrong, but wanted to make sure that this is actually a test
> harness bug and not broken testcases before spending any more time on
> it.
> 
> It does seem like requiring support for multiple simultaneous
> connections to either a host or target would preclude testing on
> anything where all you have is a single serial connection to talk to
> it, and not (say) multiple ssh connections.

Ok, I understand.  If we can modify the test you referred to (by 
reordering operations) to make them work for your use case and it 
doesn't impact the quality of the test, then I don't see why it wouldn't 
be acceptable.

Simon