GDB 8.2.90 available for testing

GDB 8.2.90 available for testing

[Joel Brobecker]

Hello,

I have just finished creating the gdb-8.2.90 pre-release.
It is available for download at the following location:

    ftp://sourceware.org/pub/gdb/snapshots/branch/gdb-8.2.90.tar.xz

A gzip'ed version is also available: gdb-8.2.90.tar.gz.

Please give it a test if you can and report any problems you might find.

On behalf of all the GDB contributors, thank you!
-- 
Joel Brobecker

Re: GDB 8.2.90 available for testing

[Jim Wilson]

On 2/26/19 9:51 PM, Joel Brobecker wrote:
> I have just finished creating the gdb-8.2.90 pre-release.
> Please give it a test if you can and report any problems you might find.

On a riscv64-linux system (HiFive Unleashed w/ Fedora Core 29), without 
an Ada compiler in my path, I get

                 === gdb Summary ===

# of expected passes            50264
# of unexpected failures        2297
# of expected failures          57
# of unknown successes          3
# of known failures             79
# of untested testcases         132
# of unresolved testcases       110
# of unsupported tests          359

which looks OK.  There was no riscv-linux native support in gdb-8.2 so 
this is a major improvement over the last one.  I did used to have 
better results though.  I'm seeing

hifiveu017:1040$ grep ^FAIL gdb.sum | grep infcall | wc
     878    8059  103059

and I think that these mostly used to work, since I used to have about 
1500 failures.  I think something in the last 3 months broke this, but I 
haven't had time to track it down.  I see stuff like this in the log files

p/d check_arg_struct_01_01 (ref_val_struct_01_01)^M
../../binutils-gdb/gdb/riscv-tdep.c:2119: internal-error: void 
riscv_call_arg_struct(riscv_arg_info*, riscv_call_info*): Assertion 
`TYPE_LENGTH (ainfo->type) == TYPE_LENGTH (sinfo.field_type (0))' failed.^M
A problem internal to GDB has been detected,^M
further debugging may prove unreliable.^M
Quit this debugging session? (y or n) FAIL: 
gdb.base/infcall-nested-structs.exp: l=c++: types-tfc: p/d 
check_arg_struct_01_01 (ref_val_struct_01_01) (GDB internal error)

Jim

Re: GDB 8.2.90 available for testing

[Andrew Burgess]

* Jim Wilson <jimw@sifive.com> [2019-02-27 14:05:23 -0800]:

> On 2/26/19 9:51 PM, Joel Brobecker wrote:
> > I have just finished creating the gdb-8.2.90 pre-release.
> > Please give it a test if you can and report any problems you might find.
> 
> On a riscv64-linux system (HiFive Unleashed w/ Fedora Core 29), without an
> Ada compiler in my path, I get
> 
>                 === gdb Summary ===
> 
> # of expected passes            50264
> # of unexpected failures        2297
> # of expected failures          57
> # of unknown successes          3
> # of known failures             79
> # of untested testcases         132
> # of unresolved testcases       110
> # of unsupported tests          359
> 
> which looks OK.  There was no riscv-linux native support in gdb-8.2 so this
> is a major improvement over the last one.  I did used to have better results
> though.  I'm seeing
> 
> hifiveu017:1040$ grep ^FAIL gdb.sum | grep infcall | wc
>     878    8059  103059
> 
> and I think that these mostly used to work, since I used to have about 1500
> failures.  I think something in the last 3 months broke this, but I haven't
> had time to track it down.  I see stuff like this in the log files
> 
> p/d check_arg_struct_01_01 (ref_val_struct_01_01)^M
> ../../binutils-gdb/gdb/riscv-tdep.c:2119: internal-error: void
> riscv_call_arg_struct(riscv_arg_info*, riscv_call_info*): Assertion
> `TYPE_LENGTH (ainfo->type) == TYPE_LENGTH (sinfo.field_type (0))' failed.^M
> A problem internal to GDB has been detected,^M
> further debugging may prove unreliable.^M
> Quit this debugging session? (y or n) FAIL:
> gdb.base/infcall-nested-structs.exp: l=c++: types-tfc: p/d
> check_arg_struct_01_01 (ref_val_struct_01_01) (GDB internal error)

I don't think these are regressions, just new tests that expose a bug
that was present since the initial upstreaming.  The issue relates to
the difference in size between an empty struct in gcc and g++ (0 vs 1
respectively), and how this effects argument passing.

I have a fix in progress, but it will probably be a couple of days
before its ready to post.  I guess we can back-port to 8.3 if I get
the patch ready soon.  I'm not too worried if it doesn't make it in
though, if feels like an edge case.

Thanks,
Andrew

Re: GDB 8.2.90 available for testing

[Alan Hayward]

> On 4 Mar 2019, at 11:14, Andrew Burgess <andrew.burgess@embecosm.com> wrote:
> 
> * Jim Wilson <jimw@sifive.com> [2019-02-27 14:05:23 -0800]:
> 
>> On 2/26/19 9:51 PM, Joel Brobecker wrote:
>>> I have just finished creating the gdb-8.2.90 pre-release.
>>> Please give it a test if you can and report any problems you might find.
>> 
>> On a riscv64-linux system (HiFive Unleashed w/ Fedora Core 29), without an
>> Ada compiler in my path, I get
>> 
>>                === gdb Summary ===
>> 
>> # of expected passes            50264
>> # of unexpected failures        2297
>> # of expected failures          57
>> # of unknown successes          3
>> # of known failures             79
>> # of untested testcases         132
>> # of unresolved testcases       110
>> # of unsupported tests          359
>> 
>> which looks OK.  There was no riscv-linux native support in gdb-8.2 so this
>> is a major improvement over the last one.  I did used to have better results
>> though.  I'm seeing
>> 
>> hifiveu017:1040$ grep ^FAIL gdb.sum | grep infcall | wc
>>    878    8059  103059
>> 
>> and I think that these mostly used to work, since I used to have about 1500
>> failures.  I think something in the last 3 months broke this, but I haven't
>> had time to track it down.  I see stuff like this in the log files
>> 
>> p/d check_arg_struct_01_01 (ref_val_struct_01_01)^M
>> ../../binutils-gdb/gdb/riscv-tdep.c:2119: internal-error: void
>> riscv_call_arg_struct(riscv_arg_info*, riscv_call_info*): Assertion
>> `TYPE_LENGTH (ainfo->type) == TYPE_LENGTH (sinfo.field_type (0))' failed.^M
>> A problem internal to GDB has been detected,^M
>> further debugging may prove unreliable.^M
>> Quit this debugging session? (y or n) FAIL:
>> gdb.base/infcall-nested-structs.exp: l=c++: types-tfc: p/d
>> check_arg_struct_01_01 (ref_val_struct_01_01) (GDB internal error)
> 
> I don't think these are regressions, just new tests that expose a bug
> that was present since the initial upstreaming.  The issue relates to
> the difference in size between an empty struct in gcc and g++ (0 vs 1
> respectively), and how this effects argument passing.

I can confirm that for infcall-nested-structs.exp - I expanded that test fairly
significantly. You're probably running into all the same issues I fixed for
AArch64.  There are some parts that fail for X86_64 too.

> 
> I have a fix in progress, but it will probably be a couple of days
> before its ready to post.  I guess we can back-port to 8.3 if I get
> the patch ready soon.  I'm not too worried if it doesn't make it in
> though, if feels like an edge case.
> 
> Thanks,
> Andrew

Re: GDB 8.2.90 available for testing

[Jim Wilson]

On Mon, Mar 4, 2019 at 3:14 AM Andrew Burgess
<andrew.burgess@embecosm.com> wrote:
> I don't think these are regressions, just new tests that expose a bug
> that was present since the initial upstreaming.

OK. New tests failing is not a problem.  I just wanted to make sure
something didn't break in the RISC-V target dependent support.

Jim

MinGW build of GDB 8.2.90 (was: GDB 8.2.90 available for testing)

[Eli Zaretskii]

> From: Joel Brobecker <brobecker@adacore.com>
> Date: Wed, 27 Feb 2019 09:51:12 +0400 (+04)
> 
> I have just finished creating the gdb-8.2.90 pre-release.
> It is available for download at the following location:
> 
>     ftp://sourceware.org/pub/gdb/snapshots/branch/gdb-8.2.90.tar.xz

Thanks, I've built this with mingw.org's MinGW and bumped into a few
problems.

First, the recent changes to support IPv6 caused compilation errors in
several files.  The problem is with several fragments such as this
one:

  #ifdef USE_WIN32API
  #include <winsock2.h>
  #include <wspiapi.h>
  #else
  ...

mingw.org's MinGW doesn't have wspiapi.h.  Moreover, the Microsoft
documentation indicates that to get prototypes of getaddrinfo,
freeaddrinfo, etc. one needs to include ws2tcpip.h (and not include
winsock2.h separately), see, for example,

  https://docs.microsoft.com/en-us/windows/desktop/api/ws2tcpip/nf-ws2tcpip-getaddrinfo

Sergio, can you tell why you used wspiapi.h instead?

Is it okay to push changes such as the one below, master and branch
(after filing a Bugzilla report)?

--- ./gdb/common/netstuff.c~0	2019-02-27 06:51:50.000000000 +0200
+++ ./gdb/common/netstuff.c	2019-02-28 08:56:07.511568800 +0200
@@ -21,8 +21,11 @@
 #include <algorithm>
 
 #ifdef USE_WIN32API
-#include <winsock2.h>
-#include <wspiapi.h>
+#if _WIN32_WINNT < 0x0501
+# undef _WIN32_WINNT
+# define _WIN32_WINNT 0x0501
+#endif
+#include <ws2tcpip.h>
 #else
 #include <netinet/in.h>
 #include <arpa/inet.h>

Note that one other side effect of the IPv6 support additions is that
on MS-Windows GDB will no longer run on versions older than XP, I
guess this is something that should be mentioned in NEWS?

The other two problems were minor warning, one in tui.c about an
unused variable 'cap' (it is not used on Windows) and another in
xml-syscall.c:

       CXX    xml-syscall.o
     xml-syscall.c: In function 'bool xml_list_syscalls_by_group(gdbarch*, const char*, std::vector<int>*)':
     xml-syscall.c:475:14: warning: types may not be defined in a for-range-declaration
	for (const struct syscall_desc *sysdesc : groupdesc->syscalls)
		   ^~~~~~

I solved the latter by removing "struct" from the declaration.  This
is with GCC 6.3.0; is that a GCC bug? is removing "struct" the right
solution?

Thanks.

Re: MinGW build of GDB 8.2.90

[Sergio Durigan Junior]

On Thursday, February 28 2019, Eli Zaretskii wrote:

>> From: Joel Brobecker <brobecker@adacore.com>
>> Date: Wed, 27 Feb 2019 09:51:12 +0400 (+04)
>> 
>> I have just finished creating the gdb-8.2.90 pre-release.
>> It is available for download at the following location:
>> 
>>     ftp://sourceware.org/pub/gdb/snapshots/branch/gdb-8.2.90.tar.xz
>
> Thanks, I've built this with mingw.org's MinGW and bumped into a few
> problems.
>
> First, the recent changes to support IPv6 caused compilation errors in
> several files.  The problem is with several fragments such as this
> one:
>
>   #ifdef USE_WIN32API
>   #include <winsock2.h>
>   #include <wspiapi.h>
>   #else
>   ...
>
> mingw.org's MinGW doesn't have wspiapi.h.  Moreover, the Microsoft
> documentation indicates that to get prototypes of getaddrinfo,
> freeaddrinfo, etc. one needs to include ws2tcpip.h (and not include
> winsock2.h separately), see, for example,
>
>   https://docs.microsoft.com/en-us/windows/desktop/api/ws2tcpip/nf-ws2tcpip-getaddrinfo
>
> Sergio, can you tell why you used wspiapi.h instead?

Thanks for the report, Eli.

To be honest, I don't remember where I found the information that
wspiapi.h needed to be used, but I do remember finding it somewhere on
the internet, and since I don't use Windows, I assumed it was correct.

However, and more importantly, I remember testing the whole patch by
compiling it using a mingw32 compiler on Fedora, and it was working
correctly.  In fact, we even have a mingw32 builder on our BuildBot
(running on Fedora), and it is still compiling GDB without problems:

  https://gdb-build.sergiodj.net/builders/Fedora-x86_64-w64-mingw32

So apparently this error is only triggered when you use mingw on
Windows...?  I don't know.

> Is it okay to push changes such as the one below, master and branch
> (after filing a Bugzilla report)?
>
> --- ./gdb/common/netstuff.c~0	2019-02-27 06:51:50.000000000 +0200
> +++ ./gdb/common/netstuff.c	2019-02-28 08:56:07.511568800 +0200
> @@ -21,8 +21,11 @@
>  #include <algorithm>
>  
>  #ifdef USE_WIN32API
> -#include <winsock2.h>
> -#include <wspiapi.h>
> +#if _WIN32_WINNT < 0x0501
> +# undef _WIN32_WINNT
> +# define _WIN32_WINNT 0x0501
> +#endif
> +#include <ws2tcpip.h>
>  #else
>  #include <netinet/in.h>
>  #include <arpa/inet.h>

As I said, I don't use Windows and don't understand the system, but if
these changes fix the problem for you, I'd say they're justified and
should be pushed (even though I don't understand the "if
_WIN32_WINNT..." part).

> Note that one other side effect of the IPv6 support additions is that
> on MS-Windows GDB will no longer run on versions older than XP, I
> guess this is something that should be mentioned in NEWS?

I confess I did not know that.  If that's the case, then we should
indeed notify the users via the NEWS file, IMO.

> The other two problems were minor warning, one in tui.c about an
> unused variable 'cap' (it is not used on Windows) and another in
> xml-syscall.c:
>
>        CXX    xml-syscall.o
>      xml-syscall.c: In function 'bool xml_list_syscalls_by_group(gdbarch*, const char*, std::vector<int>*)':
>      xml-syscall.c:475:14: warning: types may not be defined in a for-range-declaration
> 	for (const struct syscall_desc *sysdesc : groupdesc->syscalls)
> 		   ^~~~~~
>
> I solved the latter by removing "struct" from the declaration.  This
> is with GCC 6.3.0; is that a GCC bug? is removing "struct" the right
> solution?

Yeah, this is the right thing to do.  I remember having to do this a few
times, and seeing other patches doing the same.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: MinGW build of GDB 8.2.90

[LRN]

[-- Attachment #1.1: Type: text/plain, Size: 1543 bytes --]

On 28.02.2019 21:55, Sergio Durigan Junior wrote:
> On Thursday, February 28 2019, Eli Zaretskii wrote:
>> On Wed, 27 Feb 2019 09:51:12 +0400 (+04) Joel Brobecker wrote:
>>>
>>> I have just finished creating the gdb-8.2.90 pre-release.
>>> It is available for download at the following location:
>>>
>>>     ftp://sourceware.org/pub/gdb/snapshots/branch/gdb-8.2.90.tar.xz
>>
>> Thanks, I've built this with mingw.org's MinGW and bumped into a few
>> problems.
>>
>> First, the recent changes to support IPv6 caused compilation errors in
>> several files.  The problem is with several fragments such as this
>> one:
>>
>>   #ifdef USE_WIN32API
>>   #include <winsock2.h>
>>   #include <wspiapi.h>
>>   #else
>>   ...
>>
>> mingw.org's MinGW doesn't have wspiapi.h.  Moreover, the Microsoft
>> documentation indicates that to get prototypes of getaddrinfo,
>> freeaddrinfo, etc. one needs to include ws2tcpip.h (and not include
>> winsock2.h separately)
> However, and more importantly, I remember testing the whole patch by
> compiling it using a mingw32 compiler on Fedora, and it was working
> correctly.  In fact, we even have a mingw32 builder on our BuildBot
> (running on Fedora), and it is still compiling GDB without problems:
> 
>   https://gdb-build.sergiodj.net/builders/Fedora-x86_64-w64-mingw32
> 
> So apparently this error is only triggered when you use mingw on
> Windows...?  I don't know.
> 

Obviously, Fedora cross-compiler is mingw-w64, not mingw.org. Big difference,
feature-wise.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: MinGW build of GDB 8.2.90

[Eli Zaretskii]

> From: Sergio Durigan Junior <sergiodj@redhat.com>
> Cc: Joel Brobecker <brobecker@adacore.com>,  gdb-patches@sourceware.org
> Date: Thu, 28 Feb 2019 13:55:27 -0500
> 
> However, and more importantly, I remember testing the whole patch by
> compiling it using a mingw32 compiler on Fedora, and it was working
> correctly.  In fact, we even have a mingw32 builder on our BuildBot
> (running on Fedora), and it is still compiling GDB without problems:
> 
>   https://gdb-build.sergiodj.net/builders/Fedora-x86_64-w64-mingw32
> 
> So apparently this error is only triggered when you use mingw on
> Windows...?  I don't know.

No, the problem is that there are two flavors of MinGW, and I used the
other one.

> As I said, I don't use Windows and don't understand the system, but if
> these changes fix the problem for you, I'd say they're justified and
> should be pushed (even though I don't understand the "if
> _WIN32_WINNT..." part).

For the record, the _WIN32_WINNT part is because mingw.org's MinGW by
default defines _WIN32_WINNT to target older versions of Windows,
which don't support getaddrinfo, and the Windows API headers then mask
the prototypes of those functions.

> > Note that one other side effect of the IPv6 support additions is that
> > on MS-Windows GDB will no longer run on versions older than XP, I
> > guess this is something that should be mentioned in NEWS?
> 
> I confess I did not know that.  If that's the case, then we should
> indeed notify the users via the NEWS file, IMO.

OK, will do.

> >        CXX    xml-syscall.o
> >      xml-syscall.c: In function 'bool xml_list_syscalls_by_group(gdbarch*, const char*, std::vector<int>*)':
> >      xml-syscall.c:475:14: warning: types may not be defined in a for-range-declaration
> > 	for (const struct syscall_desc *sysdesc : groupdesc->syscalls)
> > 		   ^~~~~~
> >
> > I solved the latter by removing "struct" from the declaration.  This
> > is with GCC 6.3.0; is that a GCC bug? is removing "struct" the right
> > solution?
> 
> Yeah, this is the right thing to do.  I remember having to do this a few
> times, and seeing other patches doing the same.

OK, will do that as well.

Thanks for the feedback.

Re: MinGW build of GDB 8.2.90

[Sergio Durigan Junior]

On Thursday, February 28 2019, Eli Zaretskii wrote:

>> From: Sergio Durigan Junior <sergiodj@redhat.com>
>> Cc: Joel Brobecker <brobecker@adacore.com>,  gdb-patches@sourceware.org
>> Date: Thu, 28 Feb 2019 13:55:27 -0500
>> 
>> However, and more importantly, I remember testing the whole patch by
>> compiling it using a mingw32 compiler on Fedora, and it was working
>> correctly.  In fact, we even have a mingw32 builder on our BuildBot
>> (running on Fedora), and it is still compiling GDB without problems:
>> 
>>   https://gdb-build.sergiodj.net/builders/Fedora-x86_64-w64-mingw32
>> 
>> So apparently this error is only triggered when you use mingw on
>> Windows...?  I don't know.
>
> No, the problem is that there are two flavors of MinGW, and I used the
> other one.

Understood.

>> As I said, I don't use Windows and don't understand the system, but if
>> these changes fix the problem for you, I'd say they're justified and
>> should be pushed (even though I don't understand the "if
>> _WIN32_WINNT..." part).
>
> For the record, the _WIN32_WINNT part is because mingw.org's MinGW by
> default defines _WIN32_WINNT to target older versions of Windows,
> which don't support getaddrinfo, and the Windows API headers then mask
> the prototypes of those functions.

Thanks for the explanation.

>> > Note that one other side effect of the IPv6 support additions is that
>> > on MS-Windows GDB will no longer run on versions older than XP, I
>> > guess this is something that should be mentioned in NEWS?
>> 
>> I confess I did not know that.  If that's the case, then we should
>> indeed notify the users via the NEWS file, IMO.
>
> OK, will do.
>
>> >        CXX    xml-syscall.o
>> >      xml-syscall.c: In function 'bool xml_list_syscalls_by_group(gdbarch*, const char*, std::vector<int>*)':
>> >      xml-syscall.c:475:14: warning: types may not be defined in a for-range-declaration
>> > 	for (const struct syscall_desc *sysdesc : groupdesc->syscalls)
>> > 		   ^~~~~~
>> >
>> > I solved the latter by removing "struct" from the declaration.  This
>> > is with GCC 6.3.0; is that a GCC bug? is removing "struct" the right
>> > solution?
>> 
>> Yeah, this is the right thing to do.  I remember having to do this a few
>> times, and seeing other patches doing the same.
>
> OK, will do that as well.
>
> Thanks for the feedback.

Thank you for taking care of it.

Just as a reminder, these changes need to be pushed to origin/master as
well.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: MinGW build of GDB 8.2.90

[Eli Zaretskii]

> From: Sergio Durigan Junior <sergiodj@redhat.com>
> Cc: brobecker@adacore.com,  gdb-patches@sourceware.org
> Date: Thu, 28 Feb 2019 15:17:02 -0500
> 
> Just as a reminder, these changes need to be pushed to origin/master as
> well.

Yes, I usually push to master and then cherry-pick to the branch.  I
believe this is the procedure?

Re: MinGW build of GDB 8.2.90

[Sergio Durigan Junior]

On Thursday, February 28 2019, Eli Zaretskii wrote:

>> From: Sergio Durigan Junior <sergiodj@redhat.com>
>> Cc: brobecker@adacore.com,  gdb-patches@sourceware.org
>> Date: Thu, 28 Feb 2019 15:17:02 -0500
>> 
>> Just as a reminder, these changes need to be pushed to origin/master as
>> well.
>
> Yes, I usually push to master and then cherry-pick to the branch.  I
> believe this is the procedure?

Sure, that works.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Joel Brobecker <brobecker@adacore.com>
> Date: Wed, 27 Feb 2019 09:51:12 +0400 (+04)
> 
> I have just finished creating the gdb-8.2.90 pre-release.
> It is available for download at the following location:
> 
>     ftp://sourceware.org/pub/gdb/snapshots/branch/gdb-8.2.90.tar.xz

This pretest shows a regression in the TUI interface: stepping through
the code of the debuggee with "next" overwrites the right edge of the
frame of the source window on those lines which we display in reverse
video as we step.  Does anyone else see this, or is this specific to
Windows?

I'd like this fixed before 8.3 is released, so if no one else sees
this, I will look into debugging the problem.

Re: GDB 8.2.90 available for testing

[Pedro Alves]

On 02/28/2019 06:33 PM, Eli Zaretskii wrote:
>> From: Joel Brobecker <brobecker@adacore.com>
>> Date: Wed, 27 Feb 2019 09:51:12 +0400 (+04)
>>
>> I have just finished creating the gdb-8.2.90 pre-release.
>> It is available for download at the following location:
>>
>>     ftp://sourceware.org/pub/gdb/snapshots/branch/gdb-8.2.90.tar.xz
> 
> This pretest shows a regression in the TUI interface: stepping through
> the code of the debuggee with "next" overwrites the right edge of the
> frame of the source window on those lines which we display in reverse
> video as we step.  Does anyone else see this, or is this specific to
> Windows?

I see that on GNU/Linux as well (Fedora 27).

I see other issues in the TUI.  Here they are:

#1 - Run a program to main, so that the source window displays the source.
  
  a) Press the "up" key.  That manages to actually move the cursor to the line above.
  
  b) alternatively, enter a command, like "(gdb) p 1\n", then press up.
     You'll see:
   
  (top-gdb) p 1
  $1 = 1
  (top-gdb) 16    in /home/pedro/gdb/src/gdb/gdb.c
  (top-gdb) 

  with the cursor sitting in the "16" above.

#2 - Run a program to main, so that the source window displays the source.

  - Press "up", and note that despite the nasty effects of #1 above, it
    scrolls the source window successfully.

    However, the "down" key doesn't work at all.  It doesn't scroll down.

#3 - Start gdb with a program, and enable the TUI.  The source window
     displays "No Source Available".  Now type "list" in the TUI console.
     I just tried it now, and the first time, it printed the source name in
     the console window and didn't update/display the source in the source window.
     The second time, it displayed the source in the source window, and the third
     time it crashed GDB with an uncaught exception:

(top-gdb) list          in /home/pedro/gdb/src/gdb/gdb.c
(top-gdb) list
(top-gdb) list
(top-gdb) listterminate called after throwing an instance of 'std::out_of_range'
                                                                                  what():  basic_string::substr: __pos (which is 18446744073709551615) > this->size() (which is 1500)
         Aborted (core dumped)

I was using GDB itself as the inferior program in the examples above, and
I'm building with Source Highlight, but I don't know whether that makes
a difference.

#4 - Enable the TUI, and make sure there's some source listed in
     the source window.

  - Try "set style enabled off".  Note styling in the source window
    is still enabled.

  - Press "Ctrl-L" to redraw screen.  Note styling in the source window
    is _still_ enabled.

> 
> I'd like this fixed before 8.3 is released, so if no one else sees
> this, I will look into debugging the problem.

Thanks.  Indeed, I think these TUI regressions should be fixed before
the release somehow.  Worse off, we get to disable styling in the TUI
for the release.

Thanks,
Pedro Alves

Re: GDB 8.2.90 available for testing

[Tom Tromey]

>>>>> "Pedro" == Pedro Alves <palves@redhat.com> writes:

>> I'd like this fixed before 8.3 is released, so if no one else sees
>> this, I will look into debugging the problem.

Pedro> Thanks.  Indeed, I think these TUI regressions should be fixed before
Pedro> the release somehow.  Worse off, we get to disable styling in the TUI
Pedro> for the release.

I'll take a look at them soon.

Tom

Re: GDB 8.2.90 available for testing

[Tom Tromey]

Pedro> #1 - Run a program to main, so that the source window displays the source.
  
Pedro>   a) Press the "up" key.  That manages to actually move the cursor to the line above.
  
I could not reproduce this.

Some of the odd behavior I do see seems to predate any of these changes.
With the system gdb on Fedora 28 and 29 I can:

(gdb) tui enable
(gdb) print 23
$1 = 23
(gdb) ... press Enter a a few times

Here the subsequent prompts aren't seen and the output just looks like

$2 = 23
$3 = 23
...

With git master this is worse and the output looks like:

(gdb) print 23
$1 = 23
(gdb) 
(gdb) $2 = 23

(gdb) $3 = 23

(gdb) $4 = 23


This I tracked down to the "nonl" patch.
I haven't yet debugged that one seriously.

Pedro> #3 - Start gdb with a program, and enable the TUI.  The source window
Pedro>      displays "No Source Available".  Now type "list" in the TUI console.
Pedro>      I just tried it now, and the first time, it printed the source name in
Pedro>      the console window and didn't update/display the source in the source window.
Pedro>      The second time, it displayed the source in the source window, and the third
Pedro>      time it crashed GDB with an uncaught exception:

I will look into the crash, but note that the "list" command is pretty
broken in the TUI.

https://sourceware.org/bugzilla/show_bug.cgi?id=18932

So it isn't super surprising to me that it doesn't work correctly in all
cases.

Tom

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Joel Brobecker <brobecker@adacore.com>,  gdb-patches@sourceware.org
> Date: Thu, 07 Mar 2019 15:44:46 -0700
> 
> Some of the odd behavior I do see seems to predate any of these changes.
> With the system gdb on Fedora 28 and 29 I can:
> 
> (gdb) tui enable
> (gdb) print 23
> $1 = 23
> (gdb) ... press Enter a a few times
> 
> Here the subsequent prompts aren't seen and the output just looks like
> 
> $2 = 23
> $3 = 23
> ...

The above is not odd, IMO, or at least I'm used to it when I use the
TUI: in that configuration GDB minimizes what it displays in the
command window.  For example, if you repeatedly type "n <Enter>", the
command window appears not to change at all, because the last command
overwrites the previous one on the screen.  I regard this as a
feature, which allows me to see the most of the interaction, without
wasting screen estate to show redundant content.

Re: GDB 8.2.90 available for testing

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> Here the subsequent prompts aren't seen and the output just looks like
>> 
>> $2 = 23
>> $3 = 23

Eli> The above is not odd, IMO, or at least I'm used to it when I use the
Eli> TUI: in that configuration GDB minimizes what it displays in the
Eli> command window.

Thanks, that makes sense.

I have played with the "nonl" code a bit and I can't seem to find an
approach that fixes this.  We may have to revert that patch :(

Tom

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com,  gdb-patches@sourceware.org
> Date: Fri, 08 Mar 2019 13:57:36 -0700
> 
> I have played with the "nonl" code a bit and I can't seem to find an
> approach that fixes this.  We may have to revert that patch :(

Maybe we should ask on bug-ncurses@gnu.org, perhaps Thomas Dickey will
have some suggestion for dealing with this?  If the only solution is
to revert, we have nothing to lose, I think.

Re: GDB 8.2.90 available for testing

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom Tromey <tom@tromey.com>
>> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com,  gdb-patches@sourceware.org
>> Date: Fri, 08 Mar 2019 13:57:36 -0700
>> 
>> I have played with the "nonl" code a bit and I can't seem to find an
>> approach that fixes this.  We may have to revert that patch :(

Eli> Maybe we should ask on bug-ncurses@gnu.org, perhaps Thomas Dickey will
Eli> have some suggestion for dealing with this?  If the only solution is
Eli> to revert, we have nothing to lose, I think.

The appended patch seems to work ok for me, at the cost of leaving the
gdb prompt in the TUI console window.

Tom

diff --git a/gdb/tui/tui-io.c b/gdb/tui/tui-io.c
index ef1e88507aa..f3eb2273e6b 100644
--- a/gdb/tui/tui-io.c
+++ b/gdb/tui/tui-io.c
@@ -631,7 +631,7 @@ gdb_wgetch (WINDOW *win)
      after the command.  So, if we read \r, emit a \r now, after nl
      mode has been re-entered, so that the output looks correct.  */
   if (r == '\r')
-    puts ("\r");
+    waddch (win, '\n');
   return r;
 }
 

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com,  gdb-patches@sourceware.org
> Date: Thu, 14 Mar 2019 11:32:40 -0600
> 
> The appended patch seems to work ok for me, at the cost of leaving the
> gdb prompt in the TUI console window.

Is this patch relative to the GDB 8.2.90 sources, or should it be
applied after removing the nonl parts?  I'd like to try using the
patch.

Thanks.

Re: GDB 8.2.90 available for testing

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> Is this patch relative to the GDB 8.2.90 sources, or should it be
Eli> applied after removing the nonl parts?  I'd like to try using the
Eli> patch.

It's relative to a clean checkout.  It should apply fine to either 8.3
or the master branch.

Tom

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com,  gdb-patches@sourceware.org
> Date: Thu, 14 Mar 2019 11:32:40 -0600
> 
> The appended patch seems to work ok for me, at the cost of leaving the
> gdb prompt in the TUI console window.
> 
> Tom
> 
> diff --git a/gdb/tui/tui-io.c b/gdb/tui/tui-io.c
> index ef1e88507aa..f3eb2273e6b 100644
> --- a/gdb/tui/tui-io.c
> +++ b/gdb/tui/tui-io.c
> @@ -631,7 +631,7 @@ gdb_wgetch (WINDOW *win)
>       after the command.  So, if we read \r, emit a \r now, after nl
>       mode has been re-entered, so that the output looks correct.  */
>    if (r == '\r')
> -    puts ("\r");
> +    waddch (win, '\n');
>    return r;
>  }

Works for me, and I think that "cost" is a small one to pay.

However, your change gave me an idea, and I think I came up with a bit
better fix, see below (the patch is again the current HEAD).  With
that, the original problem seems to be fixed, and we still don't
produce redundant lines in the command window.  WDYT?

--- gdb/tui/tui-io.c~6	2019-03-17 14:03:10.448181200 +0200
+++ gdb/tui/tui-io.c	2019-03-17 14:03:53.951772800 +0200
@@ -696,12 +696,6 @@ gdb_wgetch (WINDOW *win)
   nonl ();
   int r = wgetch (win);
   nl ();
-  /* In nonl mode, if the user types Enter, it will not be echoed
-     properly.  This will result in gdb output appearing immediately
-     after the command.  So, if we read \r, emit a \r now, after nl
-     mode has been re-entered, so that the output looks correct.  */
-  if (r == '\r')
-    puts ("\r");
   return r;
 }
 
@@ -928,7 +922,7 @@ tui_getc (FILE *fp)
 
   /* The \n must be echoed because it will not be printed by
      readline.  */
-  if (ch == '\n')
+  if (ch == '\n' || ch == '\r')
     {
       /* When hitting return with an empty input, gdb executes the last
          command.  If we emit a newline, this fills up the command window

Re: GDB 8.2.90 available for testing

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> However, your change gave me an idea, and I think I came up with a bit
Eli> better fix, see below (the patch is again the current HEAD).  With
Eli> that, the original problem seems to be fixed, and we still don't
Eli> produce redundant lines in the command window.  WDYT?

I think as long as it works, it is good.  Thanks.

Tom

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com,  gdb-patches@sourceware.org
> Date: Sun, 17 Mar 2019 11:31:27 -0600
> 
> >>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:
> 
> Eli> However, your change gave me an idea, and I think I came up with a bit
> Eli> better fix, see below (the patch is again the current HEAD).  With
> Eli> that, the original problem seems to be fixed, and we still don't
> Eli> produce redundant lines in the command window.  WDYT?
> 
> I think as long as it works, it is good.  Thanks.

So okay to push to both branches?

Re: GDB 8.2.90 available for testing

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom Tromey <tom@tromey.com>
>> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com,  gdb-patches@sourceware.org
>> Date: Sun, 17 Mar 2019 11:31:27 -0600
>> 
>> >>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:
>> 
Eli> However, your change gave me an idea, and I think I came up with a bit
Eli> better fix, see below (the patch is again the current HEAD).  With
Eli> that, the original problem seems to be fixed, and we still don't
Eli> produce redundant lines in the command window.  WDYT?
>> 
>> I think as long as it works, it is good.  Thanks.

Eli> So okay to push to both branches?

Yes, please.  Thanks.

Tom

Re: GDB 8.2.90 available for testing

[Eli Zaretskii]

> From: Tom Tromey <tom@tromey.com>
> Cc: Tom Tromey <tom@tromey.com>,  palves@redhat.com,  brobecker@adacore.com, 	gdb-patches@sourceware.org
> Date: Mon, 18 Mar 2019 08:13:30 -0600
> 
> >> I think as long as it works, it is good.  Thanks.
> 
> Eli> So okay to push to both branches?
> 
> Yes, please.  Thanks.

Thanks, done.

Re: GDB 8.2.90 available for testing

[Pedro Franco de Carvalho]

Joel Brobecker <brobecker@adacore.com> writes:

Hello,

There is a regression in the testsuite itself, triggered by errors being
raised from within gdb_test_multiple.  This looks like a dejagnu issue,
but it started happening due to a commit that was introduced for this
version, and it can cause the testsuite to hang.

The commit that triggers this is:

fe1a5cad302b5535030cdf62895e79512713d738
[gdb/testsuite] Log wait status on process no longer exists error

The issue is that this commit introduces a new "eof" block in
gdb_test_multiple.  It doesn't look incorrect to me, but dejagnu uses
this block as a "default" action when an error is raised from within the
commands inside a call to gdb_test_multiple, which means that it can be
executed even when there was no actual eof and the GDB process is still
running, so the wait introduced in the commit that tries to get the exit
status of GDB hangs forever, while GDB itself waits for input.

This only happens when there are internal testsuite errors (not testcase
failures).  I had noticed it before but the issue that caused the error
and triggered this problem was resolved.  I saw it again now when I ran
the testsuite on a system that restricts ptrace attach operations, which
causes an error in "gdb.multi/multi-term-settings.exp", which then hangs
the testsuite.

This can be reproduced more easily with a testcase such as:

gdb_start

gdb_test_multiple "show version" "show version" {
	    -re ".*" {
		error "forced error"
	    }
}

I'm not sure if there is a proper way to work around this in GDB, but it
would be useful so that the testsuite doesn't hang in these cases.

Adding an empty "default" block at the end of the expect body in
gdb_test_multiple doesn't work, because the dejagnu routine that parses
this body and selects the default block to execute after an error is
affected by the comments in the body (since they are also parsed).  This
is proc remote_expect (in dejagnu/lib/remote.exp).  In fact, there is
already a second eof block that isn't used due to this issue.

If this comment in gdb_test_multiple

# patterns below apply to any spawn id specified.

Is changed to

# The patterns below apply to any spawn id specified.

Then the second eof block is selected and there is no hang-up.

Any comment at that same place with an even-numbered of tokens also
works.  Of course, this is an ugly solution, and there could be other
weird side-effects.

Alternatively, the GDB commit that introduced this new eof block could
be reverted.  This would be unfortunate since it seems useful, but the
comment for proc remote_expect does specify that the eof/default/timeout
block is used as an error handler, just not which one.

Thanks!

--
Pedro Franco de Carvalho

[PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Alves]

Hi Pedro,

Thanks for the detailed report.

On 03/07/2019 05:42 PM, Pedro Franco de Carvalho wrote:

> I'm not sure if there is a proper way to work around this in GDB, but it
> would be useful so that the testsuite doesn't hang in these cases.

I spent a while trying to fix this, and I came up with the patch below.

WDYT?

From 17e8f28072cce040524eab7b85b8767764a54cd2 Mon Sep 17 00:00:00 2001
From: Pedro Alves <palves@redhat.com>
Date: Fri, 22 Mar 2019 11:33:19 +0000
Subject: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out

This commit fixes a regression in the testsuite itself, triggered by
errors being raised from within gdb_test_multiple, originally reported
by Pedro Franco de Carvalho's at
<https://sourceware.org/ml/gdb-patches/2019-03/msg00160.html>.  Parts
of the commit message are based on his report.

This started happening due to a commit that was introduced recently,
and it can cause the testsuite to hang.

The commit that triggers this is:

 fe1a5cad302b5535030cdf62895e79512713d738
 [gdb/testsuite] Log wait status on process no longer exists error

That commit introduces a new "eof" block in gdb_test_multiple.  That
is not incorrect itself, but dejagnu's remote_expect is picking that
block as the "default" action when an error is raised from within the
commands inside a call to gdb_test_multiple:

  # remote_expect works basically the same as standard expect, but it
  # also takes care of getting the file descriptor from the specified
  # host and also calling the timeout/eof/default section if there is an
  # error on the expect call.
  #
  proc remote_expect { board timeout args } {

I find that "feature" surprising, and I don't really know why it
exists, but this means that the eof section that remote_expect picks
as the error block can be executed even when there was no actual eof
and the GDB process is still running, so the wait introduced in the
commit that tries to get the exit status of GDB hangs forever, while
GDB itself waits for input.

This only happens when there are internal testsuite errors (not
testcase failures).  This can be reproduced easily with a testcase
such as:

  gdb_start
  gdb_test_multiple "show version" "show version" {
    -re ".*" {
       error "forced error"
    }
  }

I think that working around this in GDB is useful so that the
testsuite doesn't hang in these cases.

Adding an empty "default" block at the end of the expect body in
gdb_test_multiple doesn't work, because dejagnu gives preference to
"eof" blocks:

	    if { $x eq "eof" } {
		set save_next 1
	    } elseif { $x eq "default" || $x eq "timeout" } {
		if { $error_sect eq "" } {
		    set save_next 1
		}
	    }

And we do have "eof" blocks.  So we need to make sure that the last
"eof" block is safe to use as the default error block.  It's also
pedantically incorrect to print

 "ERROR: Process no longer exists"

which is what we'd get if the last eof block we have was selected
(more below on this).

So this commit solves this by appending an "eof" with an empty
spawn_id list, so that it won't ever match.

Now, why is the first "eof" block selected today as the error block,
instead of the last one?

The reason is that remote_expect, while parsing the body to select the
default block to execute after an error, is affected by the comments
in the body (since they are also parsed).

If this comment in gdb_test_multiple

 # patterns below apply to any spawn id specified.

is changed to

 # The patterns below apply to any spawn id specified.

then the second eof block is selected and there is no hang.

Any comment at that same place with an even-numbered of tokens also
works.

This is IMO a coincidence caused by how comments work in TCL.
Comments should only appear in places where a command can appear.  And
here, remote_expect is parsing a list of options, not commands, so
it's not unreasonable to not parse comments, similarly to how this:

  set a_list {
     an_element
     # another_element
  }

results in a list with three elements, not one element.

The fact that comments with an even number of tokens work is just a
coincidence of how remote_expect's little state machine is
implemented.

I thought we could solve this by stripping out comment lines in
gdb_expect, but I didn't find an easy way to do that.  Particularly, a
couple naive approaches I tried run into complications.  For example,
we have gdb_test calls with regular expressions that include sequences
like "\r\n#", and by the time we get to gdb_expect, the \r\n have
already been expanded to a real newline, so just splitting the whole
body at newline boundaries, looking for lines that start with #
results in incorrectly stripping out half of the gdb_text regexp.  I
think it's better (at least in this commit), to move the comments out
of the list, because it's much simpler and risk free.

gdb/ChangeLog:
yyyy-mm-dd  Pedro Alves  <palves@redhat.com>

	* lib/gdb.exp (gdb_test_multiple): Split appends to $code and
          move comments outside list.  Append '-i "" eof' section.
---
 gdb/testsuite/lib/gdb.exp | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/gdb/testsuite/lib/gdb.exp b/gdb/testsuite/lib/gdb.exp
index 6800c74187..4600d2c347 100644
--- a/gdb/testsuite/lib/gdb.exp
+++ b/gdb/testsuite/lib/gdb.exp
@@ -906,10 +906,13 @@ proc gdb_test_multiple { command message user_code } {
 	}
     }
     append code $processed_code
+
+    # Reset the spawn id, in case the processed code used -i.
     append code {
-	# Reset the spawn id, in case the processed code used -i.
 	-i "$gdb_spawn_id"
+    }
 
+    append code {
 	-re "Ending remote debugging.*$gdb_prompt $" {
 	    if ![isnative] then {
 		warning "Can`t communicate to remote target."
@@ -990,8 +993,10 @@ proc gdb_test_multiple { command message user_code } {
 	    }
 	    return -1
 	}
+    }
 
-	# Patterns below apply to any spawn id specified.
+    # Now patterns that apply to any spawn id specified.
+    append code {
 	-i $any_spawn_id
 	eof {
 	    perror "Process no longer exists"
@@ -1013,6 +1018,20 @@ proc gdb_test_multiple { command message user_code } {
 	}
     }
 
+    # remote_expect calls the eof section if there is an error on the
+    # expect call.  We already have "eof" sections above, and we don't
+    # want them to get called in that situation.  Since the last "eof"
+    # section becomes the error section, here we define another "eof"
+    # section, but with an empty spawn_id list, so that it won't ever
+    # match anything.
+    append code {
+	-i "" eof {
+	    # This comment is here because the eof section must not be
+	    # the empty string, otherwise remote_expect won't realize
+	    # it exists.
+	}
+    }
+
     set result 0
     set code [catch {gdb_expect $code} string]
     if {$code == 1} {
-- 
2.14.4

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Simon Marchi]

On 2019-03-22 08:39, Pedro Alves wrote:
> Hi Pedro,
> 
> Thanks for the detailed report.
> 
> On 03/07/2019 05:42 PM, Pedro Franco de Carvalho wrote:
> 
>> I'm not sure if there is a proper way to work around this in GDB, but 
>> it
>> would be useful so that the testsuite doesn't hang in these cases.
> 
> I spent a while trying to fix this, and I came up with the patch below.
> 
> WDYT?
> 
> From 17e8f28072cce040524eab7b85b8767764a54cd2 Mon Sep 17 00:00:00 2001
> From: Pedro Alves <palves@redhat.com>
> Date: Fri, 22 Mar 2019 11:33:19 +0000
> Subject: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors 
> out
> 
> This commit fixes a regression in the testsuite itself, triggered by
> errors being raised from within gdb_test_multiple, originally reported
> by Pedro Franco de Carvalho's at
> <https://sourceware.org/ml/gdb-patches/2019-03/msg00160.html>.  Parts
> of the commit message are based on his report.
> 
> This started happening due to a commit that was introduced recently,
> and it can cause the testsuite to hang.
> 
> The commit that triggers this is:
> 
>  fe1a5cad302b5535030cdf62895e79512713d738
>  [gdb/testsuite] Log wait status on process no longer exists error
> 
> That commit introduces a new "eof" block in gdb_test_multiple.  That
> is not incorrect itself, but dejagnu's remote_expect is picking that
> block as the "default" action when an error is raised from within the
> commands inside a call to gdb_test_multiple:
> 
>   # remote_expect works basically the same as standard expect, but it
>   # also takes care of getting the file descriptor from the specified
>   # host and also calling the timeout/eof/default section if there is 
> an
>   # error on the expect call.
>   #
>   proc remote_expect { board timeout args } {
> 
> I find that "feature" surprising, and I don't really know why it
> exists, but this means that the eof section that remote_expect picks
> as the error block can be executed even when there was no actual eof
> and the GDB process is still running, so the wait introduced in the
> commit that tries to get the exit status of GDB hangs forever, while
> GDB itself waits for input.
> 
> This only happens when there are internal testsuite errors (not
> testcase failures).  This can be reproduced easily with a testcase
> such as:
> 
>   gdb_start
>   gdb_test_multiple "show version" "show version" {
>     -re ".*" {
>        error "forced error"
>     }
>   }
> 
> I think that working around this in GDB is useful so that the
> testsuite doesn't hang in these cases.
> 
> Adding an empty "default" block at the end of the expect body in
> gdb_test_multiple doesn't work, because dejagnu gives preference to
> "eof" blocks:
> 
> 	    if { $x eq "eof" } {
> 		set save_next 1
> 	    } elseif { $x eq "default" || $x eq "timeout" } {
> 		if { $error_sect eq "" } {
> 		    set save_next 1
> 		}
> 	    }
> 
> And we do have "eof" blocks.  So we need to make sure that the last
> "eof" block is safe to use as the default error block.  It's also
> pedantically incorrect to print
> 
>  "ERROR: Process no longer exists"
> 
> which is what we'd get if the last eof block we have was selected
> (more below on this).
> 
> So this commit solves this by appending an "eof" with an empty
> spawn_id list, so that it won't ever match.
> 
> Now, why is the first "eof" block selected today as the error block,
> instead of the last one?
> 
> The reason is that remote_expect, while parsing the body to select the
> default block to execute after an error, is affected by the comments
> in the body (since they are also parsed).
> 
> If this comment in gdb_test_multiple
> 
>  # patterns below apply to any spawn id specified.
> 
> is changed to
> 
>  # The patterns below apply to any spawn id specified.
> 
> then the second eof block is selected and there is no hang.
> 
> Any comment at that same place with an even-numbered of tokens also
> works.
> 
> This is IMO a coincidence caused by how comments work in TCL.
> Comments should only appear in places where a command can appear.  And
> here, remote_expect is parsing a list of options, not commands, so
> it's not unreasonable to not parse comments, similarly to how this:
> 
>   set a_list {
>      an_element
>      # another_element
>   }
> 
> results in a list with three elements, not one element.
> 
> The fact that comments with an even number of tokens work is just a
> coincidence of how remote_expect's little state machine is
> implemented.
> 
> I thought we could solve this by stripping out comment lines in
> gdb_expect, but I didn't find an easy way to do that.  Particularly, a
> couple naive approaches I tried run into complications.  For example,
> we have gdb_test calls with regular expressions that include sequences
> like "\r\n#", and by the time we get to gdb_expect, the \r\n have
> already been expanded to a real newline, so just splitting the whole
> body at newline boundaries, looking for lines that start with #
> results in incorrectly stripping out half of the gdb_text regexp.  I
> think it's better (at least in this commit), to move the comments out
> of the list, because it's much simpler and risk free.

Wow, this is top-notch TCL/expect/DejaGnu investigation skills.

I agree with just moving the comments out, since that's the correct 
thing to do.  Trying to strip them programmatically just adds some dark 
magic that can potentially introduce more weird behaviors.

But we need to remember this when writing tests, should we have an entry 
in the testcase wiki page?

https://sourceware.org/gdb/wiki/GDBTestcaseCookbook

Simon

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Alves]

On 03/22/2019 02:42 PM, Simon Marchi wrote:
> On 2019-03-22 08:39, Pedro Alves wrote:
>> Hi Pedro,
>>
>> Thanks for the detailed report.
>>
>> On 03/07/2019 05:42 PM, Pedro Franco de Carvalho wrote:
>>
>>> I'm not sure if there is a proper way to work around this in GDB, but it
>>> would be useful so that the testsuite doesn't hang in these cases.
>>
>> I spent a while trying to fix this, and I came up with the patch below.
>>
>> WDYT?
>>
>> From 17e8f28072cce040524eab7b85b8767764a54cd2 Mon Sep 17 00:00:00 2001
>> From: Pedro Alves <palves@redhat.com>
>> Date: Fri, 22 Mar 2019 11:33:19 +0000
>> Subject: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out
>>
>> This commit fixes a regression in the testsuite itself, triggered by
>> errors being raised from within gdb_test_multiple, originally reported
>> by Pedro Franco de Carvalho's at
>> <https://sourceware.org/ml/gdb-patches/2019-03/msg00160.html>.  Parts
>> of the commit message are based on his report.
>>
>> This started happening due to a commit that was introduced recently,
>> and it can cause the testsuite to hang.
>>
>> The commit that triggers this is:
>>
>>  fe1a5cad302b5535030cdf62895e79512713d738
>>  [gdb/testsuite] Log wait status on process no longer exists error
>>
>> That commit introduces a new "eof" block in gdb_test_multiple.  That
>> is not incorrect itself, but dejagnu's remote_expect is picking that
>> block as the "default" action when an error is raised from within the
>> commands inside a call to gdb_test_multiple:
>>
>>   # remote_expect works basically the same as standard expect, but it
>>   # also takes care of getting the file descriptor from the specified
>>   # host and also calling the timeout/eof/default section if there is an
>>   # error on the expect call.
>>   #
>>   proc remote_expect { board timeout args } {
>>
>> I find that "feature" surprising, and I don't really know why it
>> exists, but this means that the eof section that remote_expect picks
>> as the error block can be executed even when there was no actual eof
>> and the GDB process is still running, so the wait introduced in the
>> commit that tries to get the exit status of GDB hangs forever, while
>> GDB itself waits for input.
>>
>> This only happens when there are internal testsuite errors (not
>> testcase failures).  This can be reproduced easily with a testcase
>> such as:
>>
>>   gdb_start
>>   gdb_test_multiple "show version" "show version" {
>>     -re ".*" {
>>        error "forced error"
>>     }
>>   }
>>
>> I think that working around this in GDB is useful so that the
>> testsuite doesn't hang in these cases.
>>
>> Adding an empty "default" block at the end of the expect body in
>> gdb_test_multiple doesn't work, because dejagnu gives preference to
>> "eof" blocks:
>>
>>         if { $x eq "eof" } {
>>         set save_next 1
>>         } elseif { $x eq "default" || $x eq "timeout" } {
>>         if { $error_sect eq "" } {
>>             set save_next 1
>>         }
>>         }
>>
>> And we do have "eof" blocks.  So we need to make sure that the last
>> "eof" block is safe to use as the default error block.  It's also
>> pedantically incorrect to print
>>
>>  "ERROR: Process no longer exists"
>>
>> which is what we'd get if the last eof block we have was selected
>> (more below on this).
>>
>> So this commit solves this by appending an "eof" with an empty
>> spawn_id list, so that it won't ever match.
>>
>> Now, why is the first "eof" block selected today as the error block,
>> instead of the last one?
>>
>> The reason is that remote_expect, while parsing the body to select the
>> default block to execute after an error, is affected by the comments
>> in the body (since they are also parsed).
>>
>> If this comment in gdb_test_multiple
>>
>>  # patterns below apply to any spawn id specified.
>>
>> is changed to
>>
>>  # The patterns below apply to any spawn id specified.
>>
>> then the second eof block is selected and there is no hang.
>>
>> Any comment at that same place with an even-numbered of tokens also
>> works.
>>
>> This is IMO a coincidence caused by how comments work in TCL.
>> Comments should only appear in places where a command can appear.  And
>> here, remote_expect is parsing a list of options, not commands, so
>> it's not unreasonable to not parse comments, similarly to how this:
>>
>>   set a_list {
>>      an_element
>>      # another_element
>>   }
>>
>> results in a list with three elements, not one element.
>>
>> The fact that comments with an even number of tokens work is just a
>> coincidence of how remote_expect's little state machine is
>> implemented.
>>
>> I thought we could solve this by stripping out comment lines in
>> gdb_expect, but I didn't find an easy way to do that.  Particularly, a
>> couple naive approaches I tried run into complications.  For example,
>> we have gdb_test calls with regular expressions that include sequences
>> like "\r\n#", and by the time we get to gdb_expect, the \r\n have
>> already been expanded to a real newline, so just splitting the whole
>> body at newline boundaries, looking for lines that start with #
>> results in incorrectly stripping out half of the gdb_text regexp.  I
>> think it's better (at least in this commit), to move the comments out
>> of the list, because it's much simpler and risk free.
> 
> Wow, this is top-notch TCL/expect/DejaGnu investigation skills.

Thanks.  Team work here, the initial analysis came from
Pedro Franco de Carvalho.

> I agree with just moving the comments out, since that's the correct thing to do.  Trying to strip them programmatically just adds some dark magic that can potentially introduce more weird behaviors.
> 
> But we need to remember this when writing tests, should we have an entry in the testcase wiki page?
> 
> https://sourceware.org/gdb/wiki/GDBTestcaseCookbook

That might be a good idea.

I'm merging the patch to master now.

Thanks,
Pedro Alves

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Franco de Carvalho]

Pedro Alves <palves@redhat.com> writes:

Hello,

> I spent a while trying to fix this, and I came up with the patch below.
>
> WDYT?

Makes sense to me.

> So this commit solves this by appending an "eof" with an empty
> spawn_id list, so that it won't ever match.

This is a clever solution, but just to be sure, can we actually rely on
this behavior when the list is empty?  This did work when I tested it,
but could some Expect version conceivably do something else like
returning an error when parsing a body with a "-i" that uses an empty
list?

> gdb/ChangeLog:
> yyyy-mm-dd  Pedro Alves  <palves@redhat.com>

I think you meant gdb/testsuite/ChangeLog

Thanks for the patch!

--
Pedro Franco de Carvalho

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Alves]

On 03/22/2019 08:44 PM, Pedro Franco de Carvalho wrote:
> Pedro Alves <palves@redhat.com> writes:
> 
> Hello,
> 
>> I spent a while trying to fix this, and I came up with the patch below.
>>
>> WDYT?
> 
> Makes sense to me.
> 
>> So this commit solves this by appending an "eof" with an empty
>> spawn_id list, so that it won't ever match.
> 
> This is a clever solution, but just to be sure, can we actually rely on
> this behavior when the list is empty?  This did work when I tested it,
> but could some Expect version conceivably do something else like
> returning an error when parsing a body with a "-i" that uses an empty
> list?

I'd think not, but I can't predict the future so I can't be sure,
of course.

I looked at expect's source code a little and (in my untrained eyes)
it didn't seem like there would be a reason for it not to work,
since -i's argument is just read as a list.  I think a reasonable use
case would be to allow an empty variable as spawn id, so that
some patterns would be disabled depending on the variable being
empty or not (e.g., "-i $my_spawn_id_if_any").

When I thought of this trick, I was going to use an indirect
spawn id:

 The -i flag may also name a global variable in which case the variable
 is read for a list of spawn ids. The variable is reread whenever it changes.
 This provides a way of changing the I/O source while the command is in
 execution. Spawn ids provided this way are called "indirect" spawn ids. 

and having that global variable be an empty list.  That came to mind
since I had experience with using indirect spawn ids in another case
when I needed an empty spawn id list:

    # Use an indirect spawn id list, and remove the inferior spawn id
    # from the expected output as soon as it matches, in case
    # $inferior_pattern happens to be a prefix of the resulting full
    # gdb pattern below (e.g., "\r\n").
    global gdb_test_stdio_spawn_id_list
    set gdb_test_stdio_spawn_id_list "$inferior_spawn_id"

And then I realized the to-the-point '-i ""' would work just as well.

I think that if this stops working, likely either empty variable or
indirect spawn id pointing at an empty list would still keep working,
so we would likely be able to switch to one of those.

Maybe meanwhile, we could ask on the dejagnu list what's the purpose
of this picking one of eof/timeout/default as an error block, see if
that could be removed.  I'd guess that dejagnu would change faster
than expect here, or maybe I should say, less slowly.  :-)

Thanks,
Pedro Alves

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Franco de Carvalho]

Pedro Alves <palves@redhat.com> writes:

> On 03/22/2019 08:44 PM, Pedro Franco de Carvalho wrote:
>> Pedro Alves <palves@redhat.com> writes:
>> 
>> Hello,
>> 
>>> I spent a while trying to fix this, and I came up with the patch below.
>>>
>>> WDYT?
>> 
>> Makes sense to me.
>> 
>>> So this commit solves this by appending an "eof" with an empty
>>> spawn_id list, so that it won't ever match.
>> 
>> This is a clever solution, but just to be sure, can we actually rely on
>> this behavior when the list is empty?  This did work when I tested it,
>> but could some Expect version conceivably do something else like
>> returning an error when parsing a body with a "-i" that uses an empty
>> list?
>
> I'd think not, but I can't predict the future so I can't be sure,
> of course.
>
> I looked at expect's source code a little and (in my untrained eyes)
> it didn't seem like there would be a reason for it not to work,
> since -i's argument is just read as a list.  I think a reasonable use
> case would be to allow an empty variable as spawn id, so that
> some patterns would be disabled depending on the variable being
> empty or not (e.g., "-i $my_spawn_id_if_any").
>
> When I thought of this trick, I was going to use an indirect
> spawn id:
>
>  The -i flag may also name a global variable in which case the variable
>  is read for a list of spawn ids. The variable is reread whenever it changes.
>  This provides a way of changing the I/O source while the command is in
>  execution. Spawn ids provided this way are called "indirect" spawn ids. 
>
> and having that global variable be an empty list.  That came to mind
> since I had experience with using indirect spawn ids in another case
> when I needed an empty spawn id list:
>
>     # Use an indirect spawn id list, and remove the inferior spawn id
>     # from the expected output as soon as it matches, in case
>     # $inferior_pattern happens to be a prefix of the resulting full
>     # gdb pattern below (e.g., "\r\n").
>     global gdb_test_stdio_spawn_id_list
>     set gdb_test_stdio_spawn_id_list "$inferior_spawn_id"
>
> And then I realized the to-the-point '-i ""' would work just as well.
>
> I think that if this stops working, likely either empty variable or
> indirect spawn id pointing at an empty list would still keep working,
> so we would likely be able to switch to one of those.

Ok! Makes sense to me, thanks for the explanation.

> Maybe meanwhile, we could ask on the dejagnu list what's the purpose
> of this picking one of eof/timeout/default as an error block, see if
> that could be removed.  I'd guess that dejagnu would change faster
> than expect here, or maybe I should say, less slowly.  :-)

I can do that if you want, and let them know about the comment tokens
affecting the error block selection too.

>
> Thanks,
> Pedro Alves

Thanks for fixing this!

--
Pedro Franco de Carvalho

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Alves]

On 03/25/2019 07:43 PM, Pedro Franco de Carvalho wrote:
> Pedro Alves <palves@redhat.com> writes:

>> Maybe meanwhile, we could ask on the dejagnu list what's the purpose
>> of this picking one of eof/timeout/default as an error block, see if
>> that could be removed.  I'd guess that dejagnu would change faster
>> than expect here, or maybe I should say, less slowly.  :-)
> 
> I can do that if you want, and let them know about the comment tokens
> affecting the error block selection too.

I've done that today, after digging into expect's sources, to see how
it manages to handle comments.

[Why does remote_expect call the timeout/eof/default section if there is an error on the expect call?]
http://lists.gnu.org/archive/html/dejagnu/2019-03/msg00010.html

[remote_expect gets confused by comments, unlike raw expect]
http://lists.gnu.org/archive/html/dejagnu/2019-03/msg00011.html

Thanks,
Pedro Alves

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Franco de Carvalho]

Pedro Alves <palves@redhat.com> writes:

> I've done that today, after digging into expect's sources, to see how
> it manages to handle comments.

Thanks!

Should this patch also be included in the 8.3 branch?  I haven't seen it
there, but the patch that triggered the issue is.

--
Pedro Franco de Carvalho

Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Joel Brobecker]

> > I've done that today, after digging into expect's sources, to see how
> > it manages to handle comments.
> 
> Should this patch also be included in the 8.3 branch?  I haven't seen it
> there, but the patch that triggered the issue is.

Pedro and I agreed it was safe for 8.3 at:
    https://www.sourceware.org/ml/gdb-patches/2019-03/msg00619.html
    https://www.sourceware.org/ml/gdb-patches/2019-03/msg00644.html

So I re-tested the patch in the gdb-8.3-branch, and then pushed it.
    https://www.sourceware.org/ml/gdb-cvs/2019-03/msg00192.html

-- 
Joel