[Bug gdb/27681] New: FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[Bug gdb/27681] New: FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

            Bug ID: 27681
           Summary: FAIL: gdb.base/help.exp: apropos \(print[^[
                    bsiedf\".-]\) (timeout)
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: gdb
          Assignee: unassigned at sourceware dot org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

...
(gdb) PASS: gdb.base/help.exp: help gotcha
apropos \(print[^[ bsiedf\".-]\)^M
FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)
apropos handle signal^M
FAIL: gdb.base/help.exp: apropos handle signal (timeout)
apropos apropos^M
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #1 from Tom de Vries <vries at gcc dot gnu.org> ---
^C
Thread 1 "gdb" received signal SIGINT, Interrupt.
futex_wait (private=0, expected=2, futex_word=0x1840b78)
    at ../sysdeps/nptl/futex-internal.h:146
146       int err = lll_futex_timed_wait (futex_word, expected, NULL, private);
(gdb) bt
#0  futex_wait (private=0, expected=2, futex_word=0x1840b78)
    at ../sysdeps/nptl/futex-internal.h:146
#1  __lll_lock_wait_private (futex=futex@entry=0x1840b78) at
./lowlevellock.c:35
#2  0x00007ffff73ef84f in re_search_stub (bufp=0x7fffffffd4c0, string=0xf8d01f
"+", 
    length=1, start=0, range=<optimized out>, stop=1, regs=0x0, ret_len=false)
    at regexec.c:390
#3  0x00007ffff73efe54 in __re_search (bufp=<optimized out>, string=<optimized
out>, 
    length=<optimized out>, start=<optimized out>, range=<optimized out>, 
    regs=<optimized out>) at regexec.c:289
#4  0x000000000070af57 in compiled_regex::search (this=0x7fffffffd4c0, 
    string=0xf8d01f "+", length=1, start=0, range=1, regs=0x0)
    at /data/gdb_versions/devel/src/gdb/gdb_regex.c:56
#5  0x00000000005877d9 in apropos_cmd (stream=0x18bd490, commandlist=0x18289d0, 
    verbose=false, regex=..., prefix=0xe7aab1 "")
    at /data/gdb_versions/devel/src/gdb/cli/cli-decode.c:1167
#6  0x0000000000580cb9 in apropos_command (
    arg=0x7fffffffe1ab "\\(print[^[ bsiedf\".-]\\)", from_tty=0)
    at /data/gdb_versions/devel/src/gdb/cli/cli-cmds.c:1639
#7  0x0000000000585e06 in do_const_cfunc (c=0x1829360, 
    args=0x7fffffffe1ab "\\(print[^[ bsiedf\".-]\\)", from_tty=0)
    at /data/gdb_versions/devel/src/gdb/cli/cli-decode.c:101
#8  0x0000000000589bd5 in cmd_func (cmd=0x1829360, 
    args=0x7fffffffe1ab "\\(print[^[ bsiedf\".-]\\)", from_tty=0)
    at /data/gdb_versions/devel/src/gdb/cli/cli-decode.c:2181
#9  0x0000000000a9ae89 in execute_command (p=0x7fffffffe1c1 ")", from_tty=0)
    at /data/gdb_versions/devel/src/gdb/top.c:670
#10 0x0000000000806a16 in catch_command_errors (
    command=0xa9a918 <execute_command(char const*, int)>, 
    arg=0x7fffffffe1a3 "apropos \\(print[^[ bsiedf\".-]\\)", from_tty=0, 
    do_bp_actions=true) at /data/gdb_versions/devel/src/gdb/main.c:450
#11 0x0000000000806beb in execute_cmdargs (cmdarg_vec=0x7fffffffd920, 
    file_type=CMDARG_FILE, cmd_type=CMDARG_COMMAND, ret=0x7fffffffd8fc)
    at /data/gdb_versions/devel/src/gdb/main.c:539
#12 0x0000000000807e4f in captured_main_1 (context=0x7fffffffdb20)
    at /data/gdb_versions/devel/src/gdb/main.c:1211
#13 0x0000000000808036 in captured_main (data=0x7fffffffdb20)
    at /data/gdb_versions/devel/src/gdb/main.c:1232
#14 0x00000000008080a1 in gdb_main (args=0x7fffffffdb20)
    at /data/gdb_versions/devel/src/gdb/main.c:1257
#15 0x000000000041781d in main (argc=10, argv=0x7fffffffdc38)
    at /data/gdb_versions/devel/src/gdb/gdb.c:32

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #2 from Tom de Vries <vries at gcc dot gnu.org> ---
OK, after doing:
...
(gdb) 
28        int code = regcomp (&m_pattern, regex, cflags);
...
in compiled_regex::compiled_regex we have:
...
(gdb) p *m_pattern.buffer->lock
$6 = 1869440370
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |schwab at sourceware dot org

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #3 from Tom de Vries <vries at gcc dot gnu.org> ---
OK, it looks like the regcomp used is the one from libpcre, while the re_search
is from glibc.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

Tom Tromey <tromey at sourceware dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tromey at sourceware dot org

--- Comment #4 from Tom Tromey <tromey at sourceware dot org> ---
(In reply to Tom de Vries from comment #3)
> OK, it looks like the regcomp used is the one from libpcre, while the
> re_search is from glibc.

I wonder if we can switch over to the implementation in libiberty instead,
and if that would help.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #5 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #4)
> (In reply to Tom de Vries from comment #3)
> > OK, it looks like the regcomp used is the one from libpcre, while the
> > re_search is from glibc.
> 
> I wonder if we can switch over to the implementation in libiberty instead,
> and if that would help.

I did a bit more digging and found that the previous version, pcre had a
workaround for this, and than pcre2 on Leap 15.2 does not export that symbol. 
So for now I'm assuming it's a platform-specific issue, and filed 
https://bugzilla.opensuse.org/show_bug.cgi?id=1184279 .

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #6 from Tom de Vries <vries at gcc dot gnu.org> ---
This is a workaround:
...
$ LD_PRELOAD=/lib64/libc-2.33.so gdb -q -batch  -ex "apropos apropos"
apropos -- Search for commands matching a REGEXP.
set style title -- Title display styling.
show style title -- Title display styling.
$
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #7 from Tom de Vries <vries at gcc dot gnu.org> ---
So, what happened is that somebody enabled --with-pcre2 on the ncurses package,
and then changed:
...
$ cat /usr/lib64/libncursesw.so 
/* GNU ld script */
INPUT(/lib64/libncursesw.so.6 AS_NEEDED(-ltinfo -ldl))
...
into:
...
$ cat /usr/lib64/libncursesw.so 
/* GNU ld script */
INPUT(/lib64/libncursesw.so.6 AS_NEEDED(-ltinfo -ldl -lpcre2-posix -lpcre2-8))
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #8 from Tom de Vries <vries at gcc dot gnu.org> ---
Workaround:
...
diff --git a/gdb/Makefile.in b/gdb/Makefile.in
index 3318c1a5215..b9afac7a492 100644
--- a/gdb/Makefile.in
+++ b/gdb/Makefile.in
@@ -635,6 +635,7 @@ INTERNAL_LDFLAGS = \
 CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \
         $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \
        $(XM_CLIBS) $(GDBTKLIBS) \
+       -Wl,--no-as-needed,-lc,--as-needed \
        @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \
        $(LIBEXPAT) $(LIBLZMA) $(LIBBABELTRACE) $(LIBIPT) \
        $(WIN32LIBS) $(LIBGNU) $(LIBGNU_EXTRA_LIBS) $(LIBICONV) \
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #9 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #8)
> Workaround:
> ...
> diff --git a/gdb/Makefile.in b/gdb/Makefile.in
> index 3318c1a5215..b9afac7a492 100644
> --- a/gdb/Makefile.in
> +++ b/gdb/Makefile.in
> @@ -635,6 +635,7 @@ INTERNAL_LDFLAGS = \
>  CLIBS = $(SIM) $(READLINE) $(OPCODES) $(LIBCTF) $(BFD) $(ZLIB) \
>          $(LIBSUPPORT) $(INTL) $(LIBIBERTY) $(LIBDECNUMBER) \
>         $(XM_CLIBS) $(GDBTKLIBS) \
> +       -Wl,--no-as-needed,-lc,--as-needed \
>         @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \
>         $(LIBEXPAT) $(LIBLZMA) $(LIBBABELTRACE) $(LIBIPT) \
>         $(WIN32LIBS) $(LIBGNU) $(LIBGNU_EXTRA_LIBS) $(LIBICONV) \
> ...

Actually, what already works is -Wl,-lc.  I think -lc should be enough, filed a
gcc PR: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99896

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #10 from Tom de Vries <vries at gcc dot gnu.org> ---
Andreas, here ( https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99896#c5 ) you
mention:
...
 The bug is in gdb because re_search cannot be paired with regcomp.
...

Could you either explain this or point to some documentation that would explain
this?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #11 from Andreas Schwab <schwab@linux-m68k.org> ---
https://pubs.opengroup.org/onlinepubs/9699919799/functions/regcomp.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #12 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Andreas Schwab from comment #11)
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/regcomp.html

OK, so that is the documentation of regcomp. Now can you point out where it
says that re_search is incompatible with regcomp?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[schwab@linux-m68k.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #13 from Andreas Schwab <schwab@linux-m68k.org> ---
There is no re_search in there.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #14 from Tom de Vries <vries at gcc dot gnu.org> ---
Posted patch:
https://sourceware.org/pipermail/gdb-patches/2021-April/177514.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[matz at suse dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

Michael Matz <matz at suse dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |matz at suse dot de

--- Comment #15 from Michael Matz <matz at suse dot de> ---
To use re_search (or the other GNU extensions of regex.h) you need to use
re_compile_pattern, not regcomp.  This is the non-conforming mixture Andreas
meant.

(Sidenote: I think there's no conforming possiblity to use the GNU extension
correctly without leaks, because there's no GNU extension equivalent of
regfree,
ala re_free or somesuch).

The games with -lc are brittle at best and don't really solve the underlying
non-conformance.  I don't know a good solution except avoiding the GNU
extensions
alltogether :-/  (Certainly forcing everyone to implicitely use libpcre2-posix
by linking against ncurses is a bad idea, but it would be unnoticable with
conforming use)

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #16 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Michael Matz from comment #15)
> To use re_search (or the other GNU extensions of regex.h) you need to use
> re_compile_pattern, not regcomp.  This is the non-conforming mixture Andreas
> meant.
> 

I understand what he means.  I just want to understand why.  Sofar, no evidence
or explanation was offered.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #17 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #16)
> (In reply to Michael Matz from comment #15)
> > To use re_search (or the other GNU extensions of regex.h) you need to use
> > re_compile_pattern, not regcomp.  This is the non-conforming mixture Andreas
> > meant.
> > 
> 
> I understand what he means.  I just want to understand why.  Sofar, no
> evidence or explanation was offered.

And therefore I'm forced to proceed under the assumption that the claim is
incorrect.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[matz at suse dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #18 from Michael Matz <matz at suse dot de> ---
The types are simply incompatible:

extern int regcomp (regex_t *_Restrict_ __preg,
                    const char *_Restrict_ __pattern,
                    int __cflags);
...
extern regoff_t re_search (struct re_pattern_buffer *__buffer,
                           const char *__String, regoff_t __length,
                           regoff_t __start, regoff_t __range,
                           struct re_registers *__regs);

regex_t != struct re_pattern_buffer.  In the GNU implementation they are a
typedef
of each other (which is why there are no warnings :-/ ).  But conceptually it's
two opaque (and different) types (or rather, in POSIX a struct
re_pattern_buffer
doesn't exist).

With the pcre2 library they are _in fact_ different types, not just
conceptually. 
So regcomp (from pcre2) uses a different layout than re_search (from glibc) and
hence initializing a regex_t with regcomp (in gdb's
compiled_regex::compiled_regex() in m_pattern) but using it as a
struct re_pattern_buffer for re_search via compiled_regex::search is not going
to work.  Your work-around of putting -lc somewhere in between makes it so that
regcomp again comes from glibc (with the one in the later libpcre2 being
ignored), and hence stuff works again.  (Of course uses within libncurses that
assume pcre2 layout will be broken with the glibc regcomp).

There are multiple problems here:
1) that libpcre2 is used implicitely at all
2) If we accept (1) then it's a problem that the <regex.h> header of glibc
   is used to compile stuff, because that enables getting into the problematic
   situation to start with (with the pcre2 headers re_search would have been 
   simply undeclared, and lead to compile errors making the problem more
   obvious, or alternatively libiberty's regex.c would have been included,
   instead of relying on the mixture of glibc and pcre2)
3) Mixing re_search and regcomp on the same data would still be incorrect
   as a matter of principle, even though it happens to work when staying within
   glibc: if the design of the GNU extension would have been so that
   interoperability was intended there would have been no need for the extra
   type and initializer re_compile_pattern: regex_t and regcomp would suffice.
4) But, as something like re_free is missing the design of the GNU extension
   also isn't completely orthogonal to POSIX regex_t, leading to necessary
   mixture which is actually incorrect as per (3).

(3) and (4) together make it so that the GNU extension involving
struct re_pattern_buffer can only be used in very controlled circumstances,
and (1) breaks those circumstances :-/

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #19 from Tom Tromey <tromey at sourceware dot org> ---
(In reply to Michael Matz from comment #18)
> The types are simply incompatible:
> [...]

This has existed for a very long time in gdb.
compiled_regex (commit 2d7cc5c797) didn't even introduce it,
it existed in apropos_command + apropos_cmd before that.

I suspect the answer is that gdb was originally written
to assume the libiberty regex implementation.
xregex.h redefines the various symbols and then xregex2.h says:

   typedef struct re_pattern_buffer regex_t;

I suppose the bug then is from this code in gdb_regex.h:

#ifdef USE_INCLUDED_REGEX
# include "xregex.h"
#else

commit 8898755195db
Date:   Tue Apr 4 02:08:52 2000 +0000

So still pretty darn old.

I wonder whether we should just switch this back off and rely
on libiberty.  The main benefit here would be that we have some
code that would be improved if it could use a regex that didn't
rely on a \0-terminated buffer, namely the output functions
that strip or apply styles.  Currently these have to allocate
sometimes to ensure their input is correctly terminated.
Anyway, I think the GNU implementations already handle this,
for their Emacs support.

OTOH maybe that's niche and we should just fix the bug directly.
I didn't research to see why that patch was written in the first place.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #20 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Michael Matz from comment #15)
> (Sidenote: I think there's no conforming possiblity to use the GNU extension
> correctly without leaks, because there's no GNU extension equivalent of
> regfree,
> ala re_free or somesuch).

I did some further investigation, and have started to suspect that the reason
why the regex GNU extensions in glibc are so poorly documented, is because
they're documented at gnulib.

And at gnulib docs (
https://www.gnu.org/software/gnulib/manual/html_node/Freeing-GNU-Pattern-Buffers.html#Freeing-GNU-Pattern-Buffers
) I read:
...
To free any allocated fields of a pattern buffer, use the POSIX function
regfree:

void
regfree (regex_t *preg)

preg is the pattern buffer whose allocated fields you want freed; this works
because since the type regex_t—the type for POSIX pattern buffers—is equivalent
to the type re_pattern_buffer. 
...

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #21 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Michael Matz from comment #18)
> The types are simply incompatible:
> 
> extern int regcomp (regex_t *_Restrict_ __preg,
>                     const char *_Restrict_ __pattern,
>                     int __cflags);
> ...
> extern regoff_t re_search (struct re_pattern_buffer *__buffer,
>                            const char *__String, regoff_t __length,
>                            regoff_t __start, regoff_t __range,
>                            struct re_registers *__regs);
> 
> regex_t != struct re_pattern_buffer.  In the GNU implementation they are a
> typedef
> of each other 

Right, and that typedef in a user-interfacing header file was precisely the
reason it looked obvious to me that the types were compatible.  So thanks for
making a detailed argument, now I understand the difference in interpretation.

> (which is why there are no warnings :-/ ).

Yeah, if these types were designed to be incompatible, the implementation has a
certain invitation-to-shoot-yourself-in-the-foot quality about it.

> But conceptually
> it's two opaque (and different) types

Agreed, that could be a valid point that I overlooked.  If so, the
incompatibility might deserve a mention at the typedef, especially since there
is at least one function (regfree) that does exploit that typedef.

Anyway, after finding this (
https://www.gnu.org/software/gnulib/manual/html_node/GNU-Regex-Functions.html#GNU-Regex-Functions
) it seems that this is the documentation I've been looking for.  It lists a
set of functions that can be used together.  There's only one function from the
posix interface explicitly listed: regfree.  This corroborates your point that
the types are meant to be incompatible.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #22 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #19)
> (In reply to Michael Matz from comment #18)
> > The types are simply incompatible:
> > [...]
> 
> This has existed for a very long time in gdb.
> compiled_regex (commit 2d7cc5c797) didn't even introduce it,
> it existed in apropos_command + apropos_cmd before that.
> 
> I suspect the answer is that gdb was originally written
> to assume the libiberty regex implementation.

I wonder if libiberty has the same problem.

I'm currently building with --with-included-regex and gdb seems to work fine,
and it eliminates any dependencies on external regex providers.

But libiberty has the same difference in declared types:
...
int
regcomp (regex_t *preg, const char *pattern, int cflags)
...
and:
...
int
re_search (struct re_pattern_buffer *bufp, const char *string, int size,
           int startpos, int range, struct re_registers *regs)
...

So, are these two compatible in libiberty or not?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[matz at suse dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #23 from Michael Matz <matz at suse dot de> ---
(In reply to Tom de Vries from comment #22)
> I wonder if libiberty has the same problem.
> 
> I'm currently building with --with-included-regex and gdb seems to work
> fine, and it eliminates any dependencies on external regex providers.
> 
> But libiberty has the same difference in declared types:
> ...
> int
> regcomp (regex_t *preg, const char *pattern, int cflags)
> ...
> and:
> ...
> int
> re_search (struct re_pattern_buffer *bufp, const char *string, int size,
>            int startpos, int range, struct re_registers *regs)
> ...
> 
> So, are these two compatible in libiberty or not?

_Within_ libiberty (which is essentially the same a gnulib/regex*.c and
glibc/regex*.c) these are compatible types, and using regcomp with re_search
works.  As soon as you replace the POSIX ones (and only those) with different
implementations the same problems occur.  Which is why mixing re_search and
regcomp is basically a problem in waiting when you can't ensure that both come
from the same implementation.  (Linking statically against libiberty or gnulib
would ensure this, as would directly linking in regex.o, but linking
dynamically
against libc and pcre2-posix would not).

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[tromey at sourceware dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #24 from Tom Tromey <tromey at sourceware dot org> ---
> from the same implementation.  (Linking statically against libiberty or
> gnulib
> would ensure this, as would directly linking in regex.o, but linking
> dynamically
> against libc and pcre2-posix would not).

libiberty renames the symbols to avoid clashing, which is why that
USE_INCLUDED_REGEX / xregex.h include stuff matters.  It prepends
an "x":

murgatroyd. pwd
/home/tromey/gdb/build/libiberty
murgatroyd. nm regex.o|grep regfree
00000000000063d0 T xregfree

I think there are a few routes forward:

1. Change the code to conform to POSIX.  Fine by me.

2. Change the code to always use libiberty.  This requires understanding
   why the alternative was ever made possible.

Option 2 has this mild benefit around avoiding allocations, if we want
to pursue it.

I tend to doubt that regex performance is at all important to gdb,
so I wouldn't worry about whether pcre is better or stuff like that.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #25 from Tom de Vries <vries at gcc dot gnu.org> ---
I tried to look a bit into the history.

Timeline:
- 1989-01-31
  gdb-3.1 commit of binutils-gdb.git adds regex.c. Looks like gnu regex.c,
  has copyright 1985.
- 1992-11-08:
  Initial revision commit of gnulib.git adds regex.c copied from
  GNU regex 0.11. Copyright goes back to 1985.
- 1993-04-03 
  GNU regex 0.12 is released, final release
- 1995-05-18
  Glibc imports GNU regex, presumably from gnulib.
- 1996-05-14
  Last update regex.c from gnulib to glibc
- 1997-07-26
  Gnulib updates regex.c from glibc
- 2000-04-03
  gdb grows option to use regex from glibc
- 2001-07-11
  libiberty imports regex.c from gdb
  https://gcc.gnu.org/pipermail/gcc-patches/2001-May/050292.html
- 2001-09-01
  Gdb starts using libiberty regex implementation.
  https://sourceware.org/pipermail/gdb-patches/2001-September/010733.html
- 2003-01-02
  gdb switches to use regex from glibc by default, provided it's version 2.
  https://sourceware.org/pipermail/gdb-patches/2003-January/023377.html

(In reply to Tom Tromey from comment #24)
> I think there are a few routes forward:
> 
> 1. Change the code to conform to POSIX.  Fine by me.
> 

I'd prefer it to avoid this rewrite.  It just means you'd have to add
functionality somewhere that is already contained in gnu regex.  Sounds like
duplication of work to me.

> 2. Change the code to always use libiberty.  This requires understanding
>    why the alternative was ever made possible.
> 

I miss the option of using gnulib. We currently don't import that module, but
we could.

The commit that introduced the possibility in gdb to use glibc, had the
rationale that glibc was where gnu regex was then maintained.

AFAICT, the commit that changed the default to glibc assumed that that change
was already in place, but undone somehow. I didn't find any evidence in the
source for this, so I'm not sure if there ever was an actual decision taken
there.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #26 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Tom Tromey from comment #24)
> 2. Change the code to always use libiberty.  This requires understanding
>    why the alternative was ever made possible.
> 

Submitted patch:
https://sourceware.org/pipermail/gdb-patches/2021-April/177716.html

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[cvs-commit at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

--- Comment #27 from cvs-commit at gcc dot gnu.org <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Tom de Vries <vries@sourceware.org>:

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=ff5075202958f78e1a0bb77405e19c347d1d0bbd

commit ff5075202958f78e1a0bb77405e19c347d1d0bbd
Author: Tom de Vries <tdevries@suse.de>
Date:   Wed Apr 21 21:54:03 2021 +0200

    [gdb/build] Hardcode --with-included-regex

    Currently gdb has a configure option:
    ...
    $ ./src/gdb/configure --help
      ...
      --without-included-regex
                              don't use included regex; this is the default on
                              systems with version 2 of the GNU C library (use
                              with caution on other system)
    ...

    The configure option controls config.h macro USE_INCLUDED_REGEX, which is
    used in gdb/gdb_regex.h to choose between:
    - using regex from libiberty (which is included in the binutils-gdb.git
repo,
      hence the 'included' in USE_INCLUDED_REGEX), or
    - using regex.h.

    In the former case, the symbol regcomp is remapped to a symbol xregcomp,
which
    is then provided by libiberty.

    In the latter case, the symbol regcomp is resolved at runtime, usually
binding
    to libc.  However, there is no mechanism in place to enforce this.

    PR27681 is an example of where that causes problems.  On openSUSE
Tumbleweed,
    the ncurses package got the --with-pcre2 configure switch enabled, and
solved
    the resulting dependencies using:
    ...
     $ cat /usr/lib64/libncursesw.so
     /* GNU ld script */
    -INPUT(/lib64/libncursesw.so.6 AS_NEEDED(-ltinfo -ldl))
    +INPUT(/lib64/libncursesw.so.6 AS_NEEDED(-ltinfo -ldl -lpcre2-posix
-lpcre2-8))
    ...

    This lead to regcomp being bound to libpcre2-posix instead of libc.

    This causes problems in several ways:
    - by compiling using regex.h, we've already chosen a specific regex_t
      implementation, and the one from pcre2-posix is not the same.
    - in gdb_regex.c we use GNU regex function re_search, which pcre2-posix
      doesn't provide, so while regcomp binds to pcre2-posix, re_search binds
to
      libc.

    A note on the latter: it's actually a bug to compile a regex using regcomp
and
    then pass it to re_search.  The GNU regex interface requires one to use
    re_compile_pattern or re_compile_fastmap.  But as long we're using one of
the
    GNU regex incarnations in gnulib, glibc or libiberty, we get away with
this.

    The PR could be fixed by adding -lc in a specific position in the link
line,
    to force regcomp to be bound to glibc.  But this solution was considered
    in the discussion in the PR as being brittle, and possibly causing problems
    elsewhere.

    Another solution offered was to restrict regex usage to posix, and no
longer
    use the GNU regex API.  This however could mean having to reproduce some of
    that functionality locally, which would mean maintaining the same
    functionality in more than one place.

    The solution chosen here, is to hardcode --with-included-regex, that is,
using
    libiberty.

    The option of using glibc for regex was introduced because glibc became the
    authorative source for GNU regex, so it offered the possibility to link
    against a more up-to-date regex version.

    In that aspect, this patch is a step back.  But we have the option of using
a
    more up-to-date regex version as a follow-up step: by using the regex from
    gnulib.

    Tested on x86_64-linux.

    gdb/ChangeLog:

    2021-04-21  Tom de Vries  <tdevries@suse.de>

            PR build/27681
            * configure.ac: Remove
--without-included-regex/--with-included-regex.
            * config.in: Regenerate.
            * configure: Regenerate.
            * gdb_regex.h: Assume USE_INCLUDED_REGEX is defined.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug gdb/27681] FAIL: gdb.base/help.exp: apropos \(print[^[ bsiedf\".-]\) (timeout)

[vries at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27681

Tom de Vries <vries at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
   Target Milestone|---                         |11.1
         Resolution|---                         |FIXED

--- Comment #28 from Tom de Vries <vries at gcc dot gnu.org> ---
Patch committed, marking resolved-fixed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.