Re: RFA: libiberty: Add a limit on demangling qualifiers (PR 87241)

[Jason Merrill <jason@redhat.com>]

On Wed, Dec 12, 2018 at 6:29 AM Nick Clifton <nickc@redhat.com> wrote:
>
>   Sorry to bother you, but I have another libiberty demangler resource
>   exhaustion prevention patch to present.  This one is for:
>
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87241
>
>   Jonathan Wakely reported that __cxa_demanlge() was returning a -2
>   result, but I did not see this.  Instead I found that
>   consume_count_with_underscores() is returning a very large number
>   (because a very large value is encoded in the mangled string) and this
>   is resulting in many calls to remember_Ktype() which eventually
>   exhaust the amount of memory available.
>
>   The attached patch is a simplistic approach to solving this problem by
>   adding a hard upper limit on the number of qualifiers that will be
>   allowed by the demangler.  I am not sure if this is the best approach
>   to solving the problem, but it is a simple one, and I would think one
>   that would not prevent the demangling of any real mangled names.  The
>   limit does not have to be DEMANGLE_RECURSE_LIMIT of course.  I just
>   chose that value because it was convenient and of a size that I
>   thought was appropriate.
>
>   I also did run the libiberty testsuite this time, with no failures
>   reported. :-)
>
>   OK to apply ?
>
> Cheers
>   Nick
>
> libiberty/ChangeLog
> 2018-12-12  Nick Clifton  <nickc@redhat.com>
>
>         * cplus-dem.c (demangle_qualified): Add an upper limit on the
>         number of qualifiers supported, based upon the value of
>         DEMANGLE_RECURSE_LIMIT.

This issue also will be resolved by disabling or removing the old
demangling code, which I haven't seen anyone argue against.

Jason