[Bug analyzer/99886] New: Infinite loop in -fanalyzer seen on gcc.dg/analyzer/malloc-1.c with -fanalyzer-verbosity=0

[Bug analyzer/99886] New: Infinite loop in -fanalyzer seen on gcc.dg/analyzer/malloc-1.c with -fanalyzer-verbosity=0

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99886

            Bug ID: 99886
           Summary: Infinite loop in -fanalyzer seen on
                    gcc.dg/analyzer/malloc-1.c with -fanalyzer-verbosity=0
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: analyzer
          Assignee: dmalcolm at gcc dot gnu.org
          Reporter: dmalcolm at gcc dot gnu.org
  Target Milestone: ---

Reproducer:

#include <stdlib.h>

struct coord {
  float x;
  float y;
};

void test_34 (void)
{
  float *q;
  struct coord *p = malloc (sizeof (struct coord));
  if (!p)
    return;
  p->x = 0.0f;
  q = &p->x;
  free (p);
  *q = 1.0f; /* { dg-warning "use after 'free' of 'q'" } */
};

when compiled with:
  -fanalyzer -fanalyzer-verbosity=0

Doesn't happen at other verbosity levels.

Seems to be getting stuck here in diagnostic_manager::consolidate_conditions:

(gdb) bt
#0  0x00000000024a8777 in vec<ana::checker_event*, va_heap, vl_ptr>::length
(this=0x7fffffffca98) at ../../src/gcc/vec.h:1439
#1  0x00000000024c0d30 in ana::checker_path::num_events (this=0x7fffffffca90)
at ../../src/gcc/analyzer/checker-path.h:503
#2  0x00000000024c0138 in ana::diagnostic_manager::consolidate_conditions
(this=0x7fffffffcf00, path=0x7fffffffca90)
    at ../../src/gcc/analyzer/diagnostic-manager.cc:2203
#3  0x00000000024bf433 in ana::diagnostic_manager::prune_path
(this=0x7fffffffcf00, path=0x7fffffffca90, sm=0x0, sval=0x0, state=0x0)
    at ../../src/gcc/analyzer/diagnostic-manager.cc:1783
#4  0x00000000024be1b6 in ana::diagnostic_manager::emit_saved_diagnostic
(this=0x7fffffffcf00, eg=..., sd=...)
    at ../../src/gcc/analyzer/diagnostic-manager.cc:1131
#5  0x00000000024c1af1 in ana::dedupe_winners::emit_best (this=0x7fffffffcc20,
dm=0x7fffffffcf00, eg=...)
    at ../../src/gcc/analyzer/diagnostic-manager.cc:1051
#6  0x00000000024bdfa6 in ana::diagnostic_manager::emit_saved_diagnostics
(this=0x7fffffffcf00, eg=...)
    at ../../src/gcc/analyzer/diagnostic-manager.cc:1100
#7  0x0000000001830e13 in ana::impl_run_checkers (logger=0x35ccbf0) at
../../src/gcc/analyzer/engine.cc:4892

[Bug analyzer/99886] Delay loop in -fanalyzer seen on gcc.dg/analyzer/malloc-1.c with -fanalyzer-verbosity=0

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99886

David Malcolm <dmalcolm at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |ASSIGNED
   Last reconfirmed|                            |2021-04-02
            Summary|Infinite loop in -fanalyzer |Delay loop in -fanalyzer
                   |seen on                     |seen on
                   |gcc.dg/analyzer/malloc-1.c  |gcc.dg/analyzer/malloc-1.c
                   |with -fanalyzer-verbosity=0 |with -fanalyzer-verbosity=0

--- Comment #1 from David Malcolm <dmalcolm at gcc dot gnu.org> ---
Not an infinite loop; it "merely" counts very slowly from 0 to 0xffffffff due
to ((unsigned)0) - 1 as a loop boundary.

Testing a fix.

[Bug analyzer/99886] Delay loop in -fanalyzer seen on gcc.dg/analyzer/malloc-1.c with -fanalyzer-verbosity=0

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99886

--- Comment #2 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by David Malcolm <dmalcolm@gcc.gnu.org>:

https://gcc.gnu.org/g:69b66ff02353a87585329bb3cf4ac20d6dee1b16

commit r11-7987-g69b66ff02353a87585329bb3cf4ac20d6dee1b16
Author: David Malcolm <dmalcolm@redhat.com>
Date:   Mon Apr 5 10:48:01 2021 -0400

    analyzer: fix apparent hang with -fanalyzer-verbosity=0 [PR analyzer/99886]

    The analyzer appeared to enter an infinite loop on malloc-1.c
    when -fanalyzer-verbosity=0 was used.  In fact, it was slowly
    counting from 0 to 0xffffffff.

    Root cause is looping up to effectively ((unsigned)0) - 1 in
    diagnostic_manager::consolidate_conditions when there are no events
    in the path.

    Fixed by the following, which uses signed integers when subtracting
    from path->num_events () when simplifying checker_paths.

    gcc/analyzer/ChangeLog:
            PR analyzer/99886
            * diagnostic-manager.cc
            (diagnostic_manager::prune_interproc_events): Use signed integers
            when subtracting one from path->num_events ().
            (diagnostic_manager::consolidate_conditions): Likewise.  Convert
            next_idx to a signed int.

    gcc/testsuite/ChangeLog:
            PR analyzer/99886
            * gcc.dg/analyzer/pr99886.c: New test.

[Bug analyzer/99886] Delay loop in -fanalyzer seen on gcc.dg/analyzer/malloc-1.c with -fanalyzer-verbosity=0

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99886

David Malcolm <dmalcolm at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|ASSIGNED                    |RESOLVED

--- Comment #3 from David Malcolm <dmalcolm at gcc dot gnu.org> ---
Should be fixed by the above patch.