[Bug sanitizer/108995] Missed signed integer overflow checks in UBsan? since r8-343-g2bf54d93f159210d

["rguenth at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
We already fold this on GENERIC to

  *c = -229690488(OVF);

with optimization and

  *c = (int) b * 10921;

without.  It's almost surely through extract_muldiv, we also diagnose

t.c: In function ‘main’:
t.c:6:14: warning: integer overflow in expression ‘65526 * (int)b’ of type
‘int’ results in ‘-1378142932’ [-Woverflow]
    6 |   *c = 65526 * b / 6;
      |        ~~~~~~^~~

the issue seems to be that the C frontend, with optimization, constant folds
the
initializer of 'b' and with all-constants we ignore sanitization (but emit
a diagnostic).

Without optimization we run into extract_muldiv doing

      /* If these operations "cancel" each other, we have the main
         optimizations of this pass, which occur when either constant is a
         multiple of the other, in which case we replace this with either an
         operation or CODE or TCODE.

         If we have an unsigned type, we cannot do this since it will change
         the result if the original computation overflowed.  */
      if (TYPE_OVERFLOW_UNDEFINED (ctype)
          && ((code == MULT_EXPR && tcode == EXACT_DIV_EXPR)
              || (tcode == MULT_EXPR
                  && code != TRUNC_MOD_EXPR && code != CEIL_MOD_EXPR
                  && code != FLOOR_MOD_EXPR && code != ROUND_MOD_EXPR
                  && code != MULT_EXPR)))
        {

which is "fine".  We do have a few && !TYPE_OVERFLOW_SANITIZED checks
around but here we're missing it (I also believe we shouldn't do it this
way, but ...).

Without optimizing -Wstrict-overflow would diagnose this as well.

The following fixes the "bug" at -O0 but leaves the constant folding in the
frontend untouched (it could possibly refrain from replacing ops with
TREE_OVERFLOW constants when sanitizing overflow).

I'm not sure we want a patch like the following though.

diff --git a/gcc/fold-const.cc b/gcc/fold-const.cc
index 9aaea71a2fc..a9af4dbd0a3 100644
--- a/gcc/fold-const.cc
+++ b/gcc/fold-const.cc
@@ -7102,6 +7102,8 @@ extract_muldiv_1 (tree t, tree c, enum tree_code code,
tree wide_type,
          if (wi::multiple_of_p (wi::to_wide (op1), wi::to_wide (c),
                                 TYPE_SIGN (type)))
            {
+             if (TYPE_OVERFLOW_SANITIZED (ctype))
+               return NULL_TREE;
              if (TYPE_OVERFLOW_UNDEFINED (ctype))
                *strict_overflow_p = true;
              return fold_build2 (tcode, ctype, fold_convert (ctype, op0),
@@ -7112,6 +7114,8 @@ extract_muldiv_1 (tree t, tree c, enum tree_code code,
tree wide_type,
          else if (wi::multiple_of_p (wi::to_wide (c), wi::to_wide (op1),
                                      TYPE_SIGN (type)))
            {
+             if (TYPE_OVERFLOW_SANITIZED (ctype))
+               return NULL_TREE;
              if (TYPE_OVERFLOW_UNDEFINED (ctype))
                *strict_overflow_p = true;
              return fold_build2 (code, ctype, fold_convert (ctype, op0),