[Bug sanitizer/108995] New: Missed signed integer overflow checks in UBsan?

[Bug sanitizer/108995] New: Missed signed integer overflow checks in UBsan?

[qrzhang at gatech dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

            Bug ID: 108995
           Summary: Missed signed integer overflow checks in UBsan?
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: qrzhang at gatech dot edu
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org, marxin at gcc dot gnu.org
  Target Milestone: ---

$ cat abc.c
int printf(const char *, ...);
int a;
const int b = 44514;
int *c = &a;
void main(void) {
  *c = 65526 * b / 6;
  printf("%d\n", a);
}

Ubsan did not emit any message. However, the outputs are different.

$ gcc-trunk -O3 -fsanitize=undefined abc.c ; ./a.out
-229690488

$ gcc-trunk  -fsanitize=undefined abc.c ; ./a.out
486137394

[Bug sanitizer/108995] Missed signed integer overflow checks in UBsan? since r8-343-g2bf54d93f159210d

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

Martin Liška <marxin at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Missed signed integer       |Missed signed integer
                   |overflow checks in UBsan?   |overflow checks in UBsan?
                   |                            |since
                   |                            |r8-343-g2bf54d93f159210d
             Status|UNCONFIRMED                 |NEW
                 CC|                            |rguenth at gcc dot gnu.org
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2023-03-06

--- Comment #1 from Martin Liška <marxin at gcc dot gnu.org> ---
We regress since r8-343-g2bf54d93f159210d.

[Bug sanitizer/108995] Missed signed integer overflow checks in UBsan? since r8-343-g2bf54d93f159210d

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
We already fold this on GENERIC to

  *c = -229690488(OVF);

with optimization and

  *c = (int) b * 10921;

without.  It's almost surely through extract_muldiv, we also diagnose

t.c: In function ‘main’:
t.c:6:14: warning: integer overflow in expression ‘65526 * (int)b’ of type
‘int’ results in ‘-1378142932’ [-Woverflow]
    6 |   *c = 65526 * b / 6;
      |        ~~~~~~^~~

the issue seems to be that the C frontend, with optimization, constant folds
the
initializer of 'b' and with all-constants we ignore sanitization (but emit
a diagnostic).

Without optimization we run into extract_muldiv doing

      /* If these operations "cancel" each other, we have the main
         optimizations of this pass, which occur when either constant is a
         multiple of the other, in which case we replace this with either an
         operation or CODE or TCODE.

         If we have an unsigned type, we cannot do this since it will change
         the result if the original computation overflowed.  */
      if (TYPE_OVERFLOW_UNDEFINED (ctype)
          && ((code == MULT_EXPR && tcode == EXACT_DIV_EXPR)
              || (tcode == MULT_EXPR
                  && code != TRUNC_MOD_EXPR && code != CEIL_MOD_EXPR
                  && code != FLOOR_MOD_EXPR && code != ROUND_MOD_EXPR
                  && code != MULT_EXPR)))
        {

which is "fine".  We do have a few && !TYPE_OVERFLOW_SANITIZED checks
around but here we're missing it (I also believe we shouldn't do it this
way, but ...).

Without optimizing -Wstrict-overflow would diagnose this as well.

The following fixes the "bug" at -O0 but leaves the constant folding in the
frontend untouched (it could possibly refrain from replacing ops with
TREE_OVERFLOW constants when sanitizing overflow).

I'm not sure we want a patch like the following though.

diff --git a/gcc/fold-const.cc b/gcc/fold-const.cc
index 9aaea71a2fc..a9af4dbd0a3 100644
--- a/gcc/fold-const.cc
+++ b/gcc/fold-const.cc
@@ -7102,6 +7102,8 @@ extract_muldiv_1 (tree t, tree c, enum tree_code code,
tree wide_type,
          if (wi::multiple_of_p (wi::to_wide (op1), wi::to_wide (c),
                                 TYPE_SIGN (type)))
            {
+             if (TYPE_OVERFLOW_SANITIZED (ctype))
+               return NULL_TREE;
              if (TYPE_OVERFLOW_UNDEFINED (ctype))
                *strict_overflow_p = true;
              return fold_build2 (tcode, ctype, fold_convert (ctype, op0),
@@ -7112,6 +7114,8 @@ extract_muldiv_1 (tree t, tree c, enum tree_code code,
tree wide_type,
          else if (wi::multiple_of_p (wi::to_wide (c), wi::to_wide (op1),
                                      TYPE_SIGN (type)))
            {
+             if (TYPE_OVERFLOW_SANITIZED (ctype))
+               return NULL_TREE;
              if (TYPE_OVERFLOW_UNDEFINED (ctype))
                *strict_overflow_p = true;
              return fold_build2 (code, ctype, fold_convert (ctype, op0),

[Bug sanitizer/108995] Missed signed integer overflow checks in UBsan? since r8-343-g2bf54d93f159210d

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Assignee|unassigned at gcc dot gnu.org      |rguenth at gcc dot gnu.org

--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> ---
I have posted the patch, waiting for feedback.

[Bug sanitizer/108995] Missed signed integer overflow checks in UBsan? since r8-343-g2bf54d93f159210d

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

--- Comment #4 from CVS Commits <cvs-commit at gcc dot gnu.org> ---
The master branch has been updated by Richard Biener <rguenth@gcc.gnu.org>:

https://gcc.gnu.org/g:ace65db9215882b95e2ead1bb0dc8c54c2ea69be

commit r13-6550-gace65db9215882b95e2ead1bb0dc8c54c2ea69be
Author: Richard Biener <rguenther@suse.de>
Date:   Wed Mar 8 09:06:44 2023 +0100

    middle-end/108995 - avoid folding when sanitizing overflow

    The following plugs one place in extract_muldiv where it should avoid
    folding when sanitizing overflow.

            PR middle-end/108995
            * fold-const.cc (extract_muldiv_1): Avoid folding
            (CST * b) / CST2 when sanitizing overflow and we rely on
            overflow being undefined.

            * gcc.dg/ubsan/pr108995.c: New testcase.

[Bug c/108995] Missed signed integer overflow checks in UBsan? since r8-343-g2bf54d93f159210d

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108995

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|sanitizer                   |c
           Assignee|rguenth at gcc dot gnu.org         |unassigned at gcc dot gnu.org
            Version|unknown                     |13.0
             Status|ASSIGNED                    |NEW

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
The -O0 case is now fixed.  The optimize case isn't sanitized because it's
folded in the frontend - I've changed the component accordingly (but it's also
diagnosed at compile-time unconditionally).