[Bug c/98168] New: Optimization that can lead to security vulnerabilities

[Bug c/98168] New: Optimization that can lead to security vulnerabilities

[jpegqs at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

            Bug ID: 98168
           Summary: Optimization that can lead to security vulnerabilities
           Product: gcc
           Version: 10.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jpegqs at gmail dot com
  Target Milestone: ---

Created attachment 49692
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=49692&action=edit
bounds.c

I encountered a bug (98159) that you refused to fix because it is "undefined
behavior". But this code proves that this "compiler" behavior can lead to
security vulnerabilities in some software.

Here GCC thinks that if both signed integers are positive, then the sum of
these integers is also positive. And removes the next bounds check for the
negative values (it could be written different, but this is the common way).

int test(int a, int b, int *buf) {
  if (a >= 0 && b >= 0) {
    a += b;
    // let's check that we are not reading outside the buffer
    if (a >= 0 && a < 8) return buf[a];
  }
  return -1;
}

So this code supposed to read the element A+B from a buffer of 8 values. And if
the sum is out of the buffer, then return -1. But when compiling with GCC
-O2/O3 on x86/x86_64 (and possibly others), you can pass A=0x7fffffff,
B=0x7fffffff and access buf[-2] (as with any negative value except -1).

Thus, optimizations that falsely assume that the target machine is performing
signed integer saturation when it is not - should be considered dangerous.

In my opinion, UB in C has a different purpose, it exists because C is a
low-level language and in most cases can use a single machine instruction for a
general operation. So for compilers it should be "target machine behavior", not
"we can do anything". And compilers must maintain this behavior while removing
some operations when optimizing the code.

[Bug c/98168] Optimization that can lead to security vulnerabilities

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |INVALID

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
So no. UD means anything can happen.

[Bug c/98168] Optimization that can lead to security vulnerabilities

[msebor at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

Martin Sebor <msebor at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |msebor at gcc dot gnu.org

--- Comment #2 from Martin Sebor <msebor at gcc dot gnu.org> ---
CERT Secure Coding Standard rule "INT32-C. Ensure that operations on signed
integers do not result in overflow" shows a number of helpful examples of how
to detect and prevent signed overflow in integer arithmetic: 
https://wiki.sei.cmu.edu/confluence/x/UtYxBQ

GCC provides a number of efficient intrinsics to do the same:
https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html

[Bug c/98168] Optimization that can lead to security vulnerabilities

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

Eric Gallager <egallager at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |egallager at gcc dot gnu.org

--- Comment #3 from Eric Gallager <egallager at gcc dot gnu.org> ---
I would think that -Wstrict-overflow would catch this, but it doesn't catch it
at any level, not even -Wstrict-overflow=5...

[Bug c/98168] Optimization that can lead to security vulnerabilities

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

--- Comment #4 from Eric Gallager <egallager at gcc dot gnu.org> ---
note that -ftrapv or -fsanitize=signed-integer-overflow should successfully
catch it at runtime, though

[Bug c/98168] Optimization that can lead to security vulnerabilities

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

--- Comment #5 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to Ilya Kurdyukov from comment #0)
> Thus, optimizations that falsely assume that the target machine is
> performing signed integer saturation when it is not - should be considered
> dangerous.

That's not what the optimizations assume though. They assume that you write
correct code without undefined behaviour. If you want to be able to write
invalid C and get predictable results, use the compiler options that allow you
to do that. There are several, as mentioned above (and which you should have
tried before filing PR 98159, as requested by https://gcc.gnu.org/bugs which
you were asked to read by the banner at the top of the new bug form).

> In my opinion, UB in C has a different purpose, it exists because C is a
> low-level language and in most cases can use a single machine instruction
> for a general operation. So for compilers it should be "target machine
> behavior", not "we can do anything". And compilers must maintain this
> behavior while removing some operations when optimizing the code.

You are welcome to your opinion.

https://blog.regehr.org/archives/213 (and parts 2 and 3 of the series) is a
good read.