[Bug sanitizer/113669] New: -fsanitize=undefined failed to check a signed integer overflow

["jiajing_zheng at 163 dot com" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113669

            Bug ID: 113669
           Summary: -fsanitize=undefined failed to check a signed integer
                    overflow
           Product: gcc
           Version: 12.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jiajing_zheng at 163 dot com
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org
  Target Milestone: ---

I took a motion of the loop invariant expression of source.c and got
mutation.c.
Both the two files have a signed integer overflow problem.
I checked both files using -fsanitize=undefined at the -O0,-O1,-O2,-O3,-Os
optimization levels. The results showed that 'signed integer overflow' was
given for mutation.c at -O0,-O1,-O3,-Os, but missing at -O2. And for source.c,
the message was missing at all the above optimization levels.

jing@jing-ubuntu:~$ cat source.c 

static int g_B = -66265337;
static unsigned char g_A[2] = {0b00110110, 0b01111010};

static void func_1(void);

static void func_1(void) {
  char *arr[4];
  char ch = '1';
  int i;
  for (i = 0; i < 4; i++) {
    // source statement:
    g_A[0] += ((int)(g_B * g_A[1])) & (g_A[1] & g_A[0]) | g_A[0];
          arr[i] = &ch;
  }
}

int main(void) {
  func_1();
  return 0;
}

jing@jing-ubuntu:~$ cat mutation.c 

static int g_B = -66265337;
static unsigned char g_A[2] = {0b00110110, 0b01111010};

static void func_1(void);

static void func_1(void) {
  char *arr[4];
  char ch = '1';
  int i;
  //loop invaraint expression motion:
  int temp = (int)(g_B * g_A[1]);
  for (i = 0; i < 4; i++) {
    // mutation statement:
    g_A[0] += temp & (g_A[1] & g_A[0]) | g_A[0];
          arr[i] = &ch;
  }
}

int main(void) {
  func_1();
  return 0;
}


results for source.c:
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O0 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O1 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O2 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O3 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -Os && ./a.out

result for mutation.c at -O2:
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O2 && ./a.out

results for mutation.c at -O0,-O1,-O3,-Os:
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O0 && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O1 && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O3 && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -Os && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'


jing@jing-ubuntu:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/home/jing/gcc-12.2.0/usr/local/bin/../libexec/gcc/x86_64-pc-linux-gnu/12.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../configure -enable-checking=release -enable-languages=c,c++
-disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.2.0 (GCC)