[Bug sanitizer/113669] New: -fsanitize=undefined failed to check a signed integer overflow

[Bug sanitizer/113669] New: -fsanitize=undefined failed to check a signed integer overflow

[jiajing_zheng at 163 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113669

            Bug ID: 113669
           Summary: -fsanitize=undefined failed to check a signed integer
                    overflow
           Product: gcc
           Version: 12.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jiajing_zheng at 163 dot com
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org
  Target Milestone: ---

I took a motion of the loop invariant expression of source.c and got
mutation.c.
Both the two files have a signed integer overflow problem.
I checked both files using -fsanitize=undefined at the -O0,-O1,-O2,-O3,-Os
optimization levels. The results showed that 'signed integer overflow' was
given for mutation.c at -O0,-O1,-O3,-Os, but missing at -O2. And for source.c,
the message was missing at all the above optimization levels.

jing@jing-ubuntu:~$ cat source.c 

static int g_B = -66265337;
static unsigned char g_A[2] = {0b00110110, 0b01111010};

static void func_1(void);

static void func_1(void) {
  char *arr[4];
  char ch = '1';
  int i;
  for (i = 0; i < 4; i++) {
    // source statement:
    g_A[0] += ((int)(g_B * g_A[1])) & (g_A[1] & g_A[0]) | g_A[0];
          arr[i] = &ch;
  }
}

int main(void) {
  func_1();
  return 0;
}

jing@jing-ubuntu:~$ cat mutation.c 

static int g_B = -66265337;
static unsigned char g_A[2] = {0b00110110, 0b01111010};

static void func_1(void);

static void func_1(void) {
  char *arr[4];
  char ch = '1';
  int i;
  //loop invaraint expression motion:
  int temp = (int)(g_B * g_A[1]);
  for (i = 0; i < 4; i++) {
    // mutation statement:
    g_A[0] += temp & (g_A[1] & g_A[0]) | g_A[0];
          arr[i] = &ch;
  }
}

int main(void) {
  func_1();
  return 0;
}


results for source.c:
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O0 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O1 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O2 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -O3 && ./a.out
jing@jing-ubuntu:~$ gcc source.c -fsanitize=undefined,address -Os && ./a.out

result for mutation.c at -O2:
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O2 && ./a.out

results for mutation.c at -O0,-O1,-O3,-Os:
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O0 && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O1 && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -O3 && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'
jing@jing-ubuntu:~$ gcc mutation.c -fsanitize=undefined,address -Os && ./a.out
mutation.c:12:7: runtime error: signed integer overflow: 122 * -66265337 cannot
be represented in type 'int'


jing@jing-ubuntu:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/home/jing/gcc-12.2.0/usr/local/bin/../libexec/gcc/x86_64-pc-linux-gnu/12.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../configure -enable-checking=release -enable-languages=c,c++
-disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.2.0 (GCC)

[Bug sanitizer/113669] -fsanitize=undefined failed to check a signed integer overflow

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113669

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
This is because already the FE optimizes it, when it sees that
((int)(g_B * g_A[1])) & (g_A[1] & g_A[0]) | g_A[0]
is just being added to unsigned char element, the upper bits of it aren't
needed, so the multiplication and & and | are all performed in unsigned char
rather than wider types.

[Bug middle-end/113669] -fsanitize=undefined failed to check a signed integer overflow

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113669

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2024-01-31
             Status|UNCONFIRMED                 |NEW
     Ever confirmed|0                           |1

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
So confirmed.

[Bug middle-end/113669] -fsanitize=undefined failed to check a signed integer overflow

[jiajing_zheng at 163 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113669

--- Comment #3 from Jiajing_Zheng <jiajing_zheng at 163 dot com> ---
(In reply to Jakub Jelinek from comment #1)
> This is because already the FE optimizes it, when it sees that
> ((int)(g_B * g_A[1])) & (g_A[1] & g_A[0]) | g_A[0]
> is just being added to unsigned char element, the upper bits of it aren't
> needed, so the multiplication and & and | are all performed in unsigned char
> rather than wider types.

Thanks for your reply. I then used 'gcc -O2 mutation.c -fsanitize=undefined -S'
to generate mutation.s. As shown below, the relevant compilation sections 'addl
%r13d, %r13d' show that the statement 'g_A[0] += temp & (g_A[1] & g_A[0]) |
g_A[0];' in the loop is optimized to 'g_A[0] += g_A[0];'.

  .L8:
        addl    %r13d, %r13d
        movslq  %ebx, %rsi
        movb    %r13b, g_A(%rip)
        cmpq    $4, %rsi
        jnb     .L12

Is that what you mean by "the FE optimizes it"? I want to see the file
generated by a file.c after FE optimization, should I go to the corresponding
assembly file.s?