[Bug sanitizer/104937] New: wrong code with _Complex division and -fsanitize=undefined

[Bug sanitizer/104937] New: wrong code with _Complex division and -fsanitize=undefined

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

            Bug ID: 104937
           Summary: wrong code with _Complex division and
                    -fsanitize=undefined
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: wrong-code
          Severity: normal
          Priority: P3
         Component: sanitizer
          Assignee: unassigned at gcc dot gnu.org
          Reporter: zsojka at seznam dot cz
                CC: dodji at gcc dot gnu.org, dvyukov at gcc dot gnu.org,
                    jakub at gcc dot gnu.org, kcc at gcc dot gnu.org, marxin at gcc dot gnu.org
  Target Milestone: ---
              Host: x86_64-pc-linux-gnu
            Target: x86_64-pc-linux-gnu

Created attachment 52631
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52631&action=edit
reduced testcase

Output:
$ x86_64-pc-linux-gnu-gcc -fsanitize=undefined testcase.c
$ ./a.out 
Aborted

The __imag__ part seems to be garbage.

$ x86_64-pc-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/repo/gcc-trunk/binary-latest-amd64/bin/x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/repo/gcc-trunk/binary-trunk-r12-7654-20220315102422-gda24fce323e-checking-yes-rtl-df-extra-nobootstrap-amd64/bin/../libexec/gcc/x86_64-pc-linux-gnu/12.0.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /repo/gcc-trunk//configure --enable-languages=c,c++
--enable-valgrind-annotations --disable-nls --enable-checking=yes,rtl,df,extra
--disable-bootstrap --with-cloog --with-ppl --with-isl
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu
--target=x86_64-pc-linux-gnu --with-ld=/usr/bin/x86_64-pc-linux-gnu-ld
--with-as=/usr/bin/x86_64-pc-linux-gnu-as --disable-libstdcxx-pch
--prefix=/repo/gcc-trunk//binary-trunk-r12-7654-20220315102422-gda24fce323e-checking-yes-rtl-df-extra-nobootstrap-amd64
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.0.1 20220315 (experimental) (GCC)

[Bug sanitizer/104937] wrong code with _Complex division and -fsanitize=undefined

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jsm28 at gcc dot gnu.org

--- Comment #1 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Doesn't seem to be sanitizer specific.
Just change the testcase to:
_Complex unsigned
foo (_Complex unsigned x)
{
  _Complex unsigned y = x;
  return y / x;
}

int
main ()
{
  _Complex unsigned x = foo (6 + 43 * 1i);
  if (x != 1)
    __builtin_abort ();
  return 0;
}
so that it doesn't get folded to 1 + 0i in the FE and at -O0 you'll get abort
as well.
I think division just doesn't work at all for _Complex unsigned, seems to work
for _Complex signed.
Don't we have some PR covering it?

[Bug sanitizer/104937] wrong code with _Complex division and -fsanitize=undefined

[zsojka at seznam dot cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

--- Comment #2 from Zdenek Sojka <zsojka at seznam dot cz> ---
There was PR104604 where _Complex unsigned division was fixed, but I didn't
notice any other related wrong-code PR.

[Bug sanitizer/104937] wrong code with _Complex division and -fsanitize=undefined

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
For
_Complex unsigned
foo (_Complex unsigned x, _Complex unsigned y)
{
  return y / x;
}

int
main ()
{
  _Complex int a = foo (6 + 43i, 6 + 43i);
  __builtin_printf ("%d %d\n", __real__ a, __imag__ a);
  return 0;
}
from what I see if xr < xi we expand it as
t=xr/xi*xr + xi
((yr*xr/xi + yi) / t) + ((yi*xr/xi - yr) / t) * i
Now, for the given numbers that is t=43 and the result is
43 / 43 + (0 - 6) / 43 * i
For signed complex that "works" by producing 1 + 0i, but for unsigned
that 0U - 6U is some huge number divided by 43.
But now that I see it, I think it doesn't work properly in most cases for
_Complex int either - for signed the xr < xi is actually done as abs(xr) <
abs(xi), so for both signed and unsigned complex xr/xi is 0, and similarly the
other case where we use xi/xr, with the only exception when xr==xi and thus
xr/xi is 1.  I bet the algorithm is well suited only for floating point...

[Bug sanitizer/104937] wrong code with _Complex division and -fsanitize=undefined

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
https://gcc.gnu.org/legacy-ml/gcc/2001-11/msg00790.html
Regarding the debug info argument in there, this is being discussed for DWARF6.

[Bug c/104937] wrong code with _Complex division and -fsanitize=undefined

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|sanitizer                   |c

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
Do the division using _Complex double and convert back?  But honestly I think
rejecting _Complex int division is good enough and maybe we should indeed
deprecate all of _Complex int.

[Bug c/104937] wrong code with _Complex division and -fsanitize=undefined

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104937

--- Comment #6 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
What we could for _Complex int just use the standard
(a + b*i) / (c + d*i) = (a*c + b*d) / (c*c + d*d) + ((b*c - a*d) / (c*c +
d*d))*i
which would be 6 multiplications and 2 divisions instead of the current
3 multiplications and 3 divisions.  But in order to avoid overflows we'd need
to do that all in bigger precision.  c*c + d*d for _Complex int is in [0,
LONG_LONG_MAX+1ULL] range, a*c+b*d in [-LONG_LONG_MAX+2*INT_MAX+2,
LONG_LONG_MAX+1ULL] range etc., so perhaps with some special cases we fit into
2*prec type.
For _Complex unsigned we'd also need to watch for b*c being smaller than a*d
and handle that by doing - ((a*d - b*c) / (c*c + d*d)) instead.