[gcc r14-8349] fold-const: Fold larger VIEW_CONVERT_EXPRs [PR113462]

[Jakub Jelinek <jakub@gcc.gnu.org>]

https://gcc.gnu.org/g:5015015ae6b29b3f1734c7693ba25b88cdd531a1

commit r14-8349-g5015015ae6b29b3f1734c7693ba25b88cdd531a1
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Tue Jan 23 09:02:48 2024 +0100

    fold-const: Fold larger VIEW_CONVERT_EXPRs [PR113462]
    
    On Mon, Jan 22, 2024 at 11:27:52AM +0100, Richard Biener wrote:
    > We run into
    >
    > static tree
    > native_interpret_int (tree type, const unsigned char *ptr, int len)
    > {
    > ...
    >   if (total_bytes > len
    >       || total_bytes * BITS_PER_UNIT > HOST_BITS_PER_DOUBLE_INT)
    >     return NULL_TREE;
    >
    > OTOH using a V_C_E to "truncate" a _BitInt looks wrong?  OTOH the
    > check doesn't really handle native_encode_expr using the "proper"
    > wide_int encoding however that's exactly handled.  So it might be
    > a pre-existing issue that's only uncovered by large _BitInts
    > (__int128 might show similar issues?)
    
    I guess the || total_bytes * BITS_PER_UNIT > HOST_BITS_PER_DOUBLE_INT
    conditions make no sense, all we care is whether it fits in the buffer
    or not.
    But then there is
    fold_view_convert_expr
    (and other spots) which use
      /* We support up to 1024-bit values (for GCN/RISC-V V128QImode).  */
      unsigned char buffer[128];
    or something similar.
    This patch fixes even that by using a XALLOCAVEC allocated buffer
    if the type size is 129 .. 8192 bytes.
    
    2024-01-22  Jakub Jelinek  <jakub@redhat.com>
    
            PR tree-optimization/113462
            * fold-const.cc (native_interpret_int): Don't punt if total_bytes
            is larger than HOST_BITS_PER_DOUBLE_INT / BITS_PER_UNIT.
            (fold_view_convert_expr): Use XALLOCAVEC buffers for types with
            sizes between 129 and 8192 bytes.

Diff:
---
 gcc/fold-const.cc | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/gcc/fold-const.cc b/gcc/fold-const.cc
index 385e4a69ab3..1fd957288d4 100644
--- a/gcc/fold-const.cc
+++ b/gcc/fold-const.cc
@@ -8773,8 +8773,7 @@ native_interpret_int (tree type, const unsigned char *ptr, int len)
   else
     total_bytes = GET_MODE_SIZE (SCALAR_INT_TYPE_MODE (type));
 
-  if (total_bytes > len
-      || total_bytes * BITS_PER_UNIT > HOST_BITS_PER_DOUBLE_INT)
+  if (total_bytes > len)
     return NULL_TREE;
 
   wide_int result = wi::from_buffer (ptr, total_bytes);
@@ -9329,9 +9328,10 @@ fold_view_convert_vector_encoding (tree type, tree expr)
 static tree
 fold_view_convert_expr (tree type, tree expr)
 {
-  /* We support up to 1024-bit values (for GCN/RISC-V V128QImode).  */
   unsigned char buffer[128];
+  unsigned char *buf;
   int len;
+  HOST_WIDE_INT l;
 
   /* Check that the host and target are sane.  */
   if (CHAR_BIT != 8 || BITS_PER_UNIT != 8)
@@ -9341,11 +9341,23 @@ fold_view_convert_expr (tree type, tree expr)
     if (tree res = fold_view_convert_vector_encoding (type, expr))
       return res;
 
-  len = native_encode_expr (expr, buffer, sizeof (buffer));
+  l = int_size_in_bytes (type);
+  if (l > (int) sizeof (buffer)
+      && l <= WIDE_INT_MAX_PRECISION / BITS_PER_UNIT)
+    {
+      buf = XALLOCAVEC (unsigned char, l);
+      len = l;
+    }
+  else
+    {
+      buf = buffer;
+      len = sizeof (buffer);
+    }
+  len = native_encode_expr (expr, buf, len);
   if (len == 0)
     return NULL_TREE;
 
-  return native_interpret_expr (type, buffer, len);
+  return native_interpret_expr (type, buf, len);
 }
 
 /* Build an expression for the address of T.  Folds away INDIRECT_REF