[Bug target/110027] [11/12/13/14 regression] Misaligned vector store on detect_stack_use_after_return

["liuhongt at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110027

--- Comment #13 from Hongtao Liu <liuhongt at gcc dot gnu.org> ---
So the stack is like

----------- stack top

-32

--------- (offset -32)

-64 (32 bytes redzone)

--------- (offset -64)

-128 (64 bytes __m512)

-------- (offset -128)

 (32-bytes redzone)

-------(offset -160)   <--- __asan_stack_malloc_128 try to allocate an buffer 


  /* Emit the prologue sequence.  */
  if (asan_frame_size > 32 && asan_frame_size <= 65536 && pbase
      && param_asan_use_after_return)
    {
      use_after_return_class = floor_log2 (asan_frame_size - 1) - 5;
      /* __asan_stack_malloc_N guarantees alignment
         N < 6 ? (64 << N) : 4096 bytes.  */
      if (alignb > (use_after_return_class < 6
                    ? (64U << use_after_return_class) : 4096U))
        use_after_return_class = -1;
      else if (alignb > ASAN_RED_ZONE_SIZE && (asan_frame_size & (alignb - 1)))
        base_align_bias = ((asan_frame_size + alignb - 1)
                           & ~(alignb - HOST_WIDE_INT_1)) - asan_frame_size;
    }

  /* Align base if target is STRICT_ALIGNMENT.  */
  if (STRICT_ALIGNMENT)
    {
      const HOST_WIDE_INT align
        = (GET_MODE_ALIGNMENT (SImode) / BITS_PER_UNIT) << ASAN_SHADOW_SHIFT;
      base = expand_binop (Pmode, and_optab, base, gen_int_mode (-align,
Pmode),
                           NULL_RTX, 1, OPTAB_DIRECT);
    }

  if (use_after_return_class == -1 && pbase)
    emit_move_insn (pbase, base);

  base = expand_binop (Pmode, add_optab, base,
                       gen_int_mode (base_offset - base_align_bias, Pmode),
                       NULL_RTX, 1, OPTAB_DIRECT); ---------- suspicious add

  orig_base = NULL_RTX;
  if (use_after_return_class != -1)
    {
      ...
      ret = emit_library_call_value (ret, NULL_RTX, LCT_NORMAL, ptr_mode,
                                     GEN_INT (asan_frame_size
                                              + base_align_bias),
                                     TYPE_MODE (pointer_sized_int_node));
      /* __asan_stack_malloc_[n] returns a pointer to fake stack if succeeded
         and NULL otherwise.  Check RET value is NULL here and jump over the
         BASE reassignment in this case.  Otherwise, reassign BASE to RET.  */
      emit_cmp_and_jump_insns (ret, const0_rtx, EQ, NULL_RTX,
                               VOIDmode, 0, lab,
                               profile_probability:: very_unlikely ());
      ret = convert_memory_address (Pmode, ret);
      emit_move_insn (base, ret);
      emit_label (lab);
      emit_move_insn (pbase, expand_binop (Pmode, add_optab, base,
                                           gen_int_mode (base_align_bias
                                                         - base_offset, Pmode),
                                           NULL_RTX, 1, OPTAB_DIRECT));


base_align_bias is calculated to make (asan_frame_size(128) +
base_align_bias(0)) be multiple of alignb (64),  but didn't make `base_offset
(160) - base_align_bias (0)` be multiple of 64, so when __asan_stack_malloc_128
return an address aligned to 64, and then plus (base_offset (160) -
base_align_bias (0)), it's misaligned.