[PATCH] misaligned stores support

[Dorit Nuzman <DORIT@il.ibm.com>]

[-- Attachment #1: Type: text/plain, Size: 1740 bytes --]


This is an old patch - I think relative to autovect-branch - from over a
year ago, written by Leehod Baruch (IIRC, with some hints from rth at the
time). I don't remember if it was working all the way, and it would anyhow
need to be updated to current mainline and tested.  I'm bringing it up as
promised here - http://gcc.gnu.org/ml/gcc/2007-01/msg00289.html, in case
someone wants to take it forward.

This patch includes code to expand a MISALIGNED_INDIRECT_REF in a store
stmt, and a couple changes to the vectorizer: one is when considering to
peel a loop to align an otherwise-unsupported misaligned memory access -
currently we do that only for stores, but with this patch there's no longer
a need to treat stores differently than loads. The second change is to
check whether a movmisalign is supported rather than always returning that
there's no support for misaligned stores.

With this patch we'd probably need to change a lot of vectorizer testcases
that currently assume that misaligned stores are not supported.

Also, a couple things that, with this patch, would be even more
important/desired:
1. Add support for the "software-pipelined" (or "explicit") vectorization
scheme of handling misalignment (i.e. - add a "realign-store" idiom and use
it to handle misaligned store by doing an extra aligned load+store before
and after of the loop, and realignment operations in the loop to extract
the desired data for each iteration).
2. Add a cost model - with this patch the vectorizer will generate
misaligned vector stores, whereas peeling (which is what we do now) might
be preferable. We need a way to evaluate which vectorization scheme is
expected to be more profitable.

Dorit

(See attached file: misaaligned_store.txt)

[-- Attachment #2: misaaligned_store.txt --]
[-- Type: text/plain, Size: 8724 bytes --]

? out
? outdiff
Index: expr.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/expr.c,v
retrieving revision 1.738.2.5
diff -c -3 -p -r1.738.2.5 expr.c
*** expr.c	8 Mar 2005 08:29:50 -0000	1.738.2.5
--- expr.c	17 Nov 2005 09:00:32 -0000
*************** expand_assignment (tree to, tree from)
*** 3924,3930 ****
--- 3924,3964 ----
        pop_temp_slots ();
        return;
      }
+  
+   else if (TREE_CODE (to) == MISALIGNED_INDIRECT_REF)
+     {
+       enum machine_mode mode, op_mode1;
+       enum insn_code icode;
+       rtx reg, addr, mem, insn;
+       tree orig;
+       
+       reg = expand_expr (from, NULL_RTX, VOIDmode, EXPAND_NORMAL);
+       reg = force_not_mem (reg);
  
+       mode = TYPE_MODE (TREE_TYPE (to));
+       addr = expand_expr (TREE_OPERAND (to, 0), NULL_RTX, VOIDmode,
+ 			  EXPAND_SUM);
+       addr = memory_address (mode, addr);
+       mem = gen_rtx_MEM (mode, addr);
+       
+       orig = REF_ORIGINAL (to);
+       if (!orig)
+ 	orig = to;
+       set_mem_attributes (mem, orig, 0);
+       
+       icode = movmisalign_optab->handlers[mode].insn_code;
+       gcc_assert (icode != CODE_FOR_nothing);
+ 
+       op_mode1 = insn_data[icode].operand[1].mode;
+       if (! (*insn_data[icode].operand[1].predicate) (reg, op_mode1)
+           && op_mode1 != VOIDmode)
+         reg = copy_to_mode_reg (op_mode1, reg);
+       
+       insn = GEN_FCN (icode) (mem, reg);
+       emit_insn (insn);
+       return;  
+     }
+   
    /* Ordinary treatment.  Expand TO to get a REG or MEM rtx.
       Don't re-expand if it was expanded already (in COMPONENT_REF case).  */
  
Index: tree-vect-analyze.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/tree-vect-analyze.c,v
retrieving revision 2.4.4.9
diff -c -3 -p -r2.4.4.9 tree-vect-analyze.c
*** tree-vect-analyze.c	5 Apr 2005 09:50:38 -0000	2.4.4.9
--- tree-vect-analyze.c	17 Nov 2005 09:00:35 -0000
*************** vect_enhance_data_refs_alignment (loop_v
*** 945,955 ****
    /* While cost model enhancements are expected in the future, the high level
       view of the code at this time is as follows:
  
!      A) If there is a misaligned write then see if peeling to align this write
          can make all data references satisfy vect_supportable_dr_alignment.
!         If so, update data structures as needed and return true.  Note that
!         at this time vect_supportable_dr_alignment is known to return false
!         for a a misaligned write.
  
       B) If peeling wasn't possible and there is a data reference with an
          unknown misalignment that does not satisfy vect_supportable_dr_alignment
--- 945,953 ----
    /* While cost model enhancements are expected in the future, the high level
       view of the code at this time is as follows:
  
!      A) If there is an unsupported misaligned access then see if peeling to align this access
          can make all data references satisfy vect_supportable_dr_alignment.
!         If so, update data structures as needed and return true.
  
       B) If peeling wasn't possible and there is a data reference with an
          unknown misalignment that does not satisfy vect_supportable_dr_alignment
*************** vect_enhance_data_refs_alignment (loop_v
*** 976,996 ****
         in code size).
  
       The scheme we use FORNOW: peel to force the alignment of the first
!      misaligned store in the loop.
!      Rationale: misaligned stores are not yet supported.
  
       TODO: Use a cost model.  */
  
    for (i = 0; i < VARRAY_ACTIVE_SIZE (loop_datarefs); i++)
      {
        struct data_reference *dr = VARRAY_GENERIC_PTR (loop_datarefs, i);
!       if (!DR_IS_READ (dr) && !aligned_access_p (dr))
          {
            dr0 = dr;
            do_peeling = true;
  	  break;
          }
!     }
  
    /* Often peeling for alignment will require peeling for loop-bound, which in 
       turn requires that we know how to adjust the loop ivs after the loop.  */
--- 974,994 ----
         in code size).
  
       The scheme we use FORNOW: peel to force the alignment of the first
!      unsupported misaligned access in the loop.
  
       TODO: Use a cost model.  */
  
    for (i = 0; i < VARRAY_ACTIVE_SIZE (loop_datarefs); i++)
      {
        struct data_reference *dr = VARRAY_GENERIC_PTR (loop_datarefs, i);
!       supportable_dr_alignment = vect_supportable_dr_alignment (dr);
!       if (!supportable_dr_alignment)
          {
            dr0 = dr;
            do_peeling = true;
  	  break;
          }
!      }
  
    /* Often peeling for alignment will require peeling for loop-bound, which in 
       turn requires that we know how to adjust the loop ivs after the loop.  */
Index: tree-vect-transform.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/tree-vect-transform.c,v
retrieving revision 2.1.4.7
diff -c -3 -p -r2.1.4.7 tree-vect-transform.c
*** tree-vect-transform.c	5 Apr 2005 09:50:38 -0000	2.1.4.7
--- tree-vect-transform.c	17 Nov 2005 09:00:41 -0000
*************** Software Foundation, 59 Temple Place - S
*** 47,53 ****
  
  /* Utility functions for the code transformation.  */
  static bool vect_transform_stmt (tree, block_stmt_iterator *);
- static void vect_align_data_ref (tree);
  static tree vect_create_destination_var (tree, tree);
  static tree vect_create_data_ref_ptr 
    (tree, block_stmt_iterator *, tree, tree *, bool); 
--- 47,52 ----
*************** vect_create_addr_base_for_vector_ref (tr
*** 245,269 ****
  }
  
  
- /* Function vect_align_data_ref.
- 
-    Handle mislignment of a memory accesses.
- 
-    FORNOW: Can't handle misaligned accesses. 
-    Make sure that the dataref is aligned.  */
- 
- static void
- vect_align_data_ref (tree stmt)
- {
-   stmt_vec_info stmt_info = vinfo_for_stmt (stmt);
-   struct data_reference *dr = STMT_VINFO_DATA_REF (stmt_info);
- 
-   /* FORNOW: can't handle misaligned accesses; 
-              all accesses expected to be aligned.  */
-   gcc_assert (aligned_access_p (dr));
- }
- 
- 
  /* Function vect_create_data_ref_ptr.
  
     Create a memory reference expression for vector access, to be used in a
--- 244,249 ----
*************** vectorizable_store (tree stmt, block_stm
*** 1605,1620 ****
  
    alignment_support_cheme = vect_supportable_dr_alignment (dr);
    gcc_assert (alignment_support_cheme);
!   gcc_assert (alignment_support_cheme == dr_aligned);  /* FORNOW */
  
    /* Handle use - get the vectorized def from the defining stmt.  */
    vec_oprnd1 = vect_get_vec_def_for_operand (op, stmt);
  
    /* Handle def.  */
-   /* FORNOW: make sure the data reference is aligned.  */
-   vect_align_data_ref (stmt);
    data_ref = vect_create_data_ref_ptr (stmt, bsi, NULL_TREE, &dummy, false);
!   data_ref = build_fold_indirect_ref (data_ref);
  
    /* Arguments are ready. create the new vector stmt.  */
    *vec_stmt = build2 (MODIFY_EXPR, vectype, data_ref, vec_oprnd1);
--- 1585,1608 ----
  
    alignment_support_cheme = vect_supportable_dr_alignment (dr);
    gcc_assert (alignment_support_cheme);
!   gcc_assert (alignment_support_cheme == dr_aligned
!       || alignment_support_cheme == dr_unaligned_supported);
  
    /* Handle use - get the vectorized def from the defining stmt.  */
    vec_oprnd1 = vect_get_vec_def_for_operand (op, stmt);
  
    /* Handle def.  */
    data_ref = vect_create_data_ref_ptr (stmt, bsi, NULL_TREE, &dummy, false);
! 
!   if (aligned_access_p (dr))
!     data_ref = build_fold_indirect_ref (data_ref);
!   else
!     {
!       int mis = DR_MISALIGNMENT (dr);
!       tree tmis = (mis == -1 ? size_zero_node : size_int (mis));
!       tmis = size_binop (MULT_EXPR, tmis, size_int (BITS_PER_UNIT));
!       data_ref = build2 (MISALIGNED_INDIRECT_REF, vectype, data_ref, tmis);
!     }
  
    /* Arguments are ready. create the new vector stmt.  */
    *vec_stmt = build2 (MODIFY_EXPR, vectype, data_ref, vec_oprnd1);
Index: tree-vectorizer.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/tree-vectorizer.c,v
retrieving revision 2.25.2.24
diff -c -3 -p -r2.25.2.24 tree-vectorizer.c
*** tree-vectorizer.c	3 Apr 2005 11:55:21 -0000	2.25.2.24
--- tree-vectorizer.c	17 Nov 2005 09:00:41 -0000
*************** vect_supportable_dr_alignment (struct da
*** 1698,1703 ****
--- 1698,1708 ----
  	/* Can't software pipeline the loads, but can at least do them.  */
  	return dr_unaligned_supported;
      }
+   else
+     {
+       if (movmisalign_optab->handlers[mode].insn_code != CODE_FOR_nothing)
+         return dr_unaligned_supported;
+     }
  
    /* Unsupported.  */
    return dr_unaligned_unsupported;