Re: [5/9] Main target-independent support for direct interleaving

[Richard Guenther <richard.guenther@gmail.com>]

On Mon, Apr 18, 2011 at 1:24 PM, Richard Sandiford
<richard.sandiford@linaro.org> wrote:
> Richard Guenther <richard.guenther@gmail.com> writes:
>> On Tue, Apr 12, 2011 at 3:59 PM, Richard Sandiford
>> <richard.sandiford@linaro.org> wrote:
>>> This patch adds vec_load_lanes and vec_store_lanes optabs for instructions
>>> like NEON's vldN and vstN.  The optabs are defined this way because the
>>> vectors must be allocated to a block of consecutive registers.
>>>
>>> Tested on x86_64-linux-gnu and arm-linux-gnueabi.  OK to install?
>>>
>>> Richard
>>>
>>>
>>> gcc/
>>>        * doc/md.texi (vec_load_lanes, vec_store_lanes): Document.
>>>        * optabs.h (COI_vec_load_lanes, COI_vec_store_lanes): New
>>>        convert_optab_index values.
>>>        (vec_load_lanes_optab, vec_store_lanes_optab): New convert optabs.
>>>        * genopinit.c (optabs): Initialize the new optabs.
>>>        * internal-fn.def (LOAD_LANES, STORE_LANES): New internal functions.
>>>        * internal-fn.c (get_multi_vector_move, expand_LOAD_LANES)
>>>        (expand_STORE_LANES): New functions.
>>>        * tree.h (build_simple_array_type): Declare.
>>>        * tree.c (build_simple_array_type): New function.
>>>        * tree-vectorizer.h (vect_model_store_cost): Add a bool argument.
>>>        (vect_model_load_cost): Likewise.
>>>        (vect_store_lanes_supported, vect_load_lanes_supported)
>>>        (vect_record_strided_load_vectors): Declare.
>>>        * tree-vect-data-refs.c (vect_lanes_optab_supported_p)
>>>        (vect_store_lanes_supported, vect_load_lanes_supported): New functions.
>>>        (vect_transform_strided_load): Split out statement recording into...
>>>        (vect_record_strided_load_vectors): ...this new function.
>>>        * tree-vect-stmts.c (create_vector_array, read_vector_array)
>>>        (write_vector_array, create_array_ref): New functions.
>>>        (vect_model_store_cost): Add store_lanes_p argument.
>>>        (vect_model_load_cost): Add load_lanes_p argument.
>>>        (vectorizable_store): Try to use store-lanes functions for
>>>        interleaved stores.
>>>        (vectorizable_load): Likewise load-lanes and loads.
>>>        * tree-vect-slp.c (vect_get_and_check_slp_defs)
>>>        (vect_build_slp_tree):
>>>
>>> Index: gcc/doc/md.texi
>>> ===================================================================
>>> --- gcc/doc/md.texi     2011-04-12 12:16:46.000000000 +0100
>>> +++ gcc/doc/md.texi     2011-04-12 14:48:28.000000000 +0100
>>> @@ -3846,6 +3846,48 @@ into consecutive memory locations.  Oper
>>>  consecutive memory locations, operand 1 is the first register, and
>>>  operand 2 is a constant: the number of consecutive registers.
>>>
>>> +@cindex @code{vec_load_lanes@var{m}@var{n}} instruction pattern
>>> +@item @samp{vec_load_lanes@var{m}@var{n}}
>>> +Perform an interleaved load of several vectors from memory operand 1
>>> +into register operand 0.  Both operands have mode @var{m}.  The register
>>> +operand is viewed as holding consecutive vectors of mode @var{n},
>>> +while the memory operand is a flat array that contains the same number
>>> +of elements.  The operation is equivalent to:
>>> +
>>> +@smallexample
>>> +int c = GET_MODE_SIZE (@var{m}) / GET_MODE_SIZE (@var{n});
>>> +for (j = 0; j < GET_MODE_NUNITS (@var{n}); j++)
>>> +  for (i = 0; i < c; i++)
>>> +    operand0[i][j] = operand1[j * c + i];
>>> +@end smallexample
>>> +
>>> +For example, @samp{vec_load_lanestiv4hi} loads 8 16-bit values
>>> +from memory into a register of mode @samp{TI}@.  The register
>>> +contains two consecutive vectors of mode @samp{V4HI}@.
>>
>> So vec_load_lanestiv2qi would load ... ?  c == 8 here.  Intuitively
>> such operation would have adjacent blocks of siv2qi memory.  But
>> maybe you want to constrain the mode size to GET_MODE_SIZE (@var{n})
>> * GET_MODE_NUNITS (@var{n})?  In which case the mode m is
>> redundant?  You could specify that we load NUNITS adjacent vectors into
>> an integer mode of appropriate size.
>
> Like you say, vec_load_lanestiv2qi would load 16 QImode elements into
> 8 consecutive V2QI registers.  The first element from register vector I
> would come from operand1[I] and the second element would come from
> operand1[I + 8].  That's meant to be a valid combination.

Ok, but the C loop from the example doesn't seem to match.  Or I couldn't
wrap my head around it despite looking for 5 minutes and already having
coffee ;)  I would have expected the vectors being in memory as

  v0[0], v1[0], v0[1], v1[1], v2[0], v3[1]. v2[1], v3[1], ...

not

  v0[0], v1[0], v2[0], ...

as I would have thought the former is more useful (simple unrolling for
stride 2).  We'd need a separate set of optabs for such an interleaving
scheme?  In which case we might want to come up with a more
specific name than load_lane?

> We specifically want to allow:
>
>  GET_MODE_SIZE (@var{m})
>    != GET_MODE_SIZE (@var{n}) * GET_MODE_NUNITS (@var{n})
>
> The vec_load_lanestiv4hi example in the docs is one case of this:
>
>  GET_MODE_SIZE (@var{m}) = 16
>  GET_MODE_SIZE (@var{n}) = 8
>  GET_MODE_NUNITS (@var{n}) = 4
>
> That example maps directly to ARM's vld2.32.  We also want cases
> where @var{m} is three times the size of @var{n} (vld3.WW) and
> cases where @var{m} is four times the size of @var{n} (vld4.WW)
>
>>> +/* Return a representation of ELT_TYPE[NELTS], using indices of type
>>> +   sizetype.  */
>>> +
>>> +tree
>>> +build_simple_array_type (tree elt_type, unsigned HOST_WIDE_INT nelts)
>>
>> build_array_type_nelts
>
> OK.
>
> Richard
>