public inbox for gcc-patches@gcc.gnu.org
 help / color / mirror / Atom feed
* [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax
@ 2023-10-20  8:34 pan2.li
  2023-10-20  8:47 ` juzhe.zhong
  2023-10-20 12:59 ` [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857] pan2.li
  0 siblings, 2 replies; 6+ messages in thread
From: pan2.li @ 2023-10-20  8:34 UTC (permalink / raw)
  To: gcc-patches; +Cc: juzhe.zhong, pan2.li, yanzhang.wang, kito.cheng

From: Pan Li <pan2.li@intel.com>

Given we have code like below:

typedef char vnx16i __attribute__ ((vector_size (16)));

vnx16i __attribute__ ((noinline, noclone))
test (vnx16i x, vnx16i y)
{
  return __builtin_shufflevector (x, y, 11, 12, 13, 14, 11, 12, 13, 14,
                                        11, 12, 13, 14, 11, 12, 13, 14);
}

It can perform the auto vectorization when

-march=rv64gcv_zvl1024b --param=riscv-autovec-preference=fixed-vlmax

but cannot when

-march=rv64gcv_zvl2048b --param=riscv-autovec-preference=fixed-vlmax

The reason comes from the miniaml machine mode of QI is RVVMF8QI, which
is 1024 / 8 = 128 bits, aka the size of VNx16QI. When we set zvl2048b,
the bit size of RVVMFQI is 2048 / 8 = 256, which is not matching the
bit size of VNx16QI (128 bits).

Thus, this patch would like to enable the VLS mode for such case, aka
VNx16QI vls mode for zvl2048b.

Before this patch:
test:
  srli    a4,a1,40
  andi    a4,a4,0xff
  srli    a3,a1,32
  srli    a5,a1,48
  slli    a0,a4,8
  andi    a3,a3,0xff
  andi    a5,a5,0xff
  slli    a2,a5,16
  or      a0,a3,a0
  srli    a1,a1,56
  or      a0,a0,a2
  slli    a2,a1,24
  slli    a3,a3,32
  or      a0,a0,a2
  slli    a4,a4,40
  or      a0,a0,a3
  slli    a5,a5,48
  or      a0,a0,a4
  or      a0,a0,a5
  slli    a1,a1,56
  or      a0,a0,a1
  mv      a1,a0
  ret

After this patch:
test:
  vsetivli        zero,16,e8,mf8,ta,ma
  vle8.v  v2,0(a1)
  vsetivli        zero,4,e32,mf2,ta,ma
  vrgather.vi     v1,v2,3
  vsetivli        zero,16,e8,mf8,ta,ma
  vse8.v  v1,0(a0)
  ret

gcc/ChangeLog:

	* config/riscv/riscv-opts.h (TARGET_VECTOR_VLS): Add enabling
	condition when fixed-vlmax by passing mode as arg.
	* config/riscv/riscv-protos.h (legal_vls_mode_in_fixed_vlmax_p):
	New func decl.
	* config/riscv/riscv-v.cc (legal_vls_mode_in_fixed_vlmax_p):
	New func impl to check the vls mode is legal when fixed-vlmax.
	(autovectorize_vector_modes): Pass mode arg to TARGET_VECTOR_VLS.
	* config/riscv/riscv-vector-switch.def (VLS_ENTRY): Ditto.
	* config/riscv/vector-iterators.md: Ditto.

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c: Adjust checker.
	* gcc.target/riscv/rvv/autovec/vls/def.h: Add help define.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c: New test.

Signed-off-by: Pan Li <pan2.li@intel.com>
---
 gcc/config/riscv/riscv-opts.h                 |   13 +-
 gcc/config/riscv/riscv-protos.h               |    1 +
 gcc/config/riscv/riscv-v.cc                   |   98 +-
 gcc/config/riscv/riscv-vector-switch.def      |  188 +-
 gcc/config/riscv/vector-iterators.md          | 1843 +++++++++--------
 .../riscv/rvv/autovec/vls-vlmax/perm-4.c      |    4 +-
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-0.c  |  536 +++++
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-1.c  |  279 +++
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-2.c  |  151 ++
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-3.c  |   86 +
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-4.c  |  279 +++
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-5.c  |  151 ++
 .../riscv/rvv/autovec/vls-vlmax/vls-mode-6.c  |   86 +
 .../gcc.target/riscv/rvv/autovec/vls/def.h    |    6 +
 14 files changed, 2688 insertions(+), 1033 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c

diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
index 31ee42dea6b..9992800331a 100644
--- a/gcc/config/riscv/riscv-opts.h
+++ b/gcc/config/riscv/riscv-opts.h
@@ -119,9 +119,16 @@ enum riscv_entity
      : 32 << (__builtin_popcount (opts->x_riscv_zvl_flags) - 1))
 
 /* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode
-   is the highest priority choice and should not conflict with VLS modes.  */
-#define TARGET_VECTOR_VLS                                                      \
-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)
+   is the highest priority choice and should not conflict with VLS modes.
+
+   We also enable VLS modes for some cases in fixed-vlmax, aka the bitsize of
+   the VLS mode are smaller than the minimal vla.
+   */
+#define TARGET_VECTOR_VLS(mode)                                                \
+  (TARGET_VECTOR                                                               \
+    && (riscv_autovec_preference == RVV_SCALABLE                               \
+      || (riscv_autovec_preference == RVV_FIXED_VLMAX                          \
+	&& riscv_vector::legal_vls_mode_in_fixed_vlmax_p (mode))))
 
 /* TODO: Enable RVV movmisalign by default for now.  */
 #define TARGET_VECTOR_MISALIGN_SUPPORTED 1
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 6190faab501..c2f0da01c0f 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -552,6 +552,7 @@ unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
 bool cmp_lmul_le_one (machine_mode);
 bool cmp_lmul_gt_one (machine_mode);
 bool gather_scatter_valid_offset_mode_p (machine_mode);
+bool legal_vls_mode_in_fixed_vlmax_p (machine_mode);
 }
 
 /* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 895c11d13fc..a8ba0d46e1f 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -2423,20 +2423,19 @@ autovectorize_vector_modes (vector_modes *modes, bool)
 	    modes->safe_push (mode);
 	}
     }
-  if (TARGET_VECTOR_VLS)
-    {
-      /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
-      unsigned int i = 0;
-      unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
-      unsigned int size = base_size;
-      machine_mode mode;
-      while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
-	{
+    /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
+    unsigned int i = 0;
+    unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
+    unsigned int size = base_size;
+    machine_mode mode;
+    while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
+     {
+	if (TARGET_VECTOR_VLS (mode))
 	  modes->safe_push (mode);
-	  i++;
-	  size = base_size / (1U << i);
-	}
-    }
+
+	i++;
+	size = base_size / (1U << i);
+     }
   /* Enable LOOP_VINFO comparison in COST model.  */
   return VECT_COMPARE_COSTS;
 }
@@ -3880,6 +3879,79 @@ cmp_lmul_gt_one (machine_mode mode)
   return false;
 }
 
+/* Return true if the VLS mode is legal when preference fixed-vlmax,
+   Take vlen = 2048 as example.
+
+   Note: Below table based on vlen = 2048.
+   +----------------------------------------------------+----------------------+
+   | VLS mode                                           | VLA mode             |
+   +----------------------------------------------------+----------------------+
+   | Name       | Precision | Inner Precision | Enabled | Min mode  | Min bits |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1BI       |     1     |              1  | Yes     | RVVMF64BI |    32    |
+   | V2BI       |     2     |              1  | Yes     | RVVMF64BI |    32    |
+   | V4BI       |     4     |              1  | Yes     | RVVMF64BI |    32    |
+   | V8BI       |     8     |              1  | Yes     | RVVMF64BI |    32    |
+   | V16BI      |    16     |              1  | Yes     | RVVMF64BI |    32    |
+   | V32BI      |    32     |              1  | NO      | RVVMF64BI |    32    |
+   | V64BI      |    64     |              1  | NO      | RVVMF64BI |    32    |
+   | ...        |   ...     |            ...  | ...     | RVVMF64BI |    32    |
+   | V4096BI    |  4096     |              1  | NO      | RVVMF64BI |    32    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1QI       |     8     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V2QI       |    16     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V4QI       |    32     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V8QI       |    64     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V16QI      |   128     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V32QI      |   256     |              8  | NO      | RVVMF8QI  |   256    |
+   | V64QI      |   512     |              8  | NO      | RVVMF8QI  |   256    |
+   | ...        |   ...     |              .. | ...     | RVVMF8QI  |   256    |
+   | V4096QI    | 32768     |              8  | NO      | RVVMF8QI  |   256    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1HI       |    16     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V2HI       |    32     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V4HI       |    64     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V8HI       |   128     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V16HI      |   256     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V32HI      |   512     |              16 | NO      | RVVMF4HI  |   512    |
+   | V64HI      |  1024     |              16 | NO      | RVVMF4HI  |   512    |
+   | ...        |   ...     |              .. | ...     | RVVMF4HI  |   512    |
+   | V2048HI    | 32768     |              16 | NO      | RVVMF4HI  |   512    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1SI/SF    |    32     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V2SI/SF    |    64     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V4SI/SF    |   128     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V8SI/SF    |   256     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V16SI/SF   |   512     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V32SI/SF   |  1024     |              32 | NO      | RVVMF2SI  |  1024    |
+   | V64SI/SF   |  2048     |              32 | NO      | RVVMF2SI  |  1024    |
+   | ...        |   ...     |              .. | ...     | RVVMF2SI  |  1024    |
+   | V1024SI/SF | 32768     |              32 | NO      | RVVMF2SI  |  1024    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1DI/DF    |    64     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V2DI/DF    |   128     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V4DI/DF    |   256     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V8DI/DF    |   512     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V16DI/DF   |  1024     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V32DI/DF   |  2048     |              64 | NO      | RVVM1DI   |  2048    |
+   | V64DI/DF   |  4096     |              64 | NO      | RVVM1DI   |  2048    |
+   | ...        |   ...     |              .. | ...     | RVVM1DI   |  2048    |
+   | V512DI/DF  | 32768     |              64 | NO      | RVVM1DI   |  2048    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+
+   Then we can have the condition for VLS mode in fixed-vlmax, aka:
+     PRECISION (VLSmode) < VLEN / (64 / PRECISION(VLS_inner_mode)).  */
+bool
+legal_vls_mode_in_fixed_vlmax_p (machine_mode vls_mode)
+{
+  machine_mode inner_mode = GET_MODE_INNER (vls_mode);
+
+  int min_vlmax_bitsize
+    = TARGET_MIN_VLEN / (64 / GET_MODE_PRECISION (inner_mode).to_constant ());
+
+  return GET_MODE_PRECISION (vls_mode).to_constant () < min_vlmax_bitsize;
+}
+
 /* Return true if the gather/scatter offset mode is valid.  */
 bool
 gather_scatter_valid_offset_mode_p (machine_mode mode)
diff --git a/gcc/config/riscv/riscv-vector-switch.def b/gcc/config/riscv/riscv-vector-switch.def
index 174e5a181c6..c1eeba2ca64 100644
--- a/gcc/config/riscv/riscv-vector-switch.def
+++ b/gcc/config/riscv/riscv-vector-switch.def
@@ -292,100 +292,100 @@ TUPLE_ENTRY (RVVM1x2DF, TARGET_VECTOR_ELEN_FP_64, RVVM1DF, 2, LMUL_1, 16)
 
 /* This following VLS modes should satisfy the constraint:
    GET_MODE_BITSIZE (MODE) <= TARGET_MIN_VLEN * 8.  */
-VLS_ENTRY (V1BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V2DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V4DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V8DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V2HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V4HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V8HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V16HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V32HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V2SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V4SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V8SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V16SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V2DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V4DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V8DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1BI, TARGET_VECTOR_VLS (V1BImode))
+VLS_ENTRY (V2BI, TARGET_VECTOR_VLS (V2BImode))
+VLS_ENTRY (V4BI, TARGET_VECTOR_VLS (V4BImode))
+VLS_ENTRY (V8BI, TARGET_VECTOR_VLS (V8BImode))
+VLS_ENTRY (V16BI, TARGET_VECTOR_VLS (V16BImode))
+VLS_ENTRY (V32BI, TARGET_VECTOR_VLS (V32BImode))
+VLS_ENTRY (V64BI, TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128BI, TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256BI, TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512BI, TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1QI, TARGET_VECTOR_VLS (V1QImode))
+VLS_ENTRY (V2QI, TARGET_VECTOR_VLS (V2QImode))
+VLS_ENTRY (V4QI, TARGET_VECTOR_VLS (V4QImode))
+VLS_ENTRY (V8QI, TARGET_VECTOR_VLS (V8QImode))
+VLS_ENTRY (V16QI, TARGET_VECTOR_VLS (V16QImode))
+VLS_ENTRY (V32QI, TARGET_VECTOR_VLS (V32QImode))
+VLS_ENTRY (V64QI, TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128QI, TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256QI, TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512QI, TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1HI, TARGET_VECTOR_VLS (V1HImode))
+VLS_ENTRY (V2HI, TARGET_VECTOR_VLS (V2HImode))
+VLS_ENTRY (V4HI, TARGET_VECTOR_VLS (V4HImode))
+VLS_ENTRY (V8HI, TARGET_VECTOR_VLS (V8HImode))
+VLS_ENTRY (V16HI, TARGET_VECTOR_VLS (V16HImode))
+VLS_ENTRY (V32HI, TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HI, TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HI, TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HI, TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HI, TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SI, TARGET_VECTOR_VLS (V1SImode))
+VLS_ENTRY (V2SI, TARGET_VECTOR_VLS (V2SImode))
+VLS_ENTRY (V4SI, TARGET_VECTOR_VLS (V4SImode))
+VLS_ENTRY (V8SI, TARGET_VECTOR_VLS (V8SImode))
+VLS_ENTRY (V16SI, TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SI, TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SI, TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SI, TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SI, TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SI, TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DI, TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V2DI, TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V4DI, TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V8DI, TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DI, TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DI, TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DI, TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DI, TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DI, TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DI, TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1HF, TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V2HF, TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V4HF, TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V8HF, TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V16HF, TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V32HF, TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HF, TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HF, TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HF, TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HF, TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SF, TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V2SF, TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V4SF, TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V8SF, TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V16SF, TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SF, TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SF, TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SF, TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SF, TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SF, TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DF, TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V2DF, TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V4DF, TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V8DF, TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DF, TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DF, TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DF, TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DF, TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DF, TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DF, TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
 
 #undef VLS_ENTRY
 #undef TUPLE_ENTRY
diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md
index 0850475edc1..2eac4355ee5 100644
--- a/gcc/config/riscv/vector-iterators.md
+++ b/gcc/config/riscv/vector-iterators.md
@@ -1,3 +1,4 @@
+
 ;; Iterators for RISC-V 'V' Extension for GNU compiler.
 ;; Copyright (C) 2022-2023 Free Software Foundation, Inc.
 ;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies Ltd.
@@ -146,85 +147,85 @@ (define_mode_iterator V_VLS [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
   ;; VLS modes.
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VEEWEXT2 [
@@ -315,84 +316,84 @@ (define_mode_iterator VEI16 [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI [
@@ -416,52 +417,52 @@ (define_mode_iterator V_VLSI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator V_VLSF [
@@ -473,39 +474,39 @@ (define_mode_iterator V_VLSF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VF_ZVFHMIN [
@@ -531,39 +532,39 @@ (define_mode_iterator V_VLSF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 ;; This iterator is the same as above but with TARGET_VECTOR_ELEN_FP_16
@@ -594,52 +595,52 @@ (define_mode_iterator VFULLI [
 
   (RVVM8DI "TARGET_FULL_V") (RVVM4DI "TARGET_FULL_V") (RVVM2DI "TARGET_FULL_V") (RVVM1DI "TARGET_FULL_V")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_FULL_V")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_FULL_V")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_FULL_V")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI_QH [
@@ -655,42 +656,42 @@ (define_mode_iterator VI_QHS [
 
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI_QHS_NO_M8 [
@@ -700,39 +701,39 @@ (define_mode_iterator VI_QHS_NO_M8 [
 
   RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
 ])
 
 (define_mode_iterator VF_HS [
@@ -743,29 +744,29 @@ (define_mode_iterator VF_HS [
   (RVVM8SF "TARGET_VECTOR_ELEN_FP_32") (RVVM4SF "TARGET_VECTOR_ELEN_FP_32") (RVVM2SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32") (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VF_HS_NO_M8 [
@@ -779,27 +780,27 @@ (define_mode_iterator VF_HS_NO_M8 [
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
 ])
 
 (define_mode_iterator VF_HS_M8 [
@@ -814,42 +815,42 @@ (define_mode_iterator V_VLSI_QHS [
 
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI_D [
@@ -861,16 +862,16 @@ (define_mode_iterator V_VLSI_D [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VFULLI_D [
@@ -1013,19 +1014,19 @@ (define_mode_iterator VB [
 
 (define_mode_iterator VB_VLS [
   (RVVMF64BI "TARGET_MIN_VLEN > 32") RVVMF32BI RVVMF16BI RVVMF8BI RVVMF4BI RVVMF2BI RVVM1BI
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWEXTI [
@@ -1036,39 +1037,39 @@ (define_mode_iterator VWEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 ;; Same iterator split reason as VF_ZVFHMIN and VF.
@@ -1082,27 +1083,27 @@ (define_mode_iterator VWEXTF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWEXTF [
@@ -1115,27 +1116,27 @@ (define_mode_iterator VWEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWCONVERTI [
@@ -1147,27 +1148,27 @@ (define_mode_iterator VWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
 
-  (V1SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode) && TARGET_ZVFH")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode) && TARGET_ZVFH")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode) && TARGET_ZVFH")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode) && TARGET_ZVFH")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWWCONVERTI [
@@ -1176,16 +1177,16 @@ (define_mode_iterator VWWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
 
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VQEXTI [
@@ -1194,59 +1195,59 @@ (define_mode_iterator VQEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VQEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VOEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VT [
@@ -3216,28 +3217,28 @@ (define_mode_iterator V_VLS_FCONVERT_I_L_LL [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_attr VDEMOTE [
@@ -3748,281 +3749,281 @@ (define_code_attr sz [(sign_extend "s") (zero_extend "z")])
 
 ;; VLS modes.
 (define_mode_iterator VLS [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
 
 (define_mode_iterator VLSB [
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
 
 ;; VLS modes that has NUNITS < 32.
 (define_mode_iterator VLS_AVL_IMM [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")])
 
 ;; VLS modes that has NUNITS >= 32.
 (define_mode_iterator VLS_AVL_REG [
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
-
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
 
 (define_mode_iterator VLSI [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
 
 (define_mode_iterator VLSF [
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
index 7ab31043547..b235ec727b1 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
@@ -55,7 +55,7 @@
 
 TEST_ALL (PERMUTE)
 
-/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 19 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 18 } } */
 /* { dg-final { scan-assembler-times {vrgatherei16\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 12 } } */
-/* { dg-final { scan-assembler-times {vrsub\.vi} 24 } } */
+/* { dg-final { scan-assembler-times {vrsub\.vi} 23 } } */
 /* { dg-final { scan-assembler-times {vrsub\.vx} 7 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
new file mode 100644
index 00000000000..57af7db5cb1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
@@ -0,0 +1,536 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1qi,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2qi,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4qi,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8qi,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v4096qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 10 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
new file mode 100644
index 00000000000..2619658fc9b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hi,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hi,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hi,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hi,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
new file mode 100644
index 00000000000..16aadba1276
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1si,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2si,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4si,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8si,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
new file mode 100644
index 00000000000..5c5e93aef8c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1di,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2di,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4di,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8di,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
new file mode 100644
index 00000000000..6aa4a7e615c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hf,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hf,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hf,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hf,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*[v]?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
new file mode 100644
index 00000000000..a31dad5a926
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1sf,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2sf,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4sf,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8sf,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
new file mode 100644
index 00000000000..8312b21c497
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1df,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2df,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4df,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8df,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
index 8dd5bcf617d..8ee8b7b1801 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
@@ -441,6 +441,12 @@ typedef double v512df __attribute__ ((vector_size (4096)));
     *(TYPE1 *) out = v;                                                        \
   }
 
+#define DEF_OP_VV_VA(OP, TYPE1, ...)                                           \
+  TYPE1 test_##OP##_##TYPE1 (TYPE1 a, TYPE1 b)                                 \
+  {                                                                            \
+    return OP (a, b, __VA_ARGS__);                                             \
+  }
+
 #define DEF_REPEAT(TYPE1, TYPE2, NUM, ...)                                     \
   void init_##TYPE1##_##TYPE2##_##NUM (TYPE2 var0, TYPE2 var1,                 \
 				       TYPE2 *__restrict out)                  \
-- 
2.34.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax
  2023-10-20  8:34 [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax pan2.li
@ 2023-10-20  8:47 ` juzhe.zhong
  2023-10-20  8:51   ` Li, Pan2
  2023-10-20 12:59 ` [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857] pan2.li
  1 sibling, 1 reply; 6+ messages in thread
From: juzhe.zhong @ 2023-10-20  8:47 UTC (permalink / raw)
  To: pan2.li, gcc-patches; +Cc: pan2.li, yanzhang.wang, kito.cheng

[-- Attachment #1: Type: text/plain, Size: 274412 bytes --]


 /* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode
-   is the highest priority choice and should not conflict with VLS modes.  */
-#define TARGET_VECTOR_VLS                                                      \
-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)
+   is the highest priority choice and should not conflict with VLS modes.
+
+   We also enable VLS modes for some cases in fixed-vlmax, aka the bitsize of
+   the VLS mode are smaller than the minimal vla.
+   */
+#define TARGET_VECTOR_VLS(mode)                                                \
+  (TARGET_VECTOR                                                               \
+    && (riscv_autovec_preference == RVV_SCALABLE                               \
+      || (riscv_autovec_preference == RVV_FIXED_VLMAX                          \
+	&& riscv_vector::legal_vls_mode_in_fixed_vlmax_p (mode))))
Remove this macro. Create a function called "vls_mode_valid_p"and use riscv_vector::vls_mode_valid_p in all places.

Indicate this PR in commit title: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111857

-> RISC-V: Support partial VLS mode when preference fixed-vlmax[PR111857]

+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c

Change  all tests name into pr111857-1.c...etc.



juzhe.zhong@rivai.ai
 
From: pan2.li
Date: 2023-10-20 16:34
To: gcc-patches
CC: juzhe.zhong; pan2.li; yanzhang.wang; kito.cheng
Subject: [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax
From: Pan Li <pan2.li@intel.com>
 
Given we have code like below:
 
typedef char vnx16i __attribute__ ((vector_size (16)));
 
vnx16i __attribute__ ((noinline, noclone))
test (vnx16i x, vnx16i y)
{
  return __builtin_shufflevector (x, y, 11, 12, 13, 14, 11, 12, 13, 14,
                                        11, 12, 13, 14, 11, 12, 13, 14);
}
 
It can perform the auto vectorization when
 
-march=rv64gcv_zvl1024b --param=riscv-autovec-preference=fixed-vlmax
 
but cannot when
 
-march=rv64gcv_zvl2048b --param=riscv-autovec-preference=fixed-vlmax
 
The reason comes from the miniaml machine mode of QI is RVVMF8QI, which
is 1024 / 8 = 128 bits, aka the size of VNx16QI. When we set zvl2048b,
the bit size of RVVMFQI is 2048 / 8 = 256, which is not matching the
bit size of VNx16QI (128 bits).
 
Thus, this patch would like to enable the VLS mode for such case, aka
VNx16QI vls mode for zvl2048b.
 
Before this patch:
test:
  srli    a4,a1,40
  andi    a4,a4,0xff
  srli    a3,a1,32
  srli    a5,a1,48
  slli    a0,a4,8
  andi    a3,a3,0xff
  andi    a5,a5,0xff
  slli    a2,a5,16
  or      a0,a3,a0
  srli    a1,a1,56
  or      a0,a0,a2
  slli    a2,a1,24
  slli    a3,a3,32
  or      a0,a0,a2
  slli    a4,a4,40
  or      a0,a0,a3
  slli    a5,a5,48
  or      a0,a0,a4
  or      a0,a0,a5
  slli    a1,a1,56
  or      a0,a0,a1
  mv      a1,a0
  ret
 
After this patch:
test:
  vsetivli        zero,16,e8,mf8,ta,ma
  vle8.v  v2,0(a1)
  vsetivli        zero,4,e32,mf2,ta,ma
  vrgather.vi     v1,v2,3
  vsetivli        zero,16,e8,mf8,ta,ma
  vse8.v  v1,0(a0)
  ret
 
gcc/ChangeLog:
 
* config/riscv/riscv-opts.h (TARGET_VECTOR_VLS): Add enabling
condition when fixed-vlmax by passing mode as arg.
* config/riscv/riscv-protos.h (legal_vls_mode_in_fixed_vlmax_p):
New func decl.
* config/riscv/riscv-v.cc (legal_vls_mode_in_fixed_vlmax_p):
New func impl to check the vls mode is legal when fixed-vlmax.
(autovectorize_vector_modes): Pass mode arg to TARGET_VECTOR_VLS.
* config/riscv/riscv-vector-switch.def (VLS_ENTRY): Ditto.
* config/riscv/vector-iterators.md: Ditto.
 
gcc/testsuite/ChangeLog:
 
* gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c: Adjust checker.
* gcc.target/riscv/rvv/autovec/vls/def.h: Add help define.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c: New test.
 
Signed-off-by: Pan Li <pan2.li@intel.com>
---
gcc/config/riscv/riscv-opts.h                 |   13 +-
gcc/config/riscv/riscv-protos.h               |    1 +
gcc/config/riscv/riscv-v.cc                   |   98 +-
gcc/config/riscv/riscv-vector-switch.def      |  188 +-
gcc/config/riscv/vector-iterators.md          | 1843 +++++++++--------
.../riscv/rvv/autovec/vls-vlmax/perm-4.c      |    4 +-
.../riscv/rvv/autovec/vls-vlmax/vls-mode-0.c  |  536 +++++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-1.c  |  279 +++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-2.c  |  151 ++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-3.c  |   86 +
.../riscv/rvv/autovec/vls-vlmax/vls-mode-4.c  |  279 +++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-5.c  |  151 ++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-6.c  |   86 +
.../gcc.target/riscv/rvv/autovec/vls/def.h    |    6 +
14 files changed, 2688 insertions(+), 1033 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
 
diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
index 31ee42dea6b..9992800331a 100644
--- a/gcc/config/riscv/riscv-opts.h
+++ b/gcc/config/riscv/riscv-opts.h
@@ -119,9 +119,16 @@ enum riscv_entity
      : 32 << (__builtin_popcount (opts->x_riscv_zvl_flags) - 1))
/* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode
-   is the highest priority choice and should not conflict with VLS modes.  */
-#define TARGET_VECTOR_VLS                                                      \
-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)
+   is the highest priority choice and should not conflict with VLS modes.
+
+   We also enable VLS modes for some cases in fixed-vlmax, aka the bitsize of
+   the VLS mode are smaller than the minimal vla.
+   */
+#define TARGET_VECTOR_VLS(mode)                                                \
+  (TARGET_VECTOR                                                               \
+    && (riscv_autovec_preference == RVV_SCALABLE                               \
+      || (riscv_autovec_preference == RVV_FIXED_VLMAX                          \
+ && riscv_vector::legal_vls_mode_in_fixed_vlmax_p (mode))))
/* TODO: Enable RVV movmisalign by default for now.  */
#define TARGET_VECTOR_MISALIGN_SUPPORTED 1
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 6190faab501..c2f0da01c0f 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -552,6 +552,7 @@ unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
bool cmp_lmul_le_one (machine_mode);
bool cmp_lmul_gt_one (machine_mode);
bool gather_scatter_valid_offset_mode_p (machine_mode);
+bool legal_vls_mode_in_fixed_vlmax_p (machine_mode);
}
/* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 895c11d13fc..a8ba0d46e1f 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -2423,20 +2423,19 @@ autovectorize_vector_modes (vector_modes *modes, bool)
    modes->safe_push (mode);
}
     }
-  if (TARGET_VECTOR_VLS)
-    {
-      /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
-      unsigned int i = 0;
-      unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
-      unsigned int size = base_size;
-      machine_mode mode;
-      while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
- {
+    /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
+    unsigned int i = 0;
+    unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
+    unsigned int size = base_size;
+    machine_mode mode;
+    while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
+     {
+ if (TARGET_VECTOR_VLS (mode))
  modes->safe_push (mode);
-   i++;
-   size = base_size / (1U << i);
- }
-    }
+
+ i++;
+ size = base_size / (1U << i);
+     }
   /* Enable LOOP_VINFO comparison in COST model.  */
   return VECT_COMPARE_COSTS;
}
@@ -3880,6 +3879,79 @@ cmp_lmul_gt_one (machine_mode mode)
   return false;
}
+/* Return true if the VLS mode is legal when preference fixed-vlmax,
+   Take vlen = 2048 as example.
+
+   Note: Below table based on vlen = 2048.
+   +----------------------------------------------------+----------------------+
+   | VLS mode                                           | VLA mode             |
+   +----------------------------------------------------+----------------------+
+   | Name       | Precision | Inner Precision | Enabled | Min mode  | Min bits |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1BI       |     1     |              1  | Yes     | RVVMF64BI |    32    |
+   | V2BI       |     2     |              1  | Yes     | RVVMF64BI |    32    |
+   | V4BI       |     4     |              1  | Yes     | RVVMF64BI |    32    |
+   | V8BI       |     8     |              1  | Yes     | RVVMF64BI |    32    |
+   | V16BI      |    16     |              1  | Yes     | RVVMF64BI |    32    |
+   | V32BI      |    32     |              1  | NO      | RVVMF64BI |    32    |
+   | V64BI      |    64     |              1  | NO      | RVVMF64BI |    32    |
+   | ...        |   ...     |            ...  | ...     | RVVMF64BI |    32    |
+   | V4096BI    |  4096     |              1  | NO      | RVVMF64BI |    32    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1QI       |     8     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V2QI       |    16     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V4QI       |    32     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V8QI       |    64     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V16QI      |   128     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V32QI      |   256     |              8  | NO      | RVVMF8QI  |   256    |
+   | V64QI      |   512     |              8  | NO      | RVVMF8QI  |   256    |
+   | ...        |   ...     |              .. | ...     | RVVMF8QI  |   256    |
+   | V4096QI    | 32768     |              8  | NO      | RVVMF8QI  |   256    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1HI       |    16     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V2HI       |    32     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V4HI       |    64     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V8HI       |   128     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V16HI      |   256     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V32HI      |   512     |              16 | NO      | RVVMF4HI  |   512    |
+   | V64HI      |  1024     |              16 | NO      | RVVMF4HI  |   512    |
+   | ...        |   ...     |              .. | ...     | RVVMF4HI  |   512    |
+   | V2048HI    | 32768     |              16 | NO      | RVVMF4HI  |   512    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1SI/SF    |    32     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V2SI/SF    |    64     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V4SI/SF    |   128     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V8SI/SF    |   256     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V16SI/SF   |   512     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V32SI/SF   |  1024     |              32 | NO      | RVVMF2SI  |  1024    |
+   | V64SI/SF   |  2048     |              32 | NO      | RVVMF2SI  |  1024    |
+   | ...        |   ...     |              .. | ...     | RVVMF2SI  |  1024    |
+   | V1024SI/SF | 32768     |              32 | NO      | RVVMF2SI  |  1024    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1DI/DF    |    64     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V2DI/DF    |   128     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V4DI/DF    |   256     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V8DI/DF    |   512     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V16DI/DF   |  1024     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V32DI/DF   |  2048     |              64 | NO      | RVVM1DI   |  2048    |
+   | V64DI/DF   |  4096     |              64 | NO      | RVVM1DI   |  2048    |
+   | ...        |   ...     |              .. | ...     | RVVM1DI   |  2048    |
+   | V512DI/DF  | 32768     |              64 | NO      | RVVM1DI   |  2048    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+
+   Then we can have the condition for VLS mode in fixed-vlmax, aka:
+     PRECISION (VLSmode) < VLEN / (64 / PRECISION(VLS_inner_mode)).  */
+bool
+legal_vls_mode_in_fixed_vlmax_p (machine_mode vls_mode)
+{
+  machine_mode inner_mode = GET_MODE_INNER (vls_mode);
+
+  int min_vlmax_bitsize
+    = TARGET_MIN_VLEN / (64 / GET_MODE_PRECISION (inner_mode).to_constant ());
+
+  return GET_MODE_PRECISION (vls_mode).to_constant () < min_vlmax_bitsize;
+}
+
/* Return true if the gather/scatter offset mode is valid.  */
bool
gather_scatter_valid_offset_mode_p (machine_mode mode)
diff --git a/gcc/config/riscv/riscv-vector-switch.def b/gcc/config/riscv/riscv-vector-switch.def
index 174e5a181c6..c1eeba2ca64 100644
--- a/gcc/config/riscv/riscv-vector-switch.def
+++ b/gcc/config/riscv/riscv-vector-switch.def
@@ -292,100 +292,100 @@ TUPLE_ENTRY (RVVM1x2DF, TARGET_VECTOR_ELEN_FP_64, RVVM1DF, 2, LMUL_1, 16)
/* This following VLS modes should satisfy the constraint:
    GET_MODE_BITSIZE (MODE) <= TARGET_MIN_VLEN * 8.  */
-VLS_ENTRY (V1BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V2DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V4DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V8DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V2HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V4HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V8HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V16HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V32HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V2SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V4SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V8SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V16SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V2DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V4DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V8DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1BI, TARGET_VECTOR_VLS (V1BImode))
+VLS_ENTRY (V2BI, TARGET_VECTOR_VLS (V2BImode))
+VLS_ENTRY (V4BI, TARGET_VECTOR_VLS (V4BImode))
+VLS_ENTRY (V8BI, TARGET_VECTOR_VLS (V8BImode))
+VLS_ENTRY (V16BI, TARGET_VECTOR_VLS (V16BImode))
+VLS_ENTRY (V32BI, TARGET_VECTOR_VLS (V32BImode))
+VLS_ENTRY (V64BI, TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128BI, TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256BI, TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512BI, TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1QI, TARGET_VECTOR_VLS (V1QImode))
+VLS_ENTRY (V2QI, TARGET_VECTOR_VLS (V2QImode))
+VLS_ENTRY (V4QI, TARGET_VECTOR_VLS (V4QImode))
+VLS_ENTRY (V8QI, TARGET_VECTOR_VLS (V8QImode))
+VLS_ENTRY (V16QI, TARGET_VECTOR_VLS (V16QImode))
+VLS_ENTRY (V32QI, TARGET_VECTOR_VLS (V32QImode))
+VLS_ENTRY (V64QI, TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128QI, TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256QI, TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512QI, TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1HI, TARGET_VECTOR_VLS (V1HImode))
+VLS_ENTRY (V2HI, TARGET_VECTOR_VLS (V2HImode))
+VLS_ENTRY (V4HI, TARGET_VECTOR_VLS (V4HImode))
+VLS_ENTRY (V8HI, TARGET_VECTOR_VLS (V8HImode))
+VLS_ENTRY (V16HI, TARGET_VECTOR_VLS (V16HImode))
+VLS_ENTRY (V32HI, TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HI, TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HI, TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HI, TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HI, TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SI, TARGET_VECTOR_VLS (V1SImode))
+VLS_ENTRY (V2SI, TARGET_VECTOR_VLS (V2SImode))
+VLS_ENTRY (V4SI, TARGET_VECTOR_VLS (V4SImode))
+VLS_ENTRY (V8SI, TARGET_VECTOR_VLS (V8SImode))
+VLS_ENTRY (V16SI, TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SI, TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SI, TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SI, TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SI, TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SI, TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DI, TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V2DI, TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V4DI, TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V8DI, TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DI, TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DI, TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DI, TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DI, TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DI, TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DI, TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1HF, TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V2HF, TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V4HF, TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V8HF, TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V16HF, TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V32HF, TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HF, TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HF, TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HF, TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HF, TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SF, TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V2SF, TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V4SF, TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V8SF, TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V16SF, TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SF, TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SF, TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SF, TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SF, TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SF, TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DF, TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V2DF, TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V4DF, TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V8DF, TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DF, TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DF, TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DF, TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DF, TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DF, TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DF, TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
#undef VLS_ENTRY
#undef TUPLE_ENTRY
diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md
index 0850475edc1..2eac4355ee5 100644
--- a/gcc/config/riscv/vector-iterators.md
+++ b/gcc/config/riscv/vector-iterators.md
@@ -1,3 +1,4 @@
+
;; Iterators for RISC-V 'V' Extension for GNU compiler.
;; Copyright (C) 2022-2023 Free Software Foundation, Inc.
;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies Ltd.
@@ -146,85 +147,85 @@ (define_mode_iterator V_VLS [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
   ;; VLS modes.
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VEEWEXT2 [
@@ -315,84 +316,84 @@ (define_mode_iterator VEI16 [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI [
@@ -416,52 +417,52 @@ (define_mode_iterator V_VLSI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator V_VLSF [
@@ -473,39 +474,39 @@ (define_mode_iterator V_VLSF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VF_ZVFHMIN [
@@ -531,39 +532,39 @@ (define_mode_iterator V_VLSF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
;; This iterator is the same as above but with TARGET_VECTOR_ELEN_FP_16
@@ -594,52 +595,52 @@ (define_mode_iterator VFULLI [
   (RVVM8DI "TARGET_FULL_V") (RVVM4DI "TARGET_FULL_V") (RVVM2DI "TARGET_FULL_V") (RVVM1DI "TARGET_FULL_V")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_FULL_V")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_FULL_V")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_FULL_V")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_QH [
@@ -655,42 +656,42 @@ (define_mode_iterator VI_QHS [
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_QHS_NO_M8 [
@@ -700,39 +701,39 @@ (define_mode_iterator VI_QHS_NO_M8 [
   RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
])
(define_mode_iterator VF_HS [
@@ -743,29 +744,29 @@ (define_mode_iterator VF_HS [
   (RVVM8SF "TARGET_VECTOR_ELEN_FP_32") (RVVM4SF "TARGET_VECTOR_ELEN_FP_32") (RVVM2SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32") (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VF_HS_NO_M8 [
@@ -779,27 +780,27 @@ (define_mode_iterator VF_HS_NO_M8 [
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
])
(define_mode_iterator VF_HS_M8 [
@@ -814,42 +815,42 @@ (define_mode_iterator V_VLSI_QHS [
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_D [
@@ -861,16 +862,16 @@ (define_mode_iterator V_VLSI_D [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VFULLI_D [
@@ -1013,19 +1014,19 @@ (define_mode_iterator VB [
(define_mode_iterator VB_VLS [
   (RVVMF64BI "TARGET_MIN_VLEN > 32") RVVMF32BI RVVMF16BI RVVMF8BI RVVMF4BI RVVMF2BI RVVM1BI
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWEXTI [
@@ -1036,39 +1037,39 @@ (define_mode_iterator VWEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
;; Same iterator split reason as VF_ZVFHMIN and VF.
@@ -1082,27 +1083,27 @@ (define_mode_iterator VWEXTF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWEXTF [
@@ -1115,27 +1116,27 @@ (define_mode_iterator VWEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWCONVERTI [
@@ -1147,27 +1148,27 @@ (define_mode_iterator VWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V1SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode) && TARGET_ZVFH")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode) && TARGET_ZVFH")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode) && TARGET_ZVFH")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode) && TARGET_ZVFH")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWWCONVERTI [
@@ -1176,16 +1177,16 @@ (define_mode_iterator VWWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VQEXTI [
@@ -1194,59 +1195,59 @@ (define_mode_iterator VQEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VQEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VOEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VT [
@@ -3216,28 +3217,28 @@ (define_mode_iterator V_VLS_FCONVERT_I_L_LL [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_attr VDEMOTE [
@@ -3748,281 +3749,281 @@ (define_code_attr sz [(sign_extend "s") (zero_extend "z")])
;; VLS modes.
(define_mode_iterator VLS [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSB [
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
;; VLS modes that has NUNITS < 32.
(define_mode_iterator VLS_AVL_IMM [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")])
;; VLS modes that has NUNITS >= 32.
(define_mode_iterator VLS_AVL_REG [
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
-
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSI [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSF [
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
index 7ab31043547..b235ec727b1 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
@@ -55,7 +55,7 @@
TEST_ALL (PERMUTE)
-/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 19 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 18 } } */
/* { dg-final { scan-assembler-times {vrgatherei16\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 12 } } */
-/* { dg-final { scan-assembler-times {vrsub\.vi} 24 } } */
+/* { dg-final { scan-assembler-times {vrsub\.vi} 23 } } */
/* { dg-final { scan-assembler-times {vrsub\.vx} 7 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
new file mode 100644
index 00000000000..57af7db5cb1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
@@ -0,0 +1,536 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1qi,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2qi,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4qi,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8qi,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v4096qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 10 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
new file mode 100644
index 00000000000..2619658fc9b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hi,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hi,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hi,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hi,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
new file mode 100644
index 00000000000..16aadba1276
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1si,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2si,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4si,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8si,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
new file mode 100644
index 00000000000..5c5e93aef8c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1di,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2di,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4di,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8di,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
new file mode 100644
index 00000000000..6aa4a7e615c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hf,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hf,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hf,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hf,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*[v]?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
new file mode 100644
index 00000000000..a31dad5a926
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1sf,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2sf,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4sf,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8sf,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
new file mode 100644
index 00000000000..8312b21c497
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1df,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2df,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4df,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8df,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
index 8dd5bcf617d..8ee8b7b1801 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
@@ -441,6 +441,12 @@ typedef double v512df __attribute__ ((vector_size (4096)));
     *(TYPE1 *) out = v;                                                        \
   }
+#define DEF_OP_VV_VA(OP, TYPE1, ...)                                           \
+  TYPE1 test_##OP##_##TYPE1 (TYPE1 a, TYPE1 b)                                 \
+  {                                                                            \
+    return OP (a, b, __VA_ARGS__);                                             \
+  }
+
#define DEF_REPEAT(TYPE1, TYPE2, NUM, ...)                                     \
   void init_##TYPE1##_##TYPE2##_##NUM (TYPE2 var0, TYPE2 var1,                 \
       TYPE2 *__restrict out)                  \
-- 
2.34.1
 
 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax
  2023-10-20  8:47 ` juzhe.zhong
@ 2023-10-20  8:51   ` Li, Pan2
  0 siblings, 0 replies; 6+ messages in thread
From: Li, Pan2 @ 2023-10-20  8:51 UTC (permalink / raw)
  To: juzhe.zhong, gcc-patches; +Cc: Wang, Yanzhang, kito.cheng

[-- Attachment #1: Type: text/plain, Size: 275175 bytes --]

Thanks Juzhe, will send V2 after test.

Pan

From: juzhe.zhong@rivai.ai <juzhe.zhong@rivai.ai>
Sent: Friday, October 20, 2023 4:47 PM
To: Li, Pan2 <pan2.li@intel.com>; gcc-patches <gcc-patches@gcc.gnu.org>
Cc: Li, Pan2 <pan2.li@intel.com>; Wang, Yanzhang <yanzhang.wang@intel.com>; kito.cheng <kito.cheng@gmail.com>
Subject: Re: [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax



 /* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode

-   is the highest priority choice and should not conflict with VLS modes.  */

-#define TARGET_VECTOR_VLS                                                      \

-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)

+   is the highest priority choice and should not conflict with VLS modes.

+

+   We also enable VLS modes for some cases in fixed-vlmax, aka the bitsize of

+   the VLS mode are smaller than the minimal vla.

+   */

+#define TARGET_VECTOR_VLS(mode)                                                \

+  (TARGET_VECTOR                                                               \

+    && (riscv_autovec_preference == RVV_SCALABLE                               \

+      || (riscv_autovec_preference == RVV_FIXED_VLMAX                          \

+       && riscv_vector::legal_vls_mode_in_fixed_vlmax_p (mode))))



Remove this macro. Create a function called "vls_mode_valid_p"

and use riscv_vector::vls_mode_valid_p in all places.

Indicate this PR in commit title: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111857

-> RISC-V: Support partial VLS mode when preference fixed-vlmax[PR111857]


+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
Change  all tests name into pr111857-1.c...etc.

________________________________
juzhe.zhong@rivai.ai<mailto:juzhe.zhong@rivai.ai>

From: pan2.li<mailto:pan2.li@intel.com>
Date: 2023-10-20 16:34
To: gcc-patches<mailto:gcc-patches@gcc.gnu.org>
CC: juzhe.zhong<mailto:juzhe.zhong@rivai.ai>; pan2.li<mailto:pan2.li@intel.com>; yanzhang.wang<mailto:yanzhang.wang@intel.com>; kito.cheng<mailto:kito.cheng@gmail.com>
Subject: [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax
From: Pan Li <pan2.li@intel.com<mailto:pan2.li@intel.com>>

Given we have code like below:

typedef char vnx16i __attribute__ ((vector_size (16)));

vnx16i __attribute__ ((noinline, noclone))
test (vnx16i x, vnx16i y)
{
  return __builtin_shufflevector (x, y, 11, 12, 13, 14, 11, 12, 13, 14,
                                        11, 12, 13, 14, 11, 12, 13, 14);
}

It can perform the auto vectorization when

-march=rv64gcv_zvl1024b --param=riscv-autovec-preference=fixed-vlmax

but cannot when

-march=rv64gcv_zvl2048b --param=riscv-autovec-preference=fixed-vlmax

The reason comes from the miniaml machine mode of QI is RVVMF8QI, which
is 1024 / 8 = 128 bits, aka the size of VNx16QI. When we set zvl2048b,
the bit size of RVVMFQI is 2048 / 8 = 256, which is not matching the
bit size of VNx16QI (128 bits).

Thus, this patch would like to enable the VLS mode for such case, aka
VNx16QI vls mode for zvl2048b.

Before this patch:
test:
  srli    a4,a1,40
  andi    a4,a4,0xff
  srli    a3,a1,32
  srli    a5,a1,48
  slli    a0,a4,8
  andi    a3,a3,0xff
  andi    a5,a5,0xff
  slli    a2,a5,16
  or      a0,a3,a0
  srli    a1,a1,56
  or      a0,a0,a2
  slli    a2,a1,24
  slli    a3,a3,32
  or      a0,a0,a2
  slli    a4,a4,40
  or      a0,a0,a3
  slli    a5,a5,48
  or      a0,a0,a4
  or      a0,a0,a5
  slli    a1,a1,56
  or      a0,a0,a1
  mv      a1,a0
  ret

After this patch:
test:
  vsetivli        zero,16,e8,mf8,ta,ma
  vle8.v  v2,0(a1)
  vsetivli        zero,4,e32,mf2,ta,ma
  vrgather.vi     v1,v2,3
  vsetivli        zero,16,e8,mf8,ta,ma
  vse8.v  v1,0(a0)
  ret

gcc/ChangeLog:

* config/riscv/riscv-opts.h (TARGET_VECTOR_VLS): Add enabling
condition when fixed-vlmax by passing mode as arg.
* config/riscv/riscv-protos.h (legal_vls_mode_in_fixed_vlmax_p):
New func decl.
* config/riscv/riscv-v.cc (legal_vls_mode_in_fixed_vlmax_p):
New func impl to check the vls mode is legal when fixed-vlmax.
(autovectorize_vector_modes): Pass mode arg to TARGET_VECTOR_VLS.
* config/riscv/riscv-vector-switch.def (VLS_ENTRY): Ditto.
* config/riscv/vector-iterators.md: Ditto.

gcc/testsuite/ChangeLog:

* gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c: Adjust checker.
* gcc.target/riscv/rvv/autovec/vls/def.h: Add help define.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c: New test.

Signed-off-by: Pan Li <pan2.li@intel.com<mailto:pan2.li@intel.com>>
---
gcc/config/riscv/riscv-opts.h                 |   13 +-
gcc/config/riscv/riscv-protos.h               |    1 +
gcc/config/riscv/riscv-v.cc                   |   98 +-
gcc/config/riscv/riscv-vector-switch.def      |  188 +-
gcc/config/riscv/vector-iterators.md          | 1843 +++++++++--------
.../riscv/rvv/autovec/vls-vlmax/perm-4.c      |    4 +-
.../riscv/rvv/autovec/vls-vlmax/vls-mode-0.c  |  536 +++++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-1.c  |  279 +++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-2.c  |  151 ++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-3.c  |   86 +
.../riscv/rvv/autovec/vls-vlmax/vls-mode-4.c  |  279 +++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-5.c  |  151 ++
.../riscv/rvv/autovec/vls-vlmax/vls-mode-6.c  |   86 +
.../gcc.target/riscv/rvv/autovec/vls/def.h    |    6 +
14 files changed, 2688 insertions(+), 1033 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c

diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
index 31ee42dea6b..9992800331a 100644
--- a/gcc/config/riscv/riscv-opts.h
+++ b/gcc/config/riscv/riscv-opts.h
@@ -119,9 +119,16 @@ enum riscv_entity
      : 32 << (__builtin_popcount (opts->x_riscv_zvl_flags) - 1))
/* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode
-   is the highest priority choice and should not conflict with VLS modes.  */
-#define TARGET_VECTOR_VLS                                                      \
-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)
+   is the highest priority choice and should not conflict with VLS modes.
+
+   We also enable VLS modes for some cases in fixed-vlmax, aka the bitsize of
+   the VLS mode are smaller than the minimal vla.
+   */
+#define TARGET_VECTOR_VLS(mode)                                                \
+  (TARGET_VECTOR                                                               \
+    && (riscv_autovec_preference == RVV_SCALABLE                               \
+      || (riscv_autovec_preference == RVV_FIXED_VLMAX                          \
+ && riscv_vector::legal_vls_mode_in_fixed_vlmax_p (mode))))
/* TODO: Enable RVV movmisalign by default for now.  */
#define TARGET_VECTOR_MISALIGN_SUPPORTED 1
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 6190faab501..c2f0da01c0f 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -552,6 +552,7 @@ unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
bool cmp_lmul_le_one (machine_mode);
bool cmp_lmul_gt_one (machine_mode);
bool gather_scatter_valid_offset_mode_p (machine_mode);
+bool legal_vls_mode_in_fixed_vlmax_p (machine_mode);
}
/* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 895c11d13fc..a8ba0d46e1f 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -2423,20 +2423,19 @@ autovectorize_vector_modes (vector_modes *modes, bool)
    modes->safe_push (mode);
}
     }
-  if (TARGET_VECTOR_VLS)
-    {
-      /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
-      unsigned int i = 0;
-      unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
-      unsigned int size = base_size;
-      machine_mode mode;
-      while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
- {
+    /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
+    unsigned int i = 0;
+    unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
+    unsigned int size = base_size;
+    machine_mode mode;
+    while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
+     {
+ if (TARGET_VECTOR_VLS (mode))
  modes->safe_push (mode);
-   i++;
-   size = base_size / (1U << i);
- }
-    }
+
+ i++;
+ size = base_size / (1U << i);
+     }
   /* Enable LOOP_VINFO comparison in COST model.  */
   return VECT_COMPARE_COSTS;
}
@@ -3880,6 +3879,79 @@ cmp_lmul_gt_one (machine_mode mode)
   return false;
}
+/* Return true if the VLS mode is legal when preference fixed-vlmax,
+   Take vlen = 2048 as example.
+
+   Note: Below table based on vlen = 2048.
+   +----------------------------------------------------+----------------------+
+   | VLS mode                                           | VLA mode             |
+   +----------------------------------------------------+----------------------+
+   | Name       | Precision | Inner Precision | Enabled | Min mode  | Min bits |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1BI       |     1     |              1  | Yes     | RVVMF64BI |    32    |
+   | V2BI       |     2     |              1  | Yes     | RVVMF64BI |    32    |
+   | V4BI       |     4     |              1  | Yes     | RVVMF64BI |    32    |
+   | V8BI       |     8     |              1  | Yes     | RVVMF64BI |    32    |
+   | V16BI      |    16     |              1  | Yes     | RVVMF64BI |    32    |
+   | V32BI      |    32     |              1  | NO      | RVVMF64BI |    32    |
+   | V64BI      |    64     |              1  | NO      | RVVMF64BI |    32    |
+   | ...        |   ...     |            ...  | ...     | RVVMF64BI |    32    |
+   | V4096BI    |  4096     |              1  | NO      | RVVMF64BI |    32    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1QI       |     8     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V2QI       |    16     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V4QI       |    32     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V8QI       |    64     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V16QI      |   128     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V32QI      |   256     |              8  | NO      | RVVMF8QI  |   256    |
+   | V64QI      |   512     |              8  | NO      | RVVMF8QI  |   256    |
+   | ...        |   ...     |              .. | ...     | RVVMF8QI  |   256    |
+   | V4096QI    | 32768     |              8  | NO      | RVVMF8QI  |   256    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1HI       |    16     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V2HI       |    32     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V4HI       |    64     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V8HI       |   128     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V16HI      |   256     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V32HI      |   512     |              16 | NO      | RVVMF4HI  |   512    |
+   | V64HI      |  1024     |              16 | NO      | RVVMF4HI  |   512    |
+   | ...        |   ...     |              .. | ...     | RVVMF4HI  |   512    |
+   | V2048HI    | 32768     |              16 | NO      | RVVMF4HI  |   512    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1SI/SF    |    32     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V2SI/SF    |    64     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V4SI/SF    |   128     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V8SI/SF    |   256     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V16SI/SF   |   512     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V32SI/SF   |  1024     |              32 | NO      | RVVMF2SI  |  1024    |
+   | V64SI/SF   |  2048     |              32 | NO      | RVVMF2SI  |  1024    |
+   | ...        |   ...     |              .. | ...     | RVVMF2SI  |  1024    |
+   | V1024SI/SF | 32768     |              32 | NO      | RVVMF2SI  |  1024    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1DI/DF    |    64     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V2DI/DF    |   128     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V4DI/DF    |   256     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V8DI/DF    |   512     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V16DI/DF   |  1024     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V32DI/DF   |  2048     |              64 | NO      | RVVM1DI   |  2048    |
+   | V64DI/DF   |  4096     |              64 | NO      | RVVM1DI   |  2048    |
+   | ...        |   ...     |              .. | ...     | RVVM1DI   |  2048    |
+   | V512DI/DF  | 32768     |              64 | NO      | RVVM1DI   |  2048    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+
+   Then we can have the condition for VLS mode in fixed-vlmax, aka:
+     PRECISION (VLSmode) < VLEN / (64 / PRECISION(VLS_inner_mode)).  */
+bool
+legal_vls_mode_in_fixed_vlmax_p (machine_mode vls_mode)
+{
+  machine_mode inner_mode = GET_MODE_INNER (vls_mode);
+
+  int min_vlmax_bitsize
+    = TARGET_MIN_VLEN / (64 / GET_MODE_PRECISION (inner_mode).to_constant ());
+
+  return GET_MODE_PRECISION (vls_mode).to_constant () < min_vlmax_bitsize;
+}
+
/* Return true if the gather/scatter offset mode is valid.  */
bool
gather_scatter_valid_offset_mode_p (machine_mode mode)
diff --git a/gcc/config/riscv/riscv-vector-switch.def b/gcc/config/riscv/riscv-vector-switch.def
index 174e5a181c6..c1eeba2ca64 100644
--- a/gcc/config/riscv/riscv-vector-switch.def
+++ b/gcc/config/riscv/riscv-vector-switch.def
@@ -292,100 +292,100 @@ TUPLE_ENTRY (RVVM1x2DF, TARGET_VECTOR_ELEN_FP_64, RVVM1DF, 2, LMUL_1, 16)
/* This following VLS modes should satisfy the constraint:
    GET_MODE_BITSIZE (MODE) <= TARGET_MIN_VLEN * 8.  */
-VLS_ENTRY (V1BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V2DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V4DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V8DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V2HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V4HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V8HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V16HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V32HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V2SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V4SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V8SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V16SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V2DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V4DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V8DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1BI, TARGET_VECTOR_VLS (V1BImode))
+VLS_ENTRY (V2BI, TARGET_VECTOR_VLS (V2BImode))
+VLS_ENTRY (V4BI, TARGET_VECTOR_VLS (V4BImode))
+VLS_ENTRY (V8BI, TARGET_VECTOR_VLS (V8BImode))
+VLS_ENTRY (V16BI, TARGET_VECTOR_VLS (V16BImode))
+VLS_ENTRY (V32BI, TARGET_VECTOR_VLS (V32BImode))
+VLS_ENTRY (V64BI, TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128BI, TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256BI, TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512BI, TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1QI, TARGET_VECTOR_VLS (V1QImode))
+VLS_ENTRY (V2QI, TARGET_VECTOR_VLS (V2QImode))
+VLS_ENTRY (V4QI, TARGET_VECTOR_VLS (V4QImode))
+VLS_ENTRY (V8QI, TARGET_VECTOR_VLS (V8QImode))
+VLS_ENTRY (V16QI, TARGET_VECTOR_VLS (V16QImode))
+VLS_ENTRY (V32QI, TARGET_VECTOR_VLS (V32QImode))
+VLS_ENTRY (V64QI, TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128QI, TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256QI, TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512QI, TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1HI, TARGET_VECTOR_VLS (V1HImode))
+VLS_ENTRY (V2HI, TARGET_VECTOR_VLS (V2HImode))
+VLS_ENTRY (V4HI, TARGET_VECTOR_VLS (V4HImode))
+VLS_ENTRY (V8HI, TARGET_VECTOR_VLS (V8HImode))
+VLS_ENTRY (V16HI, TARGET_VECTOR_VLS (V16HImode))
+VLS_ENTRY (V32HI, TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HI, TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HI, TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HI, TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HI, TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SI, TARGET_VECTOR_VLS (V1SImode))
+VLS_ENTRY (V2SI, TARGET_VECTOR_VLS (V2SImode))
+VLS_ENTRY (V4SI, TARGET_VECTOR_VLS (V4SImode))
+VLS_ENTRY (V8SI, TARGET_VECTOR_VLS (V8SImode))
+VLS_ENTRY (V16SI, TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SI, TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SI, TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SI, TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SI, TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SI, TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DI, TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V2DI, TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V4DI, TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V8DI, TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DI, TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DI, TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DI, TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DI, TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DI, TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DI, TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1HF, TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V2HF, TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V4HF, TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V8HF, TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V16HF, TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V32HF, TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HF, TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HF, TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HF, TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HF, TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SF, TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V2SF, TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V4SF, TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V8SF, TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V16SF, TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SF, TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SF, TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SF, TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SF, TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SF, TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DF, TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V2DF, TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V4DF, TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V8DF, TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DF, TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DF, TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DF, TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DF, TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DF, TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DF, TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
#undef VLS_ENTRY
#undef TUPLE_ENTRY
diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md
index 0850475edc1..2eac4355ee5 100644
--- a/gcc/config/riscv/vector-iterators.md
+++ b/gcc/config/riscv/vector-iterators.md
@@ -1,3 +1,4 @@
+
;; Iterators for RISC-V 'V' Extension for GNU compiler.
;; Copyright (C) 2022-2023 Free Software Foundation, Inc.
;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai<mailto:juzhe.zhong@rivai.ai>), RiVAI Technologies Ltd.
@@ -146,85 +147,85 @@ (define_mode_iterator V_VLS [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
   ;; VLS modes.
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VEEWEXT2 [
@@ -315,84 +316,84 @@ (define_mode_iterator VEI16 [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI [
@@ -416,52 +417,52 @@ (define_mode_iterator V_VLSI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator V_VLSF [
@@ -473,39 +474,39 @@ (define_mode_iterator V_VLSF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VF_ZVFHMIN [
@@ -531,39 +532,39 @@ (define_mode_iterator V_VLSF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
;; This iterator is the same as above but with TARGET_VECTOR_ELEN_FP_16
@@ -594,52 +595,52 @@ (define_mode_iterator VFULLI [
   (RVVM8DI "TARGET_FULL_V") (RVVM4DI "TARGET_FULL_V") (RVVM2DI "TARGET_FULL_V") (RVVM1DI "TARGET_FULL_V")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_FULL_V")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_FULL_V")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_FULL_V")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_QH [
@@ -655,42 +656,42 @@ (define_mode_iterator VI_QHS [
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_QHS_NO_M8 [
@@ -700,39 +701,39 @@ (define_mode_iterator VI_QHS_NO_M8 [
   RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
])
(define_mode_iterator VF_HS [
@@ -743,29 +744,29 @@ (define_mode_iterator VF_HS [
   (RVVM8SF "TARGET_VECTOR_ELEN_FP_32") (RVVM4SF "TARGET_VECTOR_ELEN_FP_32") (RVVM2SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32") (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VF_HS_NO_M8 [
@@ -779,27 +780,27 @@ (define_mode_iterator VF_HS_NO_M8 [
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
])
(define_mode_iterator VF_HS_M8 [
@@ -814,42 +815,42 @@ (define_mode_iterator V_VLSI_QHS [
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_D [
@@ -861,16 +862,16 @@ (define_mode_iterator V_VLSI_D [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VFULLI_D [
@@ -1013,19 +1014,19 @@ (define_mode_iterator VB [
(define_mode_iterator VB_VLS [
   (RVVMF64BI "TARGET_MIN_VLEN > 32") RVVMF32BI RVVMF16BI RVVMF8BI RVVMF4BI RVVMF2BI RVVM1BI
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWEXTI [
@@ -1036,39 +1037,39 @@ (define_mode_iterator VWEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
;; Same iterator split reason as VF_ZVFHMIN and VF.
@@ -1082,27 +1083,27 @@ (define_mode_iterator VWEXTF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWEXTF [
@@ -1115,27 +1116,27 @@ (define_mode_iterator VWEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWCONVERTI [
@@ -1147,27 +1148,27 @@ (define_mode_iterator VWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V1SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode) && TARGET_ZVFH")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode) && TARGET_ZVFH")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode) && TARGET_ZVFH")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode) && TARGET_ZVFH")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWWCONVERTI [
@@ -1176,16 +1177,16 @@ (define_mode_iterator VWWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VQEXTI [
@@ -1194,59 +1195,59 @@ (define_mode_iterator VQEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VQEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VOEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VT [
@@ -3216,28 +3217,28 @@ (define_mode_iterator V_VLS_FCONVERT_I_L_LL [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_attr VDEMOTE [
@@ -3748,281 +3749,281 @@ (define_code_attr sz [(sign_extend "s") (zero_extend "z")])
;; VLS modes.
(define_mode_iterator VLS [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSB [
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
;; VLS modes that has NUNITS < 32.
(define_mode_iterator VLS_AVL_IMM [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+
+  (V1BI "TARGET_VECTOR_VLS (V1BImode)")
+  (V2BI "TARGET_VECTOR_VLS (V2BImode)")
+  (V4BI "TARGET_VECTOR_VLS (V4BImode)")
+  (V8BI "TARGET_VECTOR_VLS (V8BImode)")
+  (V16BI "TARGET_VECTOR_VLS (V16BImode)")])
;; VLS modes that has NUNITS >= 32.
(define_mode_iterator VLS_AVL_REG [
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
-
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+
+  (V32BI "TARGET_VECTOR_VLS (V32BImode)")
+  (V64BI "TARGET_VECTOR_VLS (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "TARGET_VECTOR_VLS (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "TARGET_VECTOR_VLS (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "TARGET_VECTOR_VLS (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "TARGET_VECTOR_VLS (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "TARGET_VECTOR_VLS (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "TARGET_VECTOR_VLS (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSI [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "TARGET_VECTOR_VLS (V1QImode)")
+  (V2QI "TARGET_VECTOR_VLS (V2QImode)")
+  (V4QI "TARGET_VECTOR_VLS (V4QImode)")
+  (V8QI "TARGET_VECTOR_VLS (V8QImode)")
+  (V16QI "TARGET_VECTOR_VLS (V16QImode)")
+  (V32QI "TARGET_VECTOR_VLS (V32QImode)")
+  (V64QI "TARGET_VECTOR_VLS (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "TARGET_VECTOR_VLS (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "TARGET_VECTOR_VLS (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "TARGET_VECTOR_VLS (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "TARGET_VECTOR_VLS (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "TARGET_VECTOR_VLS (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "TARGET_VECTOR_VLS (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "TARGET_VECTOR_VLS (V1HImode)")
+  (V2HI "TARGET_VECTOR_VLS (V2HImode)")
+  (V4HI "TARGET_VECTOR_VLS (V4HImode)")
+  (V8HI "TARGET_VECTOR_VLS (V8HImode)")
+  (V16HI "TARGET_VECTOR_VLS (V16HImode)")
+  (V32HI "TARGET_VECTOR_VLS (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "TARGET_VECTOR_VLS (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "TARGET_VECTOR_VLS (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "TARGET_VECTOR_VLS (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "TARGET_VECTOR_VLS (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "TARGET_VECTOR_VLS (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "TARGET_VECTOR_VLS (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "TARGET_VECTOR_VLS (V1SImode)")
+  (V2SI "TARGET_VECTOR_VLS (V2SImode)")
+  (V4SI "TARGET_VECTOR_VLS (V4SImode)")
+  (V8SI "TARGET_VECTOR_VLS (V8SImode)")
+  (V16SI "TARGET_VECTOR_VLS (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "TARGET_VECTOR_VLS (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "TARGET_VECTOR_VLS (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "TARGET_VECTOR_VLS (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "TARGET_VECTOR_VLS (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "TARGET_VECTOR_VLS (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "TARGET_VECTOR_VLS (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "TARGET_VECTOR_VLS (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "TARGET_VECTOR_VLS (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "TARGET_VECTOR_VLS (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "TARGET_VECTOR_VLS (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "TARGET_VECTOR_VLS (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "TARGET_VECTOR_VLS (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "TARGET_VECTOR_VLS (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "TARGET_VECTOR_VLS (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "TARGET_VECTOR_VLS (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "TARGET_VECTOR_VLS (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSF [
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "TARGET_VECTOR_VLS (V1HFmode) && TARGET_ZVFH")
+  (V2HF "TARGET_VECTOR_VLS (V2HFmode) && TARGET_ZVFH")
+  (V4HF "TARGET_VECTOR_VLS (V4HFmode) && TARGET_ZVFH")
+  (V8HF "TARGET_VECTOR_VLS (V8HFmode) && TARGET_ZVFH")
+  (V16HF "TARGET_VECTOR_VLS (V16HFmode) && TARGET_ZVFH")
+  (V32HF "TARGET_VECTOR_VLS (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "TARGET_VECTOR_VLS (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "TARGET_VECTOR_VLS (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "TARGET_VECTOR_VLS (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "TARGET_VECTOR_VLS (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "TARGET_VECTOR_VLS (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "TARGET_VECTOR_VLS (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "TARGET_VECTOR_VLS (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "TARGET_VECTOR_VLS (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "TARGET_VECTOR_VLS (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "TARGET_VECTOR_VLS (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "TARGET_VECTOR_VLS (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "TARGET_VECTOR_VLS (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "TARGET_VECTOR_VLS (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "TARGET_VECTOR_VLS (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "TARGET_VECTOR_VLS (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "TARGET_VECTOR_VLS (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "TARGET_VECTOR_VLS (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "TARGET_VECTOR_VLS (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "TARGET_VECTOR_VLS (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "TARGET_VECTOR_VLS (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "TARGET_VECTOR_VLS (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "TARGET_VECTOR_VLS (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "TARGET_VECTOR_VLS (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "TARGET_VECTOR_VLS (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "TARGET_VECTOR_VLS (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "TARGET_VECTOR_VLS (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "TARGET_VECTOR_VLS (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
index 7ab31043547..b235ec727b1 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
@@ -55,7 +55,7 @@
TEST_ALL (PERMUTE)
-/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 19 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 18 } } */
/* { dg-final { scan-assembler-times {vrgatherei16\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 12 } } */
-/* { dg-final { scan-assembler-times {vrsub\.vi} 24 } } */
+/* { dg-final { scan-assembler-times {vrsub\.vi} 23 } } */
/* { dg-final { scan-assembler-times {vrsub\.vx} 7 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
new file mode 100644
index 00000000000..57af7db5cb1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c
@@ -0,0 +1,536 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1qi,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2qi,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4qi,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8qi,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v4096qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 10 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
new file mode 100644
index 00000000000..2619658fc9b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hi,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hi,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hi,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hi,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
new file mode 100644
index 00000000000..16aadba1276
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1si,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2si,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4si,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8si,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
new file mode 100644
index 00000000000..5c5e93aef8c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1di,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2di,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4di,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8di,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
new file mode 100644
index 00000000000..6aa4a7e615c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hf,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hf,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hf,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hf,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*[v]?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
new file mode 100644
index 00000000000..a31dad5a926
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1sf,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2sf,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4sf,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8sf,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
new file mode 100644
index 00000000000..8312b21c497
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1df,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2df,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4df,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8df,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
index 8dd5bcf617d..8ee8b7b1801 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
@@ -441,6 +441,12 @@ typedef double v512df __attribute__ ((vector_size (4096)));
     *(TYPE1 *) out = v;                                                        \
   }
+#define DEF_OP_VV_VA(OP, TYPE1, ...)                                           \
+  TYPE1 test_##OP##_##TYPE1 (TYPE1 a, TYPE1 b)                                 \
+  {                                                                            \
+    return OP (a, b, __VA_ARGS__);                                             \
+  }
+
#define DEF_REPEAT(TYPE1, TYPE2, NUM, ...)                                     \
   void init_##TYPE1##_##TYPE2##_##NUM (TYPE2 var0, TYPE2 var1,                 \
       TYPE2 *__restrict out)                  \
--
2.34.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857]
  2023-10-20  8:34 [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax pan2.li
  2023-10-20  8:47 ` juzhe.zhong
@ 2023-10-20 12:59 ` pan2.li
  2023-10-21  0:38   ` 钟居哲
  1 sibling, 1 reply; 6+ messages in thread
From: pan2.li @ 2023-10-20 12:59 UTC (permalink / raw)
  To: gcc-patches; +Cc: juzhe.zhong, pan2.li, yanzhang.wang, kito.cheng

From: Pan Li <pan2.li@intel.com>

Update in v2:
* Remove macro and define one gloab func for target_vls.
* Rename testcases to PR111857.
* Minor change to commit log.

Original log:

Given we have code like below:

typedef char vnx16i __attribute__ ((vector_size (16)));

vnx16i __attribute__ ((noinline, noclone))
test (vnx16i x, vnx16i y)
{
  return __builtin_shufflevector (x, y, 11, 12, 13, 14, 11, 12, 13, 14,
                                        11, 12, 13, 14, 11, 12, 13, 14);
}

It can perform the auto vectorization when

-march=rv64gcv_zvl1024b --param=riscv-autovec-preference=fixed-vlmax

but cannot when

-march=rv64gcv_zvl2048b --param=riscv-autovec-preference=fixed-vlmax

The reason comes from the miniaml machine mode of QI is RVVMF8QI, which
is 1024 / 8 = 128 bits, aka the size of VNx16QI. When we set zvl2048b,
the bit size of RVVMFQI is 2048 / 8 = 256, which is not matching the
bit size of VNx16QI (128 bits).

Thus, this patch would like to enable the VLS mode for such case, aka
VNx16QI vls mode for zvl2048b.

Before this patch:
test:
  srli    a4,a1,40
  andi    a4,a4,0xff
  srli    a3,a1,32
  srli    a5,a1,48
  slli    a0,a4,8
  andi    a3,a3,0xff
  andi    a5,a5,0xff
  slli    a2,a5,16
  or      a0,a3,a0
  srli    a1,a1,56
  or      a0,a0,a2
  slli    a2,a1,24
  slli    a3,a3,32
  or      a0,a0,a2
  slli    a4,a4,40
  or      a0,a0,a3
  slli    a5,a5,48
  or      a0,a0,a4
  or      a0,a0,a5
  slli    a1,a1,56
  or      a0,a0,a1
  mv      a1,a0
  ret

After this patch:
test:
  vsetivli        zero,16,e8,mf8,ta,ma
  vle8.v  v2,0(a1)
  vsetivli        zero,4,e32,mf2,ta,ma
  vrgather.vi     v1,v2,3
  vsetivli        zero,16,e8,mf8,ta,ma
  vse8.v  v1,0(a0)
  ret

	PR target/111857

gcc/ChangeLog:

	* config/riscv/riscv-opts.h (TARGET_VECTOR_VLS): Remove.
	* config/riscv/riscv-protos.h (vls_mode_valid_p): New func decl.
	* config/riscv/riscv-v.cc (autovectorize_vector_modes): Replace
	macro reference to func.
	(vls_mode_valid_p): New func impl for vls mode valid or not.
	* config/riscv/riscv-vector-switch.def (VLS_ENTRY): Replace
	macro reference to func.
	* config/riscv/vector-iterators.md: Ditto.

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c: Adjust checker.
	* gcc.target/riscv/rvv/autovec/vls/def.h: Add help define.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c: New test.
	* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c: New test.

	PR 111857

Signed-off-by: Pan Li <pan2.li@intel.com>
---
 gcc/config/riscv/riscv-opts.h                 |    5 -
 gcc/config/riscv/riscv-protos.h               |    1 +
 gcc/config/riscv/riscv-v.cc                   |  114 +-
 gcc/config/riscv/riscv-vector-switch.def      |  188 +-
 gcc/config/riscv/vector-iterators.md          | 1843 +++++++++--------
 .../riscv/rvv/autovec/vls-vlmax/perm-4.c      |    4 +-
 .../riscv/rvv/autovec/vls-vlmax/pr111857-0.c  |  536 +++++
 .../riscv/rvv/autovec/vls-vlmax/pr111857-1.c  |  279 +++
 .../riscv/rvv/autovec/vls-vlmax/pr111857-2.c  |  151 ++
 .../riscv/rvv/autovec/vls-vlmax/pr111857-3.c  |   86 +
 .../riscv/rvv/autovec/vls-vlmax/pr111857-4.c  |  279 +++
 .../riscv/rvv/autovec/vls-vlmax/pr111857-5.c  |  151 ++
 .../riscv/rvv/autovec/vls-vlmax/pr111857-6.c  |   86 +
 .../gcc.target/riscv/rvv/autovec/vls/def.h    |    6 +
 14 files changed, 2694 insertions(+), 1035 deletions(-)
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c
 create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c

diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
index 31ee42dea6b..e557f70f414 100644
--- a/gcc/config/riscv/riscv-opts.h
+++ b/gcc/config/riscv/riscv-opts.h
@@ -118,11 +118,6 @@ enum riscv_entity
      ? 0                                                                       \
      : 32 << (__builtin_popcount (opts->x_riscv_zvl_flags) - 1))
 
-/* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode
-   is the highest priority choice and should not conflict with VLS modes.  */
-#define TARGET_VECTOR_VLS                                                      \
-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)
-
 /* TODO: Enable RVV movmisalign by default for now.  */
 #define TARGET_VECTOR_MISALIGN_SUPPORTED 1
 
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 6190faab501..f7a9a02f1f9 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -552,6 +552,7 @@ unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
 bool cmp_lmul_le_one (machine_mode);
 bool cmp_lmul_gt_one (machine_mode);
 bool gather_scatter_valid_offset_mode_p (machine_mode);
+bool vls_mode_valid_p (machine_mode);
 }
 
 /* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 895c11d13fc..cbd36acb2ed 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -2423,20 +2423,19 @@ autovectorize_vector_modes (vector_modes *modes, bool)
 	    modes->safe_push (mode);
 	}
     }
-  if (TARGET_VECTOR_VLS)
-    {
-      /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
-      unsigned int i = 0;
-      unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
-      unsigned int size = base_size;
-      machine_mode mode;
-      while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
-	{
+    /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
+    unsigned int i = 0;
+    unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
+    unsigned int size = base_size;
+    machine_mode mode;
+    while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
+     {
+	if (vls_mode_valid_p (mode))
 	  modes->safe_push (mode);
-	  i++;
-	  size = base_size / (1U << i);
-	}
-    }
+
+	i++;
+	size = base_size / (1U << i);
+     }
   /* Enable LOOP_VINFO comparison in COST model.  */
   return VECT_COMPARE_COSTS;
 }
@@ -3880,6 +3879,95 @@ cmp_lmul_gt_one (machine_mode mode)
   return false;
 }
 
+/* Return true if the VLS mode is legal. There are 2 cases here.
+
+   1. Enable VLS modes for VLA vectorization since fixed length VLMAX mode
+      is the highest priority choice and should not conflict with VLS modes.
+   2. Enable VLS modes for some cases in fixed-vlmax, aka the bitsize of the
+      VLS mode are smaller than the minimal vla.
+
+   Take vlen = 2048 as example for case 2.
+
+   Note: Below table based on vlen = 2048.
+   +----------------------------------------------------+----------------------+
+   | VLS mode                                           | VLA mode             |
+   +----------------------------------------------------+----------------------+
+   | Name       | Precision | Inner Precision | Enabled | Min mode  | Min bits |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1BI       |     1     |              1  | Yes     | RVVMF64BI |    32    |
+   | V2BI       |     2     |              1  | Yes     | RVVMF64BI |    32    |
+   | V4BI       |     4     |              1  | Yes     | RVVMF64BI |    32    |
+   | V8BI       |     8     |              1  | Yes     | RVVMF64BI |    32    |
+   | V16BI      |    16     |              1  | Yes     | RVVMF64BI |    32    |
+   | V32BI      |    32     |              1  | NO      | RVVMF64BI |    32    |
+   | V64BI      |    64     |              1  | NO      | RVVMF64BI |    32    |
+   | ...        |   ...     |            ...  | ...     | RVVMF64BI |    32    |
+   | V4096BI    |  4096     |              1  | NO      | RVVMF64BI |    32    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1QI       |     8     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V2QI       |    16     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V4QI       |    32     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V8QI       |    64     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V16QI      |   128     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V32QI      |   256     |              8  | NO      | RVVMF8QI  |   256    |
+   | V64QI      |   512     |              8  | NO      | RVVMF8QI  |   256    |
+   | ...        |   ...     |              .. | ...     | RVVMF8QI  |   256    |
+   | V4096QI    | 32768     |              8  | NO      | RVVMF8QI  |   256    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1HI       |    16     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V2HI       |    32     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V4HI       |    64     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V8HI       |   128     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V16HI      |   256     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V32HI      |   512     |              16 | NO      | RVVMF4HI  |   512    |
+   | V64HI      |  1024     |              16 | NO      | RVVMF4HI  |   512    |
+   | ...        |   ...     |              .. | ...     | RVVMF4HI  |   512    |
+   | V2048HI    | 32768     |              16 | NO      | RVVMF4HI  |   512    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1SI/SF    |    32     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V2SI/SF    |    64     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V4SI/SF    |   128     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V8SI/SF    |   256     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V16SI/SF   |   512     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V32SI/SF   |  1024     |              32 | NO      | RVVMF2SI  |  1024    |
+   | V64SI/SF   |  2048     |              32 | NO      | RVVMF2SI  |  1024    |
+   | ...        |   ...     |              .. | ...     | RVVMF2SI  |  1024    |
+   | V1024SI/SF | 32768     |              32 | NO      | RVVMF2SI  |  1024    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1DI/DF    |    64     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V2DI/DF    |   128     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V4DI/DF    |   256     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V8DI/DF    |   512     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V16DI/DF   |  1024     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V32DI/DF   |  2048     |              64 | NO      | RVVM1DI   |  2048    |
+   | V64DI/DF   |  4096     |              64 | NO      | RVVM1DI   |  2048    |
+   | ...        |   ...     |              .. | ...     | RVVM1DI   |  2048    |
+   | V512DI/DF  | 32768     |              64 | NO      | RVVM1DI   |  2048    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+
+   Then we can have the condition for VLS mode in fixed-vlmax, aka:
+     PRECISION (VLSmode) < VLEN / (64 / PRECISION(VLS_inner_mode)).  */
+bool
+vls_mode_valid_p (machine_mode vls_mode)
+{
+  if (!TARGET_VECTOR)
+    return false;
+
+  if (riscv_autovec_preference == RVV_SCALABLE)
+    return true;
+
+  if (riscv_autovec_preference == RVV_FIXED_VLMAX)
+    {
+      machine_mode inner_mode = GET_MODE_INNER (vls_mode);
+      int precision = GET_MODE_PRECISION (inner_mode).to_constant ();
+      int min_vlmax_bitsize = TARGET_MIN_VLEN / (64 / precision);
+
+      return GET_MODE_PRECISION (vls_mode).to_constant () < min_vlmax_bitsize;
+    }
+
+  return false;
+}
+
 /* Return true if the gather/scatter offset mode is valid.  */
 bool
 gather_scatter_valid_offset_mode_p (machine_mode mode)
diff --git a/gcc/config/riscv/riscv-vector-switch.def b/gcc/config/riscv/riscv-vector-switch.def
index 174e5a181c6..5c9f9bcbc3e 100644
--- a/gcc/config/riscv/riscv-vector-switch.def
+++ b/gcc/config/riscv/riscv-vector-switch.def
@@ -292,100 +292,100 @@ TUPLE_ENTRY (RVVM1x2DF, TARGET_VECTOR_ELEN_FP_64, RVVM1DF, 2, LMUL_1, 16)
 
 /* This following VLS modes should satisfy the constraint:
    GET_MODE_BITSIZE (MODE) <= TARGET_MIN_VLEN * 8.  */
-VLS_ENTRY (V1BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V2DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V4DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V8DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V2HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V4HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V8HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V16HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V32HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V2SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V4SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V8SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V16SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V2DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V4DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V8DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1BI, riscv_vector::vls_mode_valid_p (V1BImode))
+VLS_ENTRY (V2BI, riscv_vector::vls_mode_valid_p (V2BImode))
+VLS_ENTRY (V4BI, riscv_vector::vls_mode_valid_p (V4BImode))
+VLS_ENTRY (V8BI, riscv_vector::vls_mode_valid_p (V8BImode))
+VLS_ENTRY (V16BI, riscv_vector::vls_mode_valid_p (V16BImode))
+VLS_ENTRY (V32BI, riscv_vector::vls_mode_valid_p (V32BImode))
+VLS_ENTRY (V64BI, riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128BI, riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256BI, riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512BI, riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024BI, riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048BI, riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096BI, riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1QI, riscv_vector::vls_mode_valid_p (V1QImode))
+VLS_ENTRY (V2QI, riscv_vector::vls_mode_valid_p (V2QImode))
+VLS_ENTRY (V4QI, riscv_vector::vls_mode_valid_p (V4QImode))
+VLS_ENTRY (V8QI, riscv_vector::vls_mode_valid_p (V8QImode))
+VLS_ENTRY (V16QI, riscv_vector::vls_mode_valid_p (V16QImode))
+VLS_ENTRY (V32QI, riscv_vector::vls_mode_valid_p (V32QImode))
+VLS_ENTRY (V64QI, riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128QI, riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256QI, riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512QI, riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024QI, riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048QI, riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096QI, riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1HI, riscv_vector::vls_mode_valid_p (V1HImode))
+VLS_ENTRY (V2HI, riscv_vector::vls_mode_valid_p (V2HImode))
+VLS_ENTRY (V4HI, riscv_vector::vls_mode_valid_p (V4HImode))
+VLS_ENTRY (V8HI, riscv_vector::vls_mode_valid_p (V8HImode))
+VLS_ENTRY (V16HI, riscv_vector::vls_mode_valid_p (V16HImode))
+VLS_ENTRY (V32HI, riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HI, riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HI, riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HI, riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HI, riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HI, riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HI, riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SI, riscv_vector::vls_mode_valid_p (V1SImode))
+VLS_ENTRY (V2SI, riscv_vector::vls_mode_valid_p (V2SImode))
+VLS_ENTRY (V4SI, riscv_vector::vls_mode_valid_p (V4SImode))
+VLS_ENTRY (V8SI, riscv_vector::vls_mode_valid_p (V8SImode))
+VLS_ENTRY (V16SI, riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SI, riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SI, riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SI, riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SI, riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SI, riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SI, riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DI, riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V2DI, riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V4DI, riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V8DI, riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DI, riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DI, riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DI, riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DI, riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DI, riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DI, riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1HF, riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V2HF, riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V4HF, riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V8HF, riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V16HF, riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V32HF, riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HF, riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HF, riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HF, riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HF, riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HF, riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HF, riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SF, riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V2SF, riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V4SF, riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V8SF, riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V16SF, riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SF, riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SF, riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SF, riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SF, riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SF, riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SF, riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DF, riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V2DF, riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V4DF, riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V8DF, riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DF, riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DF, riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DF, riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DF, riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DF, riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DF, riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
 
 #undef VLS_ENTRY
 #undef TUPLE_ENTRY
diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md
index 0850475edc1..e80a20dcfba 100644
--- a/gcc/config/riscv/vector-iterators.md
+++ b/gcc/config/riscv/vector-iterators.md
@@ -1,3 +1,4 @@
+
 ;; Iterators for RISC-V 'V' Extension for GNU compiler.
 ;; Copyright (C) 2022-2023 Free Software Foundation, Inc.
 ;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies Ltd.
@@ -146,85 +147,85 @@ (define_mode_iterator V_VLS [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
   ;; VLS modes.
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VEEWEXT2 [
@@ -315,84 +316,84 @@ (define_mode_iterator VEI16 [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI [
@@ -416,52 +417,52 @@ (define_mode_iterator V_VLSI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator V_VLSF [
@@ -473,39 +474,39 @@ (define_mode_iterator V_VLSF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VF_ZVFHMIN [
@@ -531,39 +532,39 @@ (define_mode_iterator V_VLSF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 ;; This iterator is the same as above but with TARGET_VECTOR_ELEN_FP_16
@@ -594,52 +595,52 @@ (define_mode_iterator VFULLI [
 
   (RVVM8DI "TARGET_FULL_V") (RVVM4DI "TARGET_FULL_V") (RVVM2DI "TARGET_FULL_V") (RVVM1DI "TARGET_FULL_V")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_FULL_V")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_FULL_V")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_FULL_V")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI_QH [
@@ -655,42 +656,42 @@ (define_mode_iterator VI_QHS [
 
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI_QHS_NO_M8 [
@@ -700,39 +701,39 @@ (define_mode_iterator VI_QHS_NO_M8 [
 
   RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
 ])
 
 (define_mode_iterator VF_HS [
@@ -743,29 +744,29 @@ (define_mode_iterator VF_HS [
   (RVVM8SF "TARGET_VECTOR_ELEN_FP_32") (RVVM4SF "TARGET_VECTOR_ELEN_FP_32") (RVVM2SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32") (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VF_HS_NO_M8 [
@@ -779,27 +780,27 @@ (define_mode_iterator VF_HS_NO_M8 [
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
 
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
 ])
 
 (define_mode_iterator VF_HS_M8 [
@@ -814,42 +815,42 @@ (define_mode_iterator V_VLSI_QHS [
 
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
 
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VI_D [
@@ -861,16 +862,16 @@ (define_mode_iterator V_VLSI_D [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VFULLI_D [
@@ -1013,19 +1014,19 @@ (define_mode_iterator VB [
 
 (define_mode_iterator VB_VLS [
   (RVVMF64BI "TARGET_MIN_VLEN > 32") RVVMF32BI RVVMF16BI RVVMF8BI RVVMF4BI RVVMF2BI RVVM1BI
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1BI "riscv_vector::vls_mode_valid_p (V1BImode)")
+  (V2BI "riscv_vector::vls_mode_valid_p (V2BImode)")
+  (V4BI "riscv_vector::vls_mode_valid_p (V4BImode)")
+  (V8BI "riscv_vector::vls_mode_valid_p (V8BImode)")
+  (V16BI "riscv_vector::vls_mode_valid_p (V16BImode)")
+  (V32BI "riscv_vector::vls_mode_valid_p (V32BImode)")
+  (V64BI "riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWEXTI [
@@ -1036,39 +1037,39 @@ (define_mode_iterator VWEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 ;; Same iterator split reason as VF_ZVFHMIN and VF.
@@ -1082,27 +1083,27 @@ (define_mode_iterator VWEXTF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWEXTF [
@@ -1115,27 +1116,27 @@ (define_mode_iterator VWEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWCONVERTI [
@@ -1147,27 +1148,27 @@ (define_mode_iterator VWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
 
-  (V1SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode) && TARGET_ZVFH")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode) && TARGET_ZVFH")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode) && TARGET_ZVFH")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode) && TARGET_ZVFH")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VWWCONVERTI [
@@ -1176,16 +1177,16 @@ (define_mode_iterator VWWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
 
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VQEXTI [
@@ -1194,59 +1195,59 @@ (define_mode_iterator VQEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VQEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VOEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
 
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_iterator VT [
@@ -3216,28 +3217,28 @@ (define_mode_iterator V_VLS_FCONVERT_I_L_LL [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
 
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
 
 (define_mode_attr VDEMOTE [
@@ -3748,281 +3749,281 @@ (define_code_attr sz [(sign_extend "s") (zero_extend "z")])
 
 ;; VLS modes.
 (define_mode_iterator VLS [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
 
 (define_mode_iterator VLSB [
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V1BI "riscv_vector::vls_mode_valid_p (V1BImode)")
+  (V2BI "riscv_vector::vls_mode_valid_p (V2BImode)")
+  (V4BI "riscv_vector::vls_mode_valid_p (V4BImode)")
+  (V8BI "riscv_vector::vls_mode_valid_p (V8BImode)")
+  (V16BI "riscv_vector::vls_mode_valid_p (V16BImode)")
+  (V32BI "riscv_vector::vls_mode_valid_p (V32BImode)")
+  (V64BI "riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
 
 ;; VLS modes that has NUNITS < 32.
 (define_mode_iterator VLS_AVL_IMM [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")])
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+
+  (V1BI "riscv_vector::vls_mode_valid_p (V1BImode)")
+  (V2BI "riscv_vector::vls_mode_valid_p (V2BImode)")
+  (V4BI "riscv_vector::vls_mode_valid_p (V4BImode)")
+  (V8BI "riscv_vector::vls_mode_valid_p (V8BImode)")
+  (V16BI "riscv_vector::vls_mode_valid_p (V16BImode)")])
 
 ;; VLS modes that has NUNITS >= 32.
 (define_mode_iterator VLS_AVL_REG [
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
-
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+
+  (V32BI "riscv_vector::vls_mode_valid_p (V32BImode)")
+  (V64BI "riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
 
 (define_mode_iterator VLSI [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
 
 (define_mode_iterator VLSF [
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
 ])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
index 7ab31043547..b235ec727b1 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
@@ -55,7 +55,7 @@
 
 TEST_ALL (PERMUTE)
 
-/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 19 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 18 } } */
 /* { dg-final { scan-assembler-times {vrgatherei16\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 12 } } */
-/* { dg-final { scan-assembler-times {vrsub\.vi} 24 } } */
+/* { dg-final { scan-assembler-times {vrsub\.vi} 23 } } */
 /* { dg-final { scan-assembler-times {vrsub\.vx} 7 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c
new file mode 100644
index 00000000000..57af7db5cb1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c
@@ -0,0 +1,536 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1qi,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2qi,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4qi,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8qi,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v4096qi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 10 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c
new file mode 100644
index 00000000000..2619658fc9b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hi,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hi,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hi,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hi,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hi,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c
new file mode 100644
index 00000000000..16aadba1276
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1si,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2si,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4si,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8si,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024si,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c
new file mode 100644
index 00000000000..5c5e93aef8c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1di,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2di,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4di,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8di,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512di,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c
new file mode 100644
index 00000000000..6aa4a7e615c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hf,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hf,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hf,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hf,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*[v]?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c
new file mode 100644
index 00000000000..a31dad5a926
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1sf,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2sf,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4sf,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8sf,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024sf,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c
new file mode 100644
index 00000000000..8312b21c497
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1df,
+	      0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2df,
+	      0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4df,
+	      0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8df,
+	      0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512df,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+	      12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
index 8dd5bcf617d..8ee8b7b1801 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
@@ -441,6 +441,12 @@ typedef double v512df __attribute__ ((vector_size (4096)));
     *(TYPE1 *) out = v;                                                        \
   }
 
+#define DEF_OP_VV_VA(OP, TYPE1, ...)                                           \
+  TYPE1 test_##OP##_##TYPE1 (TYPE1 a, TYPE1 b)                                 \
+  {                                                                            \
+    return OP (a, b, __VA_ARGS__);                                             \
+  }
+
 #define DEF_REPEAT(TYPE1, TYPE2, NUM, ...)                                     \
   void init_##TYPE1##_##TYPE2##_##NUM (TYPE2 var0, TYPE2 var1,                 \
 				       TYPE2 *__restrict out)                  \
-- 
2.34.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857]
  2023-10-20 12:59 ` [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857] pan2.li
@ 2023-10-21  0:38   ` 钟居哲
  2023-10-21  1:27     ` Li, Pan2
  0 siblings, 1 reply; 6+ messages in thread
From: 钟居哲 @ 2023-10-21  0:38 UTC (permalink / raw)
  To: pan2.li, gcc-patches; +Cc: pan2.li, yanzhang.wang, kito.cheng

[-- Attachment #1: Type: text/plain, Size: 286336 bytes --]

LGTM.



juzhe.zhong@rivai.ai
 
From: pan2.li
Date: 2023-10-20 20:59
To: gcc-patches
CC: juzhe.zhong; pan2.li; yanzhang.wang; kito.cheng
Subject: [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857]
From: Pan Li <pan2.li@intel.com>
 
Update in v2:
* Remove macro and define one gloab func for target_vls.
* Rename testcases to PR111857.
* Minor change to commit log.
 
Original log:
 
Given we have code like below:
 
typedef char vnx16i __attribute__ ((vector_size (16)));
 
vnx16i __attribute__ ((noinline, noclone))
test (vnx16i x, vnx16i y)
{
  return __builtin_shufflevector (x, y, 11, 12, 13, 14, 11, 12, 13, 14,
                                        11, 12, 13, 14, 11, 12, 13, 14);
}
 
It can perform the auto vectorization when
 
-march=rv64gcv_zvl1024b --param=riscv-autovec-preference=fixed-vlmax
 
but cannot when
 
-march=rv64gcv_zvl2048b --param=riscv-autovec-preference=fixed-vlmax
 
The reason comes from the miniaml machine mode of QI is RVVMF8QI, which
is 1024 / 8 = 128 bits, aka the size of VNx16QI. When we set zvl2048b,
the bit size of RVVMFQI is 2048 / 8 = 256, which is not matching the
bit size of VNx16QI (128 bits).
 
Thus, this patch would like to enable the VLS mode for such case, aka
VNx16QI vls mode for zvl2048b.
 
Before this patch:
test:
  srli    a4,a1,40
  andi    a4,a4,0xff
  srli    a3,a1,32
  srli    a5,a1,48
  slli    a0,a4,8
  andi    a3,a3,0xff
  andi    a5,a5,0xff
  slli    a2,a5,16
  or      a0,a3,a0
  srli    a1,a1,56
  or      a0,a0,a2
  slli    a2,a1,24
  slli    a3,a3,32
  or      a0,a0,a2
  slli    a4,a4,40
  or      a0,a0,a3
  slli    a5,a5,48
  or      a0,a0,a4
  or      a0,a0,a5
  slli    a1,a1,56
  or      a0,a0,a1
  mv      a1,a0
  ret
 
After this patch:
test:
  vsetivli        zero,16,e8,mf8,ta,ma
  vle8.v  v2,0(a1)
  vsetivli        zero,4,e32,mf2,ta,ma
  vrgather.vi     v1,v2,3
  vsetivli        zero,16,e8,mf8,ta,ma
  vse8.v  v1,0(a0)
  ret
 
PR target/111857
 
gcc/ChangeLog:
 
* config/riscv/riscv-opts.h (TARGET_VECTOR_VLS): Remove.
* config/riscv/riscv-protos.h (vls_mode_valid_p): New func decl.
* config/riscv/riscv-v.cc (autovectorize_vector_modes): Replace
macro reference to func.
(vls_mode_valid_p): New func impl for vls mode valid or not.
* config/riscv/riscv-vector-switch.def (VLS_ENTRY): Replace
macro reference to func.
* config/riscv/vector-iterators.md: Ditto.
 
gcc/testsuite/ChangeLog:
 
* gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c: Adjust checker.
* gcc.target/riscv/rvv/autovec/vls/def.h: Add help define.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-0.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-1.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-2.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-3.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-4.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-5.c: New test.
* gcc.target/riscv/rvv/autovec/vls-vlmax/vls-mode-6.c: New test.
 
PR 111857
 
Signed-off-by: Pan Li <pan2.li@intel.com>
---
gcc/config/riscv/riscv-opts.h                 |    5 -
gcc/config/riscv/riscv-protos.h               |    1 +
gcc/config/riscv/riscv-v.cc                   |  114 +-
gcc/config/riscv/riscv-vector-switch.def      |  188 +-
gcc/config/riscv/vector-iterators.md          | 1843 +++++++++--------
.../riscv/rvv/autovec/vls-vlmax/perm-4.c      |    4 +-
.../riscv/rvv/autovec/vls-vlmax/pr111857-0.c  |  536 +++++
.../riscv/rvv/autovec/vls-vlmax/pr111857-1.c  |  279 +++
.../riscv/rvv/autovec/vls-vlmax/pr111857-2.c  |  151 ++
.../riscv/rvv/autovec/vls-vlmax/pr111857-3.c  |   86 +
.../riscv/rvv/autovec/vls-vlmax/pr111857-4.c  |  279 +++
.../riscv/rvv/autovec/vls-vlmax/pr111857-5.c  |  151 ++
.../riscv/rvv/autovec/vls-vlmax/pr111857-6.c  |   86 +
.../gcc.target/riscv/rvv/autovec/vls/def.h    |    6 +
14 files changed, 2694 insertions(+), 1035 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c
create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c
 
diff --git a/gcc/config/riscv/riscv-opts.h b/gcc/config/riscv/riscv-opts.h
index 31ee42dea6b..e557f70f414 100644
--- a/gcc/config/riscv/riscv-opts.h
+++ b/gcc/config/riscv/riscv-opts.h
@@ -118,11 +118,6 @@ enum riscv_entity
      ? 0                                                                       \
      : 32 << (__builtin_popcount (opts->x_riscv_zvl_flags) - 1))
-/* We only enable VLS modes for VLA vectorization since fixed length VLMAX mode
-   is the highest priority choice and should not conflict with VLS modes.  */
-#define TARGET_VECTOR_VLS                                                      \
-  (TARGET_VECTOR && riscv_autovec_preference == RVV_SCALABLE)
-
/* TODO: Enable RVV movmisalign by default for now.  */
#define TARGET_VECTOR_MISALIGN_SUPPORTED 1
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index 6190faab501..f7a9a02f1f9 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -552,6 +552,7 @@ unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
bool cmp_lmul_le_one (machine_mode);
bool cmp_lmul_gt_one (machine_mode);
bool gather_scatter_valid_offset_mode_p (machine_mode);
+bool vls_mode_valid_p (machine_mode);
}
/* We classify builtin types into two classes:
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 895c11d13fc..cbd36acb2ed 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -2423,20 +2423,19 @@ autovectorize_vector_modes (vector_modes *modes, bool)
    modes->safe_push (mode);
}
     }
-  if (TARGET_VECTOR_VLS)
-    {
-      /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
-      unsigned int i = 0;
-      unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
-      unsigned int size = base_size;
-      machine_mode mode;
-      while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
- {
+    /* Push all VLSmodes according to TARGET_MIN_VLEN.  */
+    unsigned int i = 0;
+    unsigned int base_size = TARGET_MIN_VLEN * lmul / 8;
+    unsigned int size = base_size;
+    machine_mode mode;
+    while (size > 0 && get_vector_mode (QImode, size).exists (&mode))
+     {
+ if (vls_mode_valid_p (mode))
  modes->safe_push (mode);
-   i++;
-   size = base_size / (1U << i);
- }
-    }
+
+ i++;
+ size = base_size / (1U << i);
+     }
   /* Enable LOOP_VINFO comparison in COST model.  */
   return VECT_COMPARE_COSTS;
}
@@ -3880,6 +3879,95 @@ cmp_lmul_gt_one (machine_mode mode)
   return false;
}
+/* Return true if the VLS mode is legal. There are 2 cases here.
+
+   1. Enable VLS modes for VLA vectorization since fixed length VLMAX mode
+      is the highest priority choice and should not conflict with VLS modes.
+   2. Enable VLS modes for some cases in fixed-vlmax, aka the bitsize of the
+      VLS mode are smaller than the minimal vla.
+
+   Take vlen = 2048 as example for case 2.
+
+   Note: Below table based on vlen = 2048.
+   +----------------------------------------------------+----------------------+
+   | VLS mode                                           | VLA mode             |
+   +----------------------------------------------------+----------------------+
+   | Name       | Precision | Inner Precision | Enabled | Min mode  | Min bits |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1BI       |     1     |              1  | Yes     | RVVMF64BI |    32    |
+   | V2BI       |     2     |              1  | Yes     | RVVMF64BI |    32    |
+   | V4BI       |     4     |              1  | Yes     | RVVMF64BI |    32    |
+   | V8BI       |     8     |              1  | Yes     | RVVMF64BI |    32    |
+   | V16BI      |    16     |              1  | Yes     | RVVMF64BI |    32    |
+   | V32BI      |    32     |              1  | NO      | RVVMF64BI |    32    |
+   | V64BI      |    64     |              1  | NO      | RVVMF64BI |    32    |
+   | ...        |   ...     |            ...  | ...     | RVVMF64BI |    32    |
+   | V4096BI    |  4096     |              1  | NO      | RVVMF64BI |    32    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1QI       |     8     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V2QI       |    16     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V4QI       |    32     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V8QI       |    64     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V16QI      |   128     |              8  | Yes     | RVVMF8QI  |   256    |
+   | V32QI      |   256     |              8  | NO      | RVVMF8QI  |   256    |
+   | V64QI      |   512     |              8  | NO      | RVVMF8QI  |   256    |
+   | ...        |   ...     |              .. | ...     | RVVMF8QI  |   256    |
+   | V4096QI    | 32768     |              8  | NO      | RVVMF8QI  |   256    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1HI       |    16     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V2HI       |    32     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V4HI       |    64     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V8HI       |   128     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V16HI      |   256     |              16 | Yes     | RVVMF4HI  |   512    |
+   | V32HI      |   512     |              16 | NO      | RVVMF4HI  |   512    |
+   | V64HI      |  1024     |              16 | NO      | RVVMF4HI  |   512    |
+   | ...        |   ...     |              .. | ...     | RVVMF4HI  |   512    |
+   | V2048HI    | 32768     |              16 | NO      | RVVMF4HI  |   512    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1SI/SF    |    32     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V2SI/SF    |    64     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V4SI/SF    |   128     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V8SI/SF    |   256     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V16SI/SF   |   512     |              32 | Yes     | RVVMF2SI  |  1024    |
+   | V32SI/SF   |  1024     |              32 | NO      | RVVMF2SI  |  1024    |
+   | V64SI/SF   |  2048     |              32 | NO      | RVVMF2SI  |  1024    |
+   | ...        |   ...     |              .. | ...     | RVVMF2SI  |  1024    |
+   | V1024SI/SF | 32768     |              32 | NO      | RVVMF2SI  |  1024    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+   | V1DI/DF    |    64     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V2DI/DF    |   128     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V4DI/DF    |   256     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V8DI/DF    |   512     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V16DI/DF   |  1024     |              64 | Yes     | RVVM1DI   |  2048    |
+   | V32DI/DF   |  2048     |              64 | NO      | RVVM1DI   |  2048    |
+   | V64DI/DF   |  4096     |              64 | NO      | RVVM1DI   |  2048    |
+   | ...        |   ...     |              .. | ...     | RVVM1DI   |  2048    |
+   | V512DI/DF  | 32768     |              64 | NO      | RVVM1DI   |  2048    |
+   +------------+-----------+-----------------+---------+-----------+----------+
+
+   Then we can have the condition for VLS mode in fixed-vlmax, aka:
+     PRECISION (VLSmode) < VLEN / (64 / PRECISION(VLS_inner_mode)).  */
+bool
+vls_mode_valid_p (machine_mode vls_mode)
+{
+  if (!TARGET_VECTOR)
+    return false;
+
+  if (riscv_autovec_preference == RVV_SCALABLE)
+    return true;
+
+  if (riscv_autovec_preference == RVV_FIXED_VLMAX)
+    {
+      machine_mode inner_mode = GET_MODE_INNER (vls_mode);
+      int precision = GET_MODE_PRECISION (inner_mode).to_constant ();
+      int min_vlmax_bitsize = TARGET_MIN_VLEN / (64 / precision);
+
+      return GET_MODE_PRECISION (vls_mode).to_constant () < min_vlmax_bitsize;
+    }
+
+  return false;
+}
+
/* Return true if the gather/scatter offset mode is valid.  */
bool
gather_scatter_valid_offset_mode_p (machine_mode mode)
diff --git a/gcc/config/riscv/riscv-vector-switch.def b/gcc/config/riscv/riscv-vector-switch.def
index 174e5a181c6..5c9f9bcbc3e 100644
--- a/gcc/config/riscv/riscv-vector-switch.def
+++ b/gcc/config/riscv/riscv-vector-switch.def
@@ -292,100 +292,100 @@ TUPLE_ENTRY (RVVM1x2DF, TARGET_VECTOR_ELEN_FP_64, RVVM1DF, 2, LMUL_1, 16)
/* This following VLS modes should satisfy the constraint:
    GET_MODE_BITSIZE (MODE) <= TARGET_MIN_VLEN * 8.  */
-VLS_ENTRY (V1BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32BI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096BI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32QI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V64QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V128QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V256QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V512QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V1024QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V2048QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V4096QI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16HI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V32HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V2SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V4SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V8SI, TARGET_VECTOR_VLS)
-VLS_ENTRY (V16SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SI, TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V2DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V4DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64)
-VLS_ENTRY (V8DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DI, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
-
-VLS_ENTRY (V1HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V2HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V4HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V8HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V16HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16)
-VLS_ENTRY (V32HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V64HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V128HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V256HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V512HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V1024HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V2048HF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V2SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V4SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V8SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32)
-VLS_ENTRY (V16SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V32SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V64SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V128SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V256SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V512SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V1024SF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
-VLS_ENTRY (V1DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V2DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V4DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64)
-VLS_ENTRY (V8DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
-VLS_ENTRY (V16DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
-VLS_ENTRY (V32DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
-VLS_ENTRY (V64DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
-VLS_ENTRY (V128DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
-VLS_ENTRY (V256DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
-VLS_ENTRY (V512DF, TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1BI, riscv_vector::vls_mode_valid_p (V1BImode))
+VLS_ENTRY (V2BI, riscv_vector::vls_mode_valid_p (V2BImode))
+VLS_ENTRY (V4BI, riscv_vector::vls_mode_valid_p (V4BImode))
+VLS_ENTRY (V8BI, riscv_vector::vls_mode_valid_p (V8BImode))
+VLS_ENTRY (V16BI, riscv_vector::vls_mode_valid_p (V16BImode))
+VLS_ENTRY (V32BI, riscv_vector::vls_mode_valid_p (V32BImode))
+VLS_ENTRY (V64BI, riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128BI, riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256BI, riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512BI, riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024BI, riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048BI, riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096BI, riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1QI, riscv_vector::vls_mode_valid_p (V1QImode))
+VLS_ENTRY (V2QI, riscv_vector::vls_mode_valid_p (V2QImode))
+VLS_ENTRY (V4QI, riscv_vector::vls_mode_valid_p (V4QImode))
+VLS_ENTRY (V8QI, riscv_vector::vls_mode_valid_p (V8QImode))
+VLS_ENTRY (V16QI, riscv_vector::vls_mode_valid_p (V16QImode))
+VLS_ENTRY (V32QI, riscv_vector::vls_mode_valid_p (V32QImode))
+VLS_ENTRY (V64QI, riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V128QI, riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V256QI, riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V512QI, riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V1024QI, riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V2048QI, riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V4096QI, riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1HI, riscv_vector::vls_mode_valid_p (V1HImode))
+VLS_ENTRY (V2HI, riscv_vector::vls_mode_valid_p (V2HImode))
+VLS_ENTRY (V4HI, riscv_vector::vls_mode_valid_p (V4HImode))
+VLS_ENTRY (V8HI, riscv_vector::vls_mode_valid_p (V8HImode))
+VLS_ENTRY (V16HI, riscv_vector::vls_mode_valid_p (V16HImode))
+VLS_ENTRY (V32HI, riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HI, riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HI, riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HI, riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HI, riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HI, riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HI, riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SI, riscv_vector::vls_mode_valid_p (V1SImode))
+VLS_ENTRY (V2SI, riscv_vector::vls_mode_valid_p (V2SImode))
+VLS_ENTRY (V4SI, riscv_vector::vls_mode_valid_p (V4SImode))
+VLS_ENTRY (V8SI, riscv_vector::vls_mode_valid_p (V8SImode))
+VLS_ENTRY (V16SI, riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SI, riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SI, riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SI, riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SI, riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SI, riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SI, riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DI, riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V2DI, riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V4DI, riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64)
+VLS_ENTRY (V8DI, riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DI, riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DI, riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DI, riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DI, riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DI, riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DI, riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096)
+
+VLS_ENTRY (V1HF, riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V2HF, riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V4HF, riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V8HF, riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V16HF, riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16)
+VLS_ENTRY (V32HF, riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V64HF, riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V128HF, riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V256HF, riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V512HF, riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V1024HF, riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V2048HF, riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1SF, riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V2SF, riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V4SF, riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V8SF, riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32)
+VLS_ENTRY (V16SF, riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V32SF, riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V64SF, riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V128SF, riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V256SF, riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V512SF, riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V1024SF, riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096)
+VLS_ENTRY (V1DF, riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V2DF, riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V4DF, riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64)
+VLS_ENTRY (V8DF, riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64)
+VLS_ENTRY (V16DF, riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128)
+VLS_ENTRY (V32DF, riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256)
+VLS_ENTRY (V64DF, riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512)
+VLS_ENTRY (V128DF, riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024)
+VLS_ENTRY (V256DF, riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048)
+VLS_ENTRY (V512DF, riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096)
#undef VLS_ENTRY
#undef TUPLE_ENTRY
diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md
index 0850475edc1..e80a20dcfba 100644
--- a/gcc/config/riscv/vector-iterators.md
+++ b/gcc/config/riscv/vector-iterators.md
@@ -1,3 +1,4 @@
+
;; Iterators for RISC-V 'V' Extension for GNU compiler.
;; Copyright (C) 2022-2023 Free Software Foundation, Inc.
;; Contributed by Juzhe Zhong (juzhe.zhong@rivai.ai), RiVAI Technologies Ltd.
@@ -146,85 +147,85 @@ (define_mode_iterator V_VLS [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
   ;; VLS modes.
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VEEWEXT2 [
@@ -315,84 +316,84 @@ (define_mode_iterator VEI16 [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI [
@@ -416,52 +417,52 @@ (define_mode_iterator V_VLSI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator V_VLSF [
@@ -473,39 +474,39 @@ (define_mode_iterator V_VLSF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VF_ZVFHMIN [
@@ -531,39 +532,39 @@ (define_mode_iterator V_VLSF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
;; This iterator is the same as above but with TARGET_VECTOR_ELEN_FP_16
@@ -594,52 +595,52 @@ (define_mode_iterator VFULLI [
   (RVVM8DI "TARGET_FULL_V") (RVVM4DI "TARGET_FULL_V") (RVVM2DI "TARGET_FULL_V") (RVVM1DI "TARGET_FULL_V")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_FULL_V")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_FULL_V")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_FULL_V")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_FULL_V")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_FULL_V && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_QH [
@@ -655,42 +656,42 @@ (define_mode_iterator VI_QHS [
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_QHS_NO_M8 [
@@ -700,39 +701,39 @@ (define_mode_iterator VI_QHS_NO_M8 [
   RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
])
(define_mode_iterator VF_HS [
@@ -743,29 +744,29 @@ (define_mode_iterator VF_HS [
   (RVVM8SF "TARGET_VECTOR_ELEN_FP_32") (RVVM4SF "TARGET_VECTOR_ELEN_FP_32") (RVVM2SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32") (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VF_HS_NO_M8 [
@@ -779,27 +780,27 @@ (define_mode_iterator VF_HS_NO_M8 [
   (RVVM1SF "TARGET_VECTOR_ELEN_FP_32")
   (RVVMF2SF "TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN > 32")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
])
(define_mode_iterator VF_HS_M8 [
@@ -814,42 +815,42 @@ (define_mode_iterator V_VLSI_QHS [
   RVVM8SI RVVM4SI RVVM2SI RVVM1SI (RVVMF2SI "TARGET_MIN_VLEN > 32")
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VI_D [
@@ -861,16 +862,16 @@ (define_mode_iterator V_VLSI_D [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VFULLI_D [
@@ -1013,19 +1014,19 @@ (define_mode_iterator VB [
(define_mode_iterator VB_VLS [
   (RVVMF64BI "TARGET_MIN_VLEN > 32") RVVMF32BI RVVMF16BI RVVMF8BI RVVMF4BI RVVMF2BI RVVM1BI
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
+  (V1BI "riscv_vector::vls_mode_valid_p (V1BImode)")
+  (V2BI "riscv_vector::vls_mode_valid_p (V2BImode)")
+  (V4BI "riscv_vector::vls_mode_valid_p (V4BImode)")
+  (V8BI "riscv_vector::vls_mode_valid_p (V8BImode)")
+  (V16BI "riscv_vector::vls_mode_valid_p (V16BImode)")
+  (V32BI "riscv_vector::vls_mode_valid_p (V32BImode)")
+  (V64BI "riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWEXTI [
@@ -1036,39 +1037,39 @@ (define_mode_iterator VWEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
;; Same iterator split reason as VF_ZVFHMIN and VF.
@@ -1082,27 +1083,27 @@ (define_mode_iterator VWEXTF_ZVFHMIN [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWEXTF [
@@ -1115,27 +1116,27 @@ (define_mode_iterator VWEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWCONVERTI [
@@ -1147,27 +1148,27 @@ (define_mode_iterator VWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V1SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8SI "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode) && TARGET_ZVFH")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode) && TARGET_ZVFH")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode) && TARGET_ZVFH")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode) && TARGET_ZVFH")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VWWCONVERTI [
@@ -1176,16 +1177,16 @@ (define_mode_iterator VWWCONVERTI [
   (RVVM2DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
   (RVVM1DI "TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VQEXTI [
@@ -1194,59 +1195,59 @@ (define_mode_iterator VQEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VQEXTF [
   (RVVM8DF "TARGET_VECTOR_ELEN_FP_64") (RVVM4DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64") (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VOEXTI [
   (RVVM8DI "TARGET_VECTOR_ELEN_64") (RVVM4DI "TARGET_VECTOR_ELEN_64")
   (RVVM2DI "TARGET_VECTOR_ELEN_64") (RVVM1DI "TARGET_VECTOR_ELEN_64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_iterator VT [
@@ -3216,28 +3217,28 @@ (define_mode_iterator V_VLS_FCONVERT_I_L_LL [
   (RVVM2DF "TARGET_VECTOR_ELEN_FP_64")
   (RVVM1DF "TARGET_VECTOR_ELEN_FP_64")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
(define_mode_attr VDEMOTE [
@@ -3748,281 +3749,281 @@ (define_code_attr sz [(sign_extend "s") (zero_extend "z")])
;; VLS modes.
(define_mode_iterator VLS [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSB [
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V1BI "riscv_vector::vls_mode_valid_p (V1BImode)")
+  (V2BI "riscv_vector::vls_mode_valid_p (V2BImode)")
+  (V4BI "riscv_vector::vls_mode_valid_p (V4BImode)")
+  (V8BI "riscv_vector::vls_mode_valid_p (V8BImode)")
+  (V16BI "riscv_vector::vls_mode_valid_p (V16BImode)")
+  (V32BI "riscv_vector::vls_mode_valid_p (V32BImode)")
+  (V64BI "riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
;; VLS modes that has NUNITS < 32.
(define_mode_iterator VLS_AVL_IMM [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V1HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-
-  (V1BI "TARGET_VECTOR_VLS")
-  (V2BI "TARGET_VECTOR_VLS")
-  (V4BI "TARGET_VECTOR_VLS")
-  (V8BI "TARGET_VECTOR_VLS")
-  (V16BI "TARGET_VECTOR_VLS")])
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_VECTOR_ELEN_FP_16")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+
+  (V1BI "riscv_vector::vls_mode_valid_p (V1BImode)")
+  (V2BI "riscv_vector::vls_mode_valid_p (V2BImode)")
+  (V4BI "riscv_vector::vls_mode_valid_p (V4BImode)")
+  (V8BI "riscv_vector::vls_mode_valid_p (V8BImode)")
+  (V16BI "riscv_vector::vls_mode_valid_p (V16BImode)")])
;; VLS modes that has NUNITS >= 32.
(define_mode_iterator VLS_AVL_REG [
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
-
-  (V32BI "TARGET_VECTOR_VLS")
-  (V64BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096BI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")])
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_VECTOR_ELEN_FP_16 && TARGET_MIN_VLEN >= 4096")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+
+  (V32BI "riscv_vector::vls_mode_valid_p (V32BImode)")
+  (V64BI "riscv_vector::vls_mode_valid_p (V64BImode) && TARGET_MIN_VLEN >= 64")
+  (V128BI "riscv_vector::vls_mode_valid_p (V128BImode) && TARGET_MIN_VLEN >= 128")
+  (V256BI "riscv_vector::vls_mode_valid_p (V256BImode) && TARGET_MIN_VLEN >= 256")
+  (V512BI "riscv_vector::vls_mode_valid_p (V512BImode) && TARGET_MIN_VLEN >= 512")
+  (V1024BI "riscv_vector::vls_mode_valid_p (V1024BImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048BI "riscv_vector::vls_mode_valid_p (V2048BImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096BI "riscv_vector::vls_mode_valid_p (V4096BImode) && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSI [
-  (V1QI "TARGET_VECTOR_VLS")
-  (V2QI "TARGET_VECTOR_VLS")
-  (V4QI "TARGET_VECTOR_VLS")
-  (V8QI "TARGET_VECTOR_VLS")
-  (V16QI "TARGET_VECTOR_VLS")
-  (V32QI "TARGET_VECTOR_VLS")
-  (V64QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V128QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V256QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V512QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V1024QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V2048QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V4096QI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1HI "TARGET_VECTOR_VLS")
-  (V2HI "TARGET_VECTOR_VLS")
-  (V4HI "TARGET_VECTOR_VLS")
-  (V8HI "TARGET_VECTOR_VLS")
-  (V16HI "TARGET_VECTOR_VLS")
-  (V32HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V64HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V128HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V256HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V512HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V1024HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V2048HI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1SI "TARGET_VECTOR_VLS")
-  (V2SI "TARGET_VECTOR_VLS")
-  (V4SI "TARGET_VECTOR_VLS")
-  (V8SI "TARGET_VECTOR_VLS")
-  (V16SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 64")
-  (V32SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 128")
-  (V64SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 256")
-  (V128SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 512")
-  (V256SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 1024")
-  (V512SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 2048")
-  (V1024SI "TARGET_VECTOR_VLS && TARGET_MIN_VLEN >= 4096")
-  (V1DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V2DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V4DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64")
-  (V8DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
-  (V16DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
-  (V32DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
-  (V64DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
-  (V128DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DI "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
+  (V1QI "riscv_vector::vls_mode_valid_p (V1QImode)")
+  (V2QI "riscv_vector::vls_mode_valid_p (V2QImode)")
+  (V4QI "riscv_vector::vls_mode_valid_p (V4QImode)")
+  (V8QI "riscv_vector::vls_mode_valid_p (V8QImode)")
+  (V16QI "riscv_vector::vls_mode_valid_p (V16QImode)")
+  (V32QI "riscv_vector::vls_mode_valid_p (V32QImode)")
+  (V64QI "riscv_vector::vls_mode_valid_p (V64QImode) && TARGET_MIN_VLEN >= 64")
+  (V128QI "riscv_vector::vls_mode_valid_p (V128QImode) && TARGET_MIN_VLEN >= 128")
+  (V256QI "riscv_vector::vls_mode_valid_p (V256QImode) && TARGET_MIN_VLEN >= 256")
+  (V512QI "riscv_vector::vls_mode_valid_p (V512QImode) && TARGET_MIN_VLEN >= 512")
+  (V1024QI "riscv_vector::vls_mode_valid_p (V1024QImode) && TARGET_MIN_VLEN >= 1024")
+  (V2048QI "riscv_vector::vls_mode_valid_p (V2048QImode) && TARGET_MIN_VLEN >= 2048")
+  (V4096QI "riscv_vector::vls_mode_valid_p (V4096QImode) && TARGET_MIN_VLEN >= 4096")
+  (V1HI "riscv_vector::vls_mode_valid_p (V1HImode)")
+  (V2HI "riscv_vector::vls_mode_valid_p (V2HImode)")
+  (V4HI "riscv_vector::vls_mode_valid_p (V4HImode)")
+  (V8HI "riscv_vector::vls_mode_valid_p (V8HImode)")
+  (V16HI "riscv_vector::vls_mode_valid_p (V16HImode)")
+  (V32HI "riscv_vector::vls_mode_valid_p (V32HImode) && TARGET_MIN_VLEN >= 64")
+  (V64HI "riscv_vector::vls_mode_valid_p (V64HImode) && TARGET_MIN_VLEN >= 128")
+  (V128HI "riscv_vector::vls_mode_valid_p (V128HImode) && TARGET_MIN_VLEN >= 256")
+  (V256HI "riscv_vector::vls_mode_valid_p (V256HImode) && TARGET_MIN_VLEN >= 512")
+  (V512HI "riscv_vector::vls_mode_valid_p (V512HImode) && TARGET_MIN_VLEN >= 1024")
+  (V1024HI "riscv_vector::vls_mode_valid_p (V1024HImode) && TARGET_MIN_VLEN >= 2048")
+  (V2048HI "riscv_vector::vls_mode_valid_p (V2048HImode) && TARGET_MIN_VLEN >= 4096")
+  (V1SI "riscv_vector::vls_mode_valid_p (V1SImode)")
+  (V2SI "riscv_vector::vls_mode_valid_p (V2SImode)")
+  (V4SI "riscv_vector::vls_mode_valid_p (V4SImode)")
+  (V8SI "riscv_vector::vls_mode_valid_p (V8SImode)")
+  (V16SI "riscv_vector::vls_mode_valid_p (V16SImode) && TARGET_MIN_VLEN >= 64")
+  (V32SI "riscv_vector::vls_mode_valid_p (V32SImode) && TARGET_MIN_VLEN >= 128")
+  (V64SI "riscv_vector::vls_mode_valid_p (V64SImode) && TARGET_MIN_VLEN >= 256")
+  (V128SI "riscv_vector::vls_mode_valid_p (V128SImode) && TARGET_MIN_VLEN >= 512")
+  (V256SI "riscv_vector::vls_mode_valid_p (V256SImode) && TARGET_MIN_VLEN >= 1024")
+  (V512SI "riscv_vector::vls_mode_valid_p (V512SImode) && TARGET_MIN_VLEN >= 2048")
+  (V1024SI "riscv_vector::vls_mode_valid_p (V1024SImode) && TARGET_MIN_VLEN >= 4096")
+  (V1DI "riscv_vector::vls_mode_valid_p (V1DImode) && TARGET_VECTOR_ELEN_64")
+  (V2DI "riscv_vector::vls_mode_valid_p (V2DImode) && TARGET_VECTOR_ELEN_64")
+  (V4DI "riscv_vector::vls_mode_valid_p (V4DImode) && TARGET_VECTOR_ELEN_64")
+  (V8DI "riscv_vector::vls_mode_valid_p (V8DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 64")
+  (V16DI "riscv_vector::vls_mode_valid_p (V16DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 128")
+  (V32DI "riscv_vector::vls_mode_valid_p (V32DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 256")
+  (V64DI "riscv_vector::vls_mode_valid_p (V64DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 512")
+  (V128DI "riscv_vector::vls_mode_valid_p (V128DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DI "riscv_vector::vls_mode_valid_p (V256DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DI "riscv_vector::vls_mode_valid_p (V512DImode) && TARGET_VECTOR_ELEN_64 && TARGET_MIN_VLEN >= 4096")])
(define_mode_iterator VLSF [
-  (V1HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V2HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V4HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V8HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V16HF "TARGET_VECTOR_VLS && TARGET_ZVFH")
-  (V32HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
-  (V64HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
-  (V128HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
-  (V256HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
-  (V512HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
-  (V1024HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
-  (V2048HF "TARGET_VECTOR_VLS && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
-  (V1SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V2SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V4SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V8SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32")
-  (V16SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
-  (V32SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
-  (V64SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
-  (V128SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
-  (V256SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
-  (V512SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
-  (V1024SF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
-  (V1DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V2DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V4DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64")
-  (V8DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
-  (V16DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
-  (V32DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
-  (V64DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
-  (V128DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
-  (V256DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
-  (V512DF "TARGET_VECTOR_VLS && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
+  (V1HF "riscv_vector::vls_mode_valid_p (V1HFmode) && TARGET_ZVFH")
+  (V2HF "riscv_vector::vls_mode_valid_p (V2HFmode) && TARGET_ZVFH")
+  (V4HF "riscv_vector::vls_mode_valid_p (V4HFmode) && TARGET_ZVFH")
+  (V8HF "riscv_vector::vls_mode_valid_p (V8HFmode) && TARGET_ZVFH")
+  (V16HF "riscv_vector::vls_mode_valid_p (V16HFmode) && TARGET_ZVFH")
+  (V32HF "riscv_vector::vls_mode_valid_p (V32HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 64")
+  (V64HF "riscv_vector::vls_mode_valid_p (V64HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 128")
+  (V128HF "riscv_vector::vls_mode_valid_p (V128HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 256")
+  (V256HF "riscv_vector::vls_mode_valid_p (V256HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 512")
+  (V512HF "riscv_vector::vls_mode_valid_p (V512HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 1024")
+  (V1024HF "riscv_vector::vls_mode_valid_p (V1024HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 2048")
+  (V2048HF "riscv_vector::vls_mode_valid_p (V2048HFmode) && TARGET_ZVFH && TARGET_MIN_VLEN >= 4096")
+  (V1SF "riscv_vector::vls_mode_valid_p (V1SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V2SF "riscv_vector::vls_mode_valid_p (V2SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V4SF "riscv_vector::vls_mode_valid_p (V4SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V8SF "riscv_vector::vls_mode_valid_p (V8SFmode) && TARGET_VECTOR_ELEN_FP_32")
+  (V16SF "riscv_vector::vls_mode_valid_p (V16SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 64")
+  (V32SF "riscv_vector::vls_mode_valid_p (V32SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 128")
+  (V64SF "riscv_vector::vls_mode_valid_p (V64SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 256")
+  (V128SF "riscv_vector::vls_mode_valid_p (V128SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 512")
+  (V256SF "riscv_vector::vls_mode_valid_p (V256SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 1024")
+  (V512SF "riscv_vector::vls_mode_valid_p (V512SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 2048")
+  (V1024SF "riscv_vector::vls_mode_valid_p (V1024SFmode) && TARGET_VECTOR_ELEN_FP_32 && TARGET_MIN_VLEN >= 4096")
+  (V1DF "riscv_vector::vls_mode_valid_p (V1DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V2DF "riscv_vector::vls_mode_valid_p (V2DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V4DF "riscv_vector::vls_mode_valid_p (V4DFmode) && TARGET_VECTOR_ELEN_FP_64")
+  (V8DF "riscv_vector::vls_mode_valid_p (V8DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 64")
+  (V16DF "riscv_vector::vls_mode_valid_p (V16DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 128")
+  (V32DF "riscv_vector::vls_mode_valid_p (V32DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 256")
+  (V64DF "riscv_vector::vls_mode_valid_p (V64DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 512")
+  (V128DF "riscv_vector::vls_mode_valid_p (V128DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 1024")
+  (V256DF "riscv_vector::vls_mode_valid_p (V256DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 2048")
+  (V512DF "riscv_vector::vls_mode_valid_p (V512DFmode) && TARGET_VECTOR_ELEN_FP_64 && TARGET_MIN_VLEN >= 4096")
])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
index 7ab31043547..b235ec727b1 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/perm-4.c
@@ -55,7 +55,7 @@
TEST_ALL (PERMUTE)
-/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 19 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 18 } } */
/* { dg-final { scan-assembler-times {vrgatherei16\.vv\tv[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 12 } } */
-/* { dg-final { scan-assembler-times {vrsub\.vi} 24 } } */
+/* { dg-final { scan-assembler-times {vrsub\.vi} 23 } } */
/* { dg-final { scan-assembler-times {vrsub\.vx} 7 } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c
new file mode 100644
index 00000000000..57af7db5cb1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-0.c
@@ -0,0 +1,536 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1qi,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2qi,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4qi,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8qi,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v4096qi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 10 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c
new file mode 100644
index 00000000000..2619658fc9b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-1.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hi,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hi,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hi,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hi,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hi,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*v?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c
new file mode 100644
index 00000000000..16aadba1276
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-2.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1si,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2si,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4si,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8si,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024si,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c
new file mode 100644
index 00000000000..5c5e93aef8c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-3.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1di,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2di,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4di,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8di,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512di,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c
new file mode 100644
index 00000000000..6aa4a7e615c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-4.c
@@ -0,0 +1,279 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1hf,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2hf,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4hf,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8hf,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v2048hf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgather\.v[vi]\s+v[0-9]+,\s*v[0-9]+,\s*[v]?[0-9]+} 9 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c
new file mode 100644
index 00000000000..a31dad5a926
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-5.c
@@ -0,0 +1,151 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1sf,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2sf,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4sf,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8sf,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v1024sf,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 5 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c
new file mode 100644
index 00000000000..8312b21c497
--- /dev/null
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls-vlmax/pr111857-6.c
@@ -0,0 +1,86 @@
+/* { dg-do compile } */
+/* { dg-options "-march=rv64gcv_zvfh_zvl4096b -std=c99 -mabi=lp64d -O3 -fno-schedule-insns -fno-schedule-insns2" } */
+
+#include "../vls/def.h"
+
+DEF_OP_VV_VA (__builtin_shufflevector, v1df,
+       0)
+DEF_OP_VV_VA (__builtin_shufflevector, v2df,
+       0, 1)
+DEF_OP_VV_VA (__builtin_shufflevector, v4df,
+       0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v8df,
+       0, 1, 2, 3, 0, 1, 2, 3)
+DEF_OP_VV_VA (__builtin_shufflevector, v16df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v32df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v64df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v128df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v256df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+DEF_OP_VV_VA (__builtin_shufflevector, v512df,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15,
+       12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15, 12, 13, 14, 15)
+
+/* { dg-final { scan-assembler-times {vrgatherei16\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 3 } } */
+/* { dg-final { scan-assembler-times {vrgather\.vv\s+v[0-9]+,\s*v[0-9]+,\s*v[0-9]+} 4 } } */
+/* { dg-final { scan-assembler-not {csrr} } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
index 8dd5bcf617d..8ee8b7b1801 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/vls/def.h
@@ -441,6 +441,12 @@ typedef double v512df __attribute__ ((vector_size (4096)));
     *(TYPE1 *) out = v;                                                        \
   }
+#define DEF_OP_VV_VA(OP, TYPE1, ...)                                           \
+  TYPE1 test_##OP##_##TYPE1 (TYPE1 a, TYPE1 b)                                 \
+  {                                                                            \
+    return OP (a, b, __VA_ARGS__);                                             \
+  }
+
#define DEF_REPEAT(TYPE1, TYPE2, NUM, ...)                                     \
   void init_##TYPE1##_##TYPE2##_##NUM (TYPE2 var0, TYPE2 var1,                 \
       TYPE2 *__restrict out)                  \
-- 
2.34.1
 
 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* RE: [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857]
  2023-10-21  0:38   ` 钟居哲
@ 2023-10-21  1:27     ` Li, Pan2
  0 siblings, 0 replies; 6+ messages in thread
From: Li, Pan2 @ 2023-10-21  1:27 UTC (permalink / raw)
  To: 钟居哲, gcc-patches; +Cc: Wang, Yanzhang, kito.cheng

[-- Attachment #1: Type: text/plain, Size: 33 bytes --]

Committed, thanks Juzhe.

Pan

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2023-10-21  1:27 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-10-20  8:34 [PATCH v1] RISC-V: Support partial VLS mode when preference fixed-vlmax pan2.li
2023-10-20  8:47 ` juzhe.zhong
2023-10-20  8:51   ` Li, Pan2
2023-10-20 12:59 ` [PATCH v2] RISC-V: Support partial VLS mode when preference fixed-vlmax [PR111857] pan2.li
2023-10-21  0:38   ` 钟居哲
2023-10-21  1:27     ` Li, Pan2

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).