Re: [PATCH #3], Fix _Complex when there are multiple FP types the same size

[Michael Meissner <meissner@linux.vnet.ibm.com>]

[-- Attachment #1: Type: text/plain, Size: 5454 bytes --]

On Mon, May 02, 2016 at 12:54:43PM +0200, Bernd Schmidt wrote:
> On 04/30/2016 06:00 PM, Segher Boessenkool wrote:
> >On Fri, Apr 29, 2016 at 04:51:27PM -0400, Michael Meissner wrote:
> >>2016-04-29  Michael Meissner  <meissner@linux.vnet.ibm.com>
> >>
> >>	* config/rs6000/rs6000.c (rs6000_hard_regno_nregs_internal): Add
> >>	support for __float128 complex datatypes.
> >>	(rs6000_hard_regno_mode_ok): Likewise.
> >>	(rs6000_setup_reg_addr_masks): Likewise.
> >>	(rs6000_complex_function_value): Likewise.
> >>	* config/rs6000/rs6000.h (FLOAT128_IEEE_P): Likewise.
> >>	__float128 and __ibm128 complex.
> >>	(FLOAT128_IBM_P): Likewise.
> >>	(ALTIVEC_ARG_MAX_RETURN): Likewise.
> >>	* doc/extend.texi (Additional Floating Types): Document that
> >>	-mfloat128 must be used to enable __float128.  Document complex
> >>	__float128 and __ibm128 support.
> >>
> >>[gcc/testsuite]
> >>2016-04-29  Michael Meissner  <meissner@linux.vnet.ibm.com>
> >>
> >>	* gcc.target/powerpc/float128-complex-1.c: New tests for complex
> >>	__float128.
> >>	* gcc.target/powerpc/float128-complex-2.c: Likewise.
> >
> >The powerpc parts are okay for trunk.  Thank you!
> 
> Ok for the other parts as well. Although I wonder:
> 
> >+      /* build_complex_type has set the expected mode to allow having multiple
> >+	 complex types for multiple floating point types that have the same
> >+	 size such as the PowerPC with __ibm128 and __float128.  If this was
> >+	 not set, figure out the mode manually.  */
> >+      if (TYPE_MODE (type) == VOIDmode)
> >+	{
> >+	  unsigned int precision = TYPE_PRECISION (TREE_TYPE (type));
> >+	  enum mode_class mclass = (TREE_CODE (TREE_TYPE (type)) == REAL_TYPE
> >+				    ? MODE_COMPLEX_FLOAT : MODE_COMPLEX_INT);
> >+	  SET_TYPE_MODE (type, mode_for_size (2 * precision, mclass, 0));
> >+	}
> 
> What happens if you assert that it isn't VOIDmode?

On the PowerPC, I did a full boostrap with the code changed to a gcc_assert,
and it didn't trigger.

However, in looking at it further, there are only two places that layout_type
is called after making a complex node.

The first is build_complex_type that I provided a patch for. The other is a
similar function in the Fortran front end (gfc_build_complex_type). Now,
assuming you only use the 3 standard floating point modes (float_type_node,
double_type_node, and long_double_type_node) you wouldn't notice any
problem. But if somehow Fortran can create a __float128 type, it would trigger
the assertion (in the new patches) or generate the wrong type (in the previous
patches).

So I would like to commit the following changes (assuming they bootstrap and
have no regressions) instead. Are these patches ok for the trunk, and
eventually the gcc 6.2 branch if they don't break other back ends?

Fortran people, I'm adding you to the To: list, because of the Fortran
change. What the problem is the current layout_type uses 2*size of the base
type to create the complex type. However, in the current PowerPC, we are
transitioning from using IBM extended double format (pair of doubles to give
more mantissa bits, but no extra exponent range) to IEEE 128-bit format, and we
have two 128-bit complex types. You get all sorts of errors if you want to use
the REAL or IMAGINARY parts and you get the wrong type.

The patch builds a table that maps a given base mode to the complex mode that
uses the base mode as the two elements (i.e. GET_MODE_COMPLEX_MODE (DFmode)
would return DCmode.

The machine independent changes are in the gcc-stage7.patch003b, and the
PowerPC specific changes are in gcc-stage7.patch003c.

[gcc]
2016-05-02  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* machmode.h (mode_complex): Add support to give the complex mode
	for a given mode.
	(GET_MODE_COMPLEX_MODE): Likewise.
	* stor-layout.c (layout_type): For COMPLEX_TYPE, use the mode
	stored by build_complex_type and gfc_build_complex_type instead of
	trying to figure out the appropriate mode based on the size. Raise
	an assertion error, if the type was not set.
	* genmodes.c (struct mode_data): Add field for the complex type of
	the given type.
	(blank_mode): Likewise.
	(make_complex_modes): Remember the complex mode created in the
	base type.
	(emit_mode_complex): Write out the mode_complex array to map a
	type mode to the complex version.
	(emit_insn_modes_c): Likewise.
	* tree.c (build_complex_type): Set the complex type to use before
	calling layout_type.
	* config/rs6000/rs6000.c (rs6000_hard_regno_nregs_internal): Add
	support for __float128 complex datatypes.
	(rs6000_hard_regno_mode_ok): Likewise.
	(rs6000_setup_reg_addr_masks): Likewise.
	(rs6000_complex_function_value): Likewise.
	* config/rs6000/rs6000.h (FLOAT128_IEEE_P): Likewise.
	__float128 and __ibm128 complex.
	(FLOAT128_IBM_P): Likewise.
	(ALTIVEC_ARG_MAX_RETURN): Likewise.
	* doc/extend.texi (Additional Floating Types): Document that
	-mfloat128 must be used to enable __float128.  Document complex
	__float128 and __ibm128 support.

[gcc/fortran]
2016-05-02  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* trans-types.c (gfc_build_complex_type):

[gcc/testsuite]
2016-05-02  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* gcc.target/powerpc/float128-complex-1.c: New tests for complex
	__float128.
	* gcc.target/powerpc/float128-complex-2.c: Likewise.

-- 
Michael Meissner, IBM
IBM, M/S 2506R, 550 King Street, Littleton, MA 01460-6245, USA
email: meissner@linux.vnet.ibm.com, phone: +1 (978) 899-4797

[-- Attachment #2: gcc-stage7.patch003b --]
[-- Type: text/plain, Size: 4713 bytes --]

Index: gcc/machmode.h
===================================================================
--- gcc/machmode.h	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/machmode.h)	(revision 235776)
+++ gcc/machmode.h	(.../gcc/machmode.h)	(working copy)
@@ -269,6 +269,10 @@ extern const unsigned char mode_wider[NU
 extern const unsigned char mode_2xwider[NUM_MACHINE_MODES];
 #define GET_MODE_2XWIDER_MODE(MODE) ((machine_mode) mode_2xwider[MODE])
 
+/* Get the complex mode from the component mode.  */
+extern const unsigned char mode_complex[NUM_MACHINE_MODES];
+#define GET_MODE_COMPLEX_MODE(MODE) ((machine_mode) mode_complex[MODE])
+
 /* Return the mode for data of a given size SIZE and mode class CLASS.
    If LIMIT is nonzero, then don't use modes bigger than MAX_FIXED_MODE_SIZE.
    The value is BLKmode if no other mode is found.  */
Index: gcc/stor-layout.c
===================================================================
--- gcc/stor-layout.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/stor-layout.c)	(revision 235776)
+++ gcc/stor-layout.c	(.../gcc/stor-layout.c)	(working copy)
@@ -2146,11 +2146,13 @@ layout_type (tree type)
 
     case COMPLEX_TYPE:
       TYPE_UNSIGNED (type) = TYPE_UNSIGNED (TREE_TYPE (type));
-      SET_TYPE_MODE (type,
-		     mode_for_size (2 * TYPE_PRECISION (TREE_TYPE (type)),
-				    (TREE_CODE (TREE_TYPE (type)) == REAL_TYPE
-				     ? MODE_COMPLEX_FLOAT : MODE_COMPLEX_INT),
-				     0));
+
+      /* build_complex_type and fortran's gfc_build_complex_type have set the
+	 expected mode to allow having multiple complex types for multiple
+	 floating point types that have the same size such as the PowerPC with
+	 __ibm128 and __float128.  */
+      gcc_assert (TYPE_MODE (type) != VOIDmode);
+
       TYPE_SIZE (type) = bitsize_int (GET_MODE_BITSIZE (TYPE_MODE (type)));
       TYPE_SIZE_UNIT (type) = size_int (GET_MODE_SIZE (TYPE_MODE (type)));
       break;
Index: gcc/genmodes.c
===================================================================
--- gcc/genmodes.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/genmodes.c)	(revision 235776)
+++ gcc/genmodes.c	(.../gcc/genmodes.c)	(working copy)
@@ -66,6 +66,7 @@ struct mode_data
 				   this mode as a component.  */
   struct mode_data *next_cont;  /* Next mode in that list.  */
 
+  struct mode_data *complex;	/* complex type with mode as component.  */
   const char *file;		/* file and line of definition, */
   unsigned int line;		/* for error reporting */
   unsigned int counter;		/* Rank ordering of modes */
@@ -83,7 +84,7 @@ static struct mode_data *void_mode;
 static const struct mode_data blank_mode = {
   0, "<unknown>", MAX_MODE_CLASS,
   -1U, -1U, -1U, -1U,
-  0, 0, 0, 0, 0,
+  0, 0, 0, 0, 0, 0,
   "<unknown>", 0, 0, 0, 0, false, 0
 };
 
@@ -472,6 +473,7 @@ make_complex_modes (enum mode_class cl,
 
       c = new_mode (cclass, buf, file, line);
       c->component = m;
+      m->complex = c;
     }
 }
 
@@ -1381,6 +1383,22 @@ emit_mode_wider (void)
 }
 
 static void
+emit_mode_complex (void)
+{
+  int c;
+  struct mode_data *m;
+
+  print_decl ("unsigned char", "mode_complex", "NUM_MACHINE_MODES");
+
+  for_all_modes (c, m)
+    tagged_printf ("%smode",
+		   m->complex ? m->complex->name : void_mode->name,
+		   m->name);
+
+  print_closer ();
+}
+
+static void
 emit_mode_mask (void)
 {
   int c;
@@ -1745,6 +1763,7 @@ emit_insn_modes_c (void)
   emit_mode_size ();
   emit_mode_nunits ();
   emit_mode_wider ();
+  emit_mode_complex ();
   emit_mode_mask ();
   emit_mode_inner ();
   emit_mode_unit_size ();
Index: gcc/tree.c
===================================================================
--- gcc/tree.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/tree.c)	(revision 235776)
+++ gcc/tree.c	(.../gcc/tree.c)	(working copy)
@@ -8774,6 +8774,7 @@ build_complex_type (tree component_type)
   t = make_node (COMPLEX_TYPE);
 
   TREE_TYPE (t) = TYPE_MAIN_VARIANT (component_type);
+  SET_TYPE_MODE (t, GET_MODE_COMPLEX_MODE (TYPE_MODE (component_type)));
 
   /* If we already have such a type, use the old one.  */
   hstate.add_object (TYPE_HASH (component_type));
Index: gcc/fortran/trans-types.c
===================================================================
--- gcc/fortran/trans-types.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/fortran/trans-types.c)	(revision 235776)
+++ gcc/fortran/trans-types.c	(.../gcc/fortran/trans-types.c)	(working copy)
@@ -828,6 +828,7 @@ gfc_build_complex_type (tree scalar_type
 
   new_type = make_node (COMPLEX_TYPE);
   TREE_TYPE (new_type) = scalar_type;
+  SET_TYPE_MODE (new_type, GET_MODE_COMPLEX_MODE (TYPE_MODE (scalar_type)));
   layout_type (new_type);
   return new_type;
 }

[-- Attachment #3: gcc-stage7.patch003c --]
[-- Type: text/plain, Size: 34839 bytes --]

Index: gcc/config/rs6000/rs6000.c
===================================================================
--- gcc/config/rs6000/rs6000.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/config/rs6000)	(revision 235776)
+++ gcc/config/rs6000/rs6000.c	(.../gcc/config/rs6000)	(working copy)
@@ -1866,7 +1866,7 @@ rs6000_hard_regno_nregs_internal (int re
      128-bit floating point that can go in vector registers, which has VSX
      memory addressing.  */
   if (FP_REGNO_P (regno))
-    reg_size = (VECTOR_MEM_VSX_P (mode)
+    reg_size = (VECTOR_MEM_VSX_P (mode) || FLOAT128_VECTOR_P (mode)
 		? UNITS_PER_VSX_WORD
 		: UNITS_PER_FP_WORD);
 
@@ -1898,6 +1898,9 @@ rs6000_hard_regno_mode_ok (int regno, ma
 {
   int last_regno = regno + rs6000_hard_regno_nregs[mode][regno] - 1;
 
+  if (COMPLEX_MODE_P (mode))
+    mode = GET_MODE_INNER (mode);
+
   /* PTImode can only go in GPRs.  Quad word memory operations require even/odd
      register combinations, and use PTImode where we need to deal with quad
      word memory operations.  Don't allow quad words in the argument or frame
@@ -2700,7 +2703,16 @@ rs6000_setup_reg_addr_masks (void)
   for (m = 0; m < NUM_MACHINE_MODES; ++m)
     {
       machine_mode m2 = (machine_mode)m;
-      unsigned short msize = GET_MODE_SIZE (m2);
+      bool complex_p = false;
+      size_t msize;
+
+      if (COMPLEX_MODE_P (m2))
+	{
+	  complex_p = true;
+	  m2 = GET_MODE_INNER (m2);
+	}
+
+      msize = GET_MODE_SIZE (m2);
 
       /* SDmode is special in that we want to access it only via REG+REG
 	 addressing on power7 and above, since we want to use the LFIWZX and
@@ -2722,7 +2734,7 @@ rs6000_setup_reg_addr_masks (void)
 	      /* Indicate if the mode takes more than 1 physical register.  If
 		 it takes a single register, indicate it can do REG+REG
 		 addressing.  */
-	      if (nregs > 1 || m == BLKmode)
+	      if (nregs > 1 || m == BLKmode || complex_p)
 		addr_mask |= RELOAD_REG_MULTIPLE;
 	      else
 		addr_mask |= RELOAD_REG_INDEXED;
@@ -2738,7 +2750,7 @@ rs6000_setup_reg_addr_masks (void)
 		  && msize <= 8
 		  && !VECTOR_MODE_P (m2)
 		  && !FLOAT128_VECTOR_P (m2)
-		  && !COMPLEX_MODE_P (m2)
+		  && !complex_p
 		  && (m2 != DFmode || !TARGET_UPPER_REGS_DF)
 		  && (m2 != SFmode || !TARGET_UPPER_REGS_SF)
 		  && !(TARGET_E500_DOUBLE && msize == 8))
@@ -18202,25 +18214,33 @@ rs6000_secondary_reload_memory (rtx addr
     addr_mask = (reg_addr[mode].addr_mask[RELOAD_REG_VMX]
 		 & ~RELOAD_REG_AND_M16);
 
-  else
+  /* If the register allocator hasn't made up its mind yet on the register
+     class to use, settle on defaults to use.  */
+  else if (rclass == NO_REGS)
     {
-      if (TARGET_DEBUG_ADDR)
-	fprintf (stderr,
-		 "rs6000_secondary_reload_memory: mode = %s, class = %s, "
-		 "class is not GPR, FPR, VMX\n",
-		 GET_MODE_NAME (mode), reg_class_names[rclass]);
+      addr_mask = (reg_addr[mode].addr_mask[RELOAD_REG_ANY]
+		   & ~RELOAD_REG_AND_M16);
 
-      return -1;
+      if ((addr_mask & RELOAD_REG_MULTIPLE) != 0)
+	addr_mask &= ~(RELOAD_REG_INDEXED
+		       | RELOAD_REG_PRE_INCDEC
+		       | RELOAD_REG_PRE_MODIFY);
     }
 
+  else
+    addr_mask = 0;
+
   /* If the register isn't valid in this register class, just return now.  */
   if ((addr_mask & RELOAD_REG_VALID) == 0)
     {
       if (TARGET_DEBUG_ADDR)
-	fprintf (stderr,
-		 "rs6000_secondary_reload_memory: mode = %s, class = %s, "
-		 "not valid in class\n",
-		 GET_MODE_NAME (mode), reg_class_names[rclass]);
+	{
+	  fprintf (stderr,
+		   "rs6000_secondary_reload_memory: mode = %s, class = %s, "
+		   "not valid in class\n",
+		   GET_MODE_NAME (mode), reg_class_names[rclass]);
+	  debug_rtx (addr);
+	}
 
       return -1;
     }
@@ -18849,6 +18869,9 @@ rs6000_secondary_reload (bool in_p,
 	fprintf (stderr, ", reload func = %s, extra cost = %d",
 		 insn_data[sri->icode].name, sri->extra_cost);
 
+      else if (sri->extra_cost > 0)
+	fprintf (stderr, ", extra cost = %d", sri->extra_cost);
+
       fputs ("\n", stderr);
       debug_rtx (x);
     }
@@ -19242,6 +19265,16 @@ rs6000_preferred_reload_class (rtx x, en
   machine_mode mode = GET_MODE (x);
   bool is_constant = CONSTANT_P (x);
 
+  /* If a mode can't go in FPR/ALTIVEC/VSX registers, don't return a preferred
+     reload class for it.  */
+  if ((rclass == ALTIVEC_REGS || rclass == VSX_REGS)
+      && (reg_addr[mode].addr_mask[RELOAD_REG_VMX] & RELOAD_REG_VALID) == 0)
+    return NO_REGS;
+
+  if ((rclass == FLOAT_REGS || rclass == VSX_REGS)
+      && (reg_addr[mode].addr_mask[RELOAD_REG_FPR] & RELOAD_REG_VALID) == 0)
+    return NO_REGS;
+
   /* For VSX, see if we should prefer FLOAT_REGS or ALTIVEC_REGS.  Do not allow
      the reloading of address expressions using PLUS into floating point
      registers.  */
@@ -19291,6 +19324,25 @@ rs6000_preferred_reload_class (rtx x, en
       return NO_REGS;
     }
 
+  /* If we haven't picked a register class, and the type is a vector or
+     floating point type, prefer to use the VSX, FPR, or Altivec register
+     classes.  */
+  if (rclass == NO_REGS)
+    {
+      if (TARGET_VSX && VECTOR_MEM_VSX_OR_P8_VECTOR_P (mode))
+	return VSX_REGS;
+
+      if (TARGET_ALTIVEC && VECTOR_MEM_ALTIVEC_P (mode))
+	return ALTIVEC_REGS;
+
+      if (DECIMAL_FLOAT_MODE_P (mode))
+	return (TARGET_DFP) ? FLOAT_REGS : NO_REGS;
+
+      if (TARGET_FPRS && TARGET_HARD_FLOAT && FLOAT_MODE_P (mode)
+	  && (reg_addr[mode].addr_mask[RELOAD_REG_FPR] & RELOAD_REG_VALID) == 0)
+	return FLOAT_REGS;
+    }
+
   if (GET_MODE_CLASS (mode) == MODE_INT && rclass == NON_SPECIAL_REGS)
     return GENERAL_REGS;
 
@@ -34066,8 +34118,14 @@ rs6000_complex_function_value (machine_m
   machine_mode inner = GET_MODE_INNER (mode);
   unsigned int inner_bytes = GET_MODE_UNIT_SIZE (mode);
 
-  if (FLOAT_MODE_P (mode) && TARGET_HARD_FLOAT && TARGET_FPRS)
+  if (TARGET_FLOAT128
+      && ((mode == KCmode)
+	  || (mode == TCmode && TARGET_IEEEQUAD)))
+    regno = ALTIVEC_ARG_RETURN;
+
+  else if (FLOAT_MODE_P (mode) && TARGET_HARD_FLOAT && TARGET_FPRS)
     regno = FP_ARG_RETURN;
+
   else
     {
       regno = GP_ARG_RETURN;
Index: gcc/config/rs6000/rs6000.h
===================================================================
--- gcc/config/rs6000/rs6000.h	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/config/rs6000)	(revision 235776)
+++ gcc/config/rs6000/rs6000.h	(.../gcc/config/rs6000)	(working copy)
@@ -418,12 +418,12 @@ extern const char *host_detect_local_cpu
    Similarly IFmode is the IBM long double format even if the default is IEEE
    128-bit.  */
 #define FLOAT128_IEEE_P(MODE)						\
-  (((MODE) == TFmode && TARGET_IEEEQUAD)				\
-   || ((MODE) == KFmode))
+  ((TARGET_IEEEQUAD && ((MODE) == TFmode || (MODE) == TCmode))		\
+   || ((MODE) == KFmode) || ((MODE) == KCmode))
 
 #define FLOAT128_IBM_P(MODE)						\
-  (((MODE) == TFmode && !TARGET_IEEEQUAD)				\
-   || ((MODE) == IFmode))
+  ((!TARGET_IEEEQUAD && ((MODE) == TFmode || (MODE) == TCmode))		\
+   || ((MODE) == IFmode) || ((MODE) == ICmode))
 
 /* Helper macros to say whether a 128-bit floating point type can go in a
    single vector register, or whether it needs paired scalar values.  */
@@ -1775,7 +1775,9 @@ extern enum reg_class rs6000_constraints
 #define ALTIVEC_ARG_RETURN (FIRST_ALTIVEC_REGNO + 2)
 #define FP_ARG_MAX_RETURN (DEFAULT_ABI != ABI_ELFv2 ? FP_ARG_RETURN	\
 			   : (FP_ARG_RETURN + AGGR_ARG_NUM_REG - 1))
-#define ALTIVEC_ARG_MAX_RETURN (DEFAULT_ABI != ABI_ELFv2 ? ALTIVEC_ARG_RETURN \
+#define ALTIVEC_ARG_MAX_RETURN (DEFAULT_ABI != ABI_ELFv2		\
+				? (ALTIVEC_ARG_RETURN			\
+				   + (TARGET_FLOAT128 ? 1 : 0))		\
 			        : (ALTIVEC_ARG_RETURN + AGGR_ARG_NUM_REG - 1))
 
 /* Flags for the call/call_value rtl operations set up by function_arg */
Index: gcc/doc/extend.texi
===================================================================
--- gcc/doc/extend.texi	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/doc)	(revision 235776)
+++ gcc/doc/extend.texi	(.../gcc/doc)	(working copy)
@@ -962,8 +962,13 @@ complex @code{__float128} type.  When th
 would use the following syntax to declare @code{_Complex128} to be a
 complex @code{__float128} type:
 
+On the PowerPC Linux VSX targets, you can declare complex types using
+the corresponding internal complex type, @code{KCmode} for
+@code{__float128} type and @code{ICmode} for @code{__ibm128} type:
+
 @smallexample
-typedef _Complex float __attribute__((mode(KC))) _Complex128;
+typedef _Complex float __attribute__((mode(KC))) _Complex_float128;
+typedef _Complex float __attribute__((mode(IC))) _Complex_ibm128;
 @end smallexample
 
 Not all targets support additional floating-point types.
Index: gcc/testsuite/gcc.target/powerpc/float128-complex-1.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/float128-complex-1.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/testsuite/gcc.target/powerpc)	(revision 0)
+++ gcc/testsuite/gcc.target/powerpc/float128-complex-1.c	(.../gcc/testsuite/gcc.target/powerpc)	(revision 235777)
@@ -0,0 +1,157 @@
+/* { dg-do compile { target { powerpc*-*-linux* } } } */
+/* { dg-require-effective-target powerpc_float128_sw_ok } */
+/* { dg-skip-if "do not override -mcpu" { powerpc*-*-* } { "-mcpu=*" } { "-mcpu=power7" } } */
+/* { dg-options "-O2 -mcpu=power7 -mfloat128" } */
+
+#ifndef NO_FLOAT
+typedef _Complex float	float_complex;
+extern float_complex cfloat1 (void);
+extern float_complex cfloat2 (void);
+
+#define FLOAT_ARG(NAME, OP)	ARG_OP(float, float_complex, NAME, OP)
+#define FLOAT_PTR(NAME, OP)	PTR_OP(float, float_complex, NAME, OP)
+#define FLOAT_CALL()		CALL_OP(float, float_complex, cfloat1, cfloat2)
+
+#else
+#define FLOAT_ARG(NAME, OP)
+#define FLOAT_PTR(NAME, OP)
+#define FLOAT_CALL()
+#endif
+
+#ifndef NO_DOUBLE
+typedef _Complex double	double_complex;
+extern double_complex cdouble1 (void);
+extern double_complex cdouble2 (void);
+
+#define DOUBLE_ARG(NAME, OP)	ARG_OP(double, double_complex, NAME, OP)
+#define DOUBLE_PTR(NAME, OP)	PTR_OP(double, double_complex, NAME, OP)
+#define DOUBLE_CALL()		CALL_OP(double, double_complex, cdouble1, cdouble2)
+
+#else
+#define DOUBLE_ARG(NAME, OP)
+#define DOUBLE_PTR(NAME, OP)
+#define DOUBLE_CALL()
+#endif
+
+#ifndef NO_FLOAT128
+#ifdef __VSX__
+typedef _Complex float __attribute__((mode(KC)))	float128_complex;
+#else
+typedef _Complex float __attribute__((mode(TC)))	float128_complex;
+#endif
+
+extern float128_complex cfloat128_1 (void);
+extern float128_complex cfloat128_2 (void);
+
+#define FLOAT128_ARG(NAME, OP)	ARG_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_PTR(NAME, OP)	PTR_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_CALL()		CALL_OP(float128, float128_complex, cfloat128_1, cfloat128_2)
+
+#else
+#define FLOAT128_ARG(NAME, OP)
+#define FLOAT128_PTR(NAME, OP)
+#define FLOAT128_CALL()
+#endif
+
+#ifndef NO_LDOUBLE
+typedef _Complex long double ldouble_complex;
+extern ldouble_complex cldouble1 (void);
+extern ldouble_complex cldouble2 (void);
+
+#define LDOUBLE_ARG(NAME, OP)	ARG_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)	PTR_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_CALL()		CALL_OP(ldouble, ldouble_complex, cldouble1, cldouble2)
+
+#else
+#define LDOUBLE_ARG(NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)
+#define LDOUBLE_CALL()
+#endif
+
+
+#define ARG_OP(SUFFIX, TYPE, NAME, OP)					\
+TYPE arg_ ## NAME ## _ ## SUFFIX (TYPE a, TYPE b)			\
+{									\
+  return a OP b;							\
+}
+
+#define PTR_OP(SUFFIX, TYPE, NAME, OP)					\
+void ptr_ ## NAME ## _ ## SUFFIX (TYPE *p, TYPE *a, TYPE *b)		\
+{									\
+  *p = *a OP *b;							\
+}
+
+#define CALL_OP(SUFFIX, TYPE, FUNC1, FUNC2)				\
+TYPE call_ ## SUFFIX (void)						\
+{									\
+  TYPE value1 = FUNC1 ();						\
+  TYPE value2 = FUNC2 ();						\
+  return value1 + value2;						\
+}
+
+#ifndef NO_ARG
+#ifndef NO_ADD
+FLOAT_ARG    (add, +)
+DOUBLE_ARG   (add, +)
+FLOAT128_ARG (add, +)
+LDOUBLE_ARG  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_ARG    (sub, -)
+DOUBLE_ARG   (sub, -)
+FLOAT128_ARG (sub, -)
+LDOUBLE_ARG  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_ARG    (mul, *)
+DOUBLE_ARG   (mul, *)
+FLOAT128_ARG (mul, *)
+LDOUBLE_ARG  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_ARG    (div, /)
+DOUBLE_ARG   (div, /)
+FLOAT128_ARG (div, /)
+LDOUBLE_ARG  (div, /)
+#endif
+#endif
+
+#ifndef NO_PTR
+#ifndef NO_ADD
+FLOAT_PTR    (add, +)
+DOUBLE_PTR   (add, +)
+FLOAT128_PTR (add, +)
+LDOUBLE_PTR  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_PTR    (sub, -)
+DOUBLE_PTR   (sub, -)
+FLOAT128_PTR (sub, -)
+LDOUBLE_PTR  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_PTR    (mul, *)
+DOUBLE_PTR   (mul, *)
+FLOAT128_PTR (mul, *)
+LDOUBLE_PTR  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_PTR    (div, /)
+DOUBLE_PTR   (div, /)
+FLOAT128_PTR (div, /)
+LDOUBLE_PTR  (div, /)
+#endif
+#endif
+
+#ifndef NO_CALL
+FLOAT_CALL    ()
+DOUBLE_CALL   ()
+FLOAT128_CALL ()
+LDOUBLE_CALL  ()
+#endif
Index: gcc/testsuite/gcc.target/powerpc/float128-complex-2.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/float128-complex-2.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/testsuite/gcc.target/powerpc)	(revision 0)
+++ gcc/testsuite/gcc.target/powerpc/float128-complex-2.c	(.../gcc/testsuite/gcc.target/powerpc)	(revision 235777)
@@ -0,0 +1,160 @@
+/* { dg-do compile { target { powerpc*-*-linux* } } } */
+/* { dg-require-effective-target powerpc_float128_hw_ok } */
+/* { dg-skip-if "do not override -mcpu" { powerpc*-*-* } { "-mcpu=*" } { "-mcpu=power9" } } */
+/* { dg-options "-O2 -mcpu=power9 -mfloat128 -mfloat128-hardware" } */
+
+#ifndef NO_FLOAT
+typedef _Complex float	float_complex;
+extern float_complex cfloat1 (void);
+extern float_complex cfloat2 (void);
+
+#define FLOAT_ARG(NAME, OP)	ARG_OP(float, float_complex, NAME, OP)
+#define FLOAT_PTR(NAME, OP)	PTR_OP(float, float_complex, NAME, OP)
+#define FLOAT_CALL()		CALL_OP(float, float_complex, cfloat1, cfloat2)
+
+#else
+#define FLOAT_ARG(NAME, OP)
+#define FLOAT_PTR(NAME, OP)
+#define FLOAT_CALL()
+#endif
+
+#ifndef NO_DOUBLE
+typedef _Complex double	double_complex;
+extern double_complex cdouble1 (void);
+extern double_complex cdouble2 (void);
+
+#define DOUBLE_ARG(NAME, OP)	ARG_OP(double, double_complex, NAME, OP)
+#define DOUBLE_PTR(NAME, OP)	PTR_OP(double, double_complex, NAME, OP)
+#define DOUBLE_CALL()		CALL_OP(double, double_complex, cdouble1, cdouble2)
+
+#else
+#define DOUBLE_ARG(NAME, OP)
+#define DOUBLE_PTR(NAME, OP)
+#define DOUBLE_CALL()
+#endif
+
+#ifndef NO_FLOAT128
+#ifdef __VSX__
+typedef _Complex float __attribute__((mode(KC)))	float128_complex;
+#else
+typedef _Complex float __attribute__((mode(TC)))	float128_complex;
+#endif
+
+extern float128_complex cfloat128_1 (void);
+extern float128_complex cfloat128_2 (void);
+
+#define FLOAT128_ARG(NAME, OP)	ARG_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_PTR(NAME, OP)	PTR_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_CALL()		CALL_OP(float128, float128_complex, cfloat128_1, cfloat128_2)
+
+#else
+#define FLOAT128_ARG(NAME, OP)
+#define FLOAT128_PTR(NAME, OP)
+#define FLOAT128_CALL()
+#endif
+
+#ifndef NO_LDOUBLE
+typedef _Complex long double ldouble_complex;
+extern ldouble_complex cldouble1 (void);
+extern ldouble_complex cldouble2 (void);
+
+#define LDOUBLE_ARG(NAME, OP)	ARG_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)	PTR_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_CALL()		CALL_OP(ldouble, ldouble_complex, cldouble1, cldouble2)
+
+#else
+#define LDOUBLE_ARG(NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)
+#define LDOUBLE_CALL()
+#endif
+
+
+#define ARG_OP(SUFFIX, TYPE, NAME, OP)					\
+TYPE arg_ ## NAME ## _ ## SUFFIX (TYPE a, TYPE b)			\
+{									\
+  return a OP b;							\
+}
+
+#define PTR_OP(SUFFIX, TYPE, NAME, OP)					\
+void ptr_ ## NAME ## _ ## SUFFIX (TYPE *p, TYPE *a, TYPE *b)		\
+{									\
+  *p = *a OP *b;							\
+}
+
+#define CALL_OP(SUFFIX, TYPE, FUNC1, FUNC2)				\
+TYPE call_ ## SUFFIX (void)						\
+{									\
+  TYPE value1 = FUNC1 ();						\
+  TYPE value2 = FUNC2 ();						\
+  return value1 + value2;						\
+}
+
+#ifndef NO_ARG
+#ifndef NO_ADD
+FLOAT_ARG    (add, +)
+DOUBLE_ARG   (add, +)
+FLOAT128_ARG (add, +)
+LDOUBLE_ARG  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_ARG    (sub, -)
+DOUBLE_ARG   (sub, -)
+FLOAT128_ARG (sub, -)
+LDOUBLE_ARG  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_ARG    (mul, *)
+DOUBLE_ARG   (mul, *)
+FLOAT128_ARG (mul, *)
+LDOUBLE_ARG  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_ARG    (div, /)
+DOUBLE_ARG   (div, /)
+FLOAT128_ARG (div, /)
+LDOUBLE_ARG  (div, /)
+#endif
+#endif
+
+#ifndef NO_PTR
+#ifndef NO_ADD
+FLOAT_PTR    (add, +)
+DOUBLE_PTR   (add, +)
+FLOAT128_PTR (add, +)
+LDOUBLE_PTR  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_PTR    (sub, -)
+DOUBLE_PTR   (sub, -)
+FLOAT128_PTR (sub, -)
+LDOUBLE_PTR  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_PTR    (mul, *)
+DOUBLE_PTR   (mul, *)
+FLOAT128_PTR (mul, *)
+LDOUBLE_PTR  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_PTR    (div, /)
+DOUBLE_PTR   (div, /)
+FLOAT128_PTR (div, /)
+LDOUBLE_PTR  (div, /)
+#endif
+#endif
+
+#ifndef NO_CALL
+FLOAT_CALL    ()
+DOUBLE_CALL   ()
+FLOAT128_CALL ()
+LDOUBLE_CALL  ()
+#endif
+
+/* { dg-final { scan-assembler "xsaddqp"  } } */
+/* { dg-final { scan-assembler "xssubqp"  } } */
Index: gcc/config/rs6000/rs6000.c
===================================================================
--- gcc/config/rs6000/rs6000.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/config/rs6000)	(revision 235776)
+++ gcc/config/rs6000/rs6000.c	(.../gcc/config/rs6000)	(working copy)
@@ -1866,7 +1866,7 @@ rs6000_hard_regno_nregs_internal (int re
      128-bit floating point that can go in vector registers, which has VSX
      memory addressing.  */
   if (FP_REGNO_P (regno))
-    reg_size = (VECTOR_MEM_VSX_P (mode)
+    reg_size = (VECTOR_MEM_VSX_P (mode) || FLOAT128_VECTOR_P (mode)
 		? UNITS_PER_VSX_WORD
 		: UNITS_PER_FP_WORD);
 
@@ -1898,6 +1898,9 @@ rs6000_hard_regno_mode_ok (int regno, ma
 {
   int last_regno = regno + rs6000_hard_regno_nregs[mode][regno] - 1;
 
+  if (COMPLEX_MODE_P (mode))
+    mode = GET_MODE_INNER (mode);
+
   /* PTImode can only go in GPRs.  Quad word memory operations require even/odd
      register combinations, and use PTImode where we need to deal with quad
      word memory operations.  Don't allow quad words in the argument or frame
@@ -2699,8 +2702,17 @@ rs6000_setup_reg_addr_masks (void)
 
   for (m = 0; m < NUM_MACHINE_MODES; ++m)
     {
-      machine_mode m2 = (machine_mode)m;
-      unsigned short msize = GET_MODE_SIZE (m2);
+      machine_mode m2 = (machine_mode) m;
+      bool complex_p = false;
+      size_t msize;
+
+      if (COMPLEX_MODE_P (m2))
+	{
+	  complex_p = true;
+	  m2 = GET_MODE_INNER (m2);
+	}
+
+      msize = GET_MODE_SIZE (m2);
 
       /* SDmode is special in that we want to access it only via REG+REG
 	 addressing on power7 and above, since we want to use the LFIWZX and
@@ -2722,7 +2734,7 @@ rs6000_setup_reg_addr_masks (void)
 	      /* Indicate if the mode takes more than 1 physical register.  If
 		 it takes a single register, indicate it can do REG+REG
 		 addressing.  */
-	      if (nregs > 1 || m == BLKmode)
+	      if (nregs > 1 || m == BLKmode || complex_p)
 		addr_mask |= RELOAD_REG_MULTIPLE;
 	      else
 		addr_mask |= RELOAD_REG_INDEXED;
@@ -2738,7 +2750,7 @@ rs6000_setup_reg_addr_masks (void)
 		  && msize <= 8
 		  && !VECTOR_MODE_P (m2)
 		  && !FLOAT128_VECTOR_P (m2)
-		  && !COMPLEX_MODE_P (m2)
+		  && !complex_p
 		  && (m2 != DFmode || !TARGET_UPPER_REGS_DF)
 		  && (m2 != SFmode || !TARGET_UPPER_REGS_SF)
 		  && !(TARGET_E500_DOUBLE && msize == 8))
@@ -18202,25 +18214,33 @@ rs6000_secondary_reload_memory (rtx addr
     addr_mask = (reg_addr[mode].addr_mask[RELOAD_REG_VMX]
 		 & ~RELOAD_REG_AND_M16);
 
-  else
+  /* If the register allocator hasn't made up its mind yet on the register
+     class to use, settle on defaults to use.  */
+  else if (rclass == NO_REGS)
     {
-      if (TARGET_DEBUG_ADDR)
-	fprintf (stderr,
-		 "rs6000_secondary_reload_memory: mode = %s, class = %s, "
-		 "class is not GPR, FPR, VMX\n",
-		 GET_MODE_NAME (mode), reg_class_names[rclass]);
+      addr_mask = (reg_addr[mode].addr_mask[RELOAD_REG_ANY]
+		   & ~RELOAD_REG_AND_M16);
 
-      return -1;
+      if ((addr_mask & RELOAD_REG_MULTIPLE) != 0)
+	addr_mask &= ~(RELOAD_REG_INDEXED
+		       | RELOAD_REG_PRE_INCDEC
+		       | RELOAD_REG_PRE_MODIFY);
     }
 
+  else
+    addr_mask = 0;
+
   /* If the register isn't valid in this register class, just return now.  */
   if ((addr_mask & RELOAD_REG_VALID) == 0)
     {
       if (TARGET_DEBUG_ADDR)
-	fprintf (stderr,
-		 "rs6000_secondary_reload_memory: mode = %s, class = %s, "
-		 "not valid in class\n",
-		 GET_MODE_NAME (mode), reg_class_names[rclass]);
+	{
+	  fprintf (stderr,
+		   "rs6000_secondary_reload_memory: mode = %s, class = %s, "
+		   "not valid in class\n",
+		   GET_MODE_NAME (mode), reg_class_names[rclass]);
+	  debug_rtx (addr);
+	}
 
       return -1;
     }
@@ -18849,6 +18869,9 @@ rs6000_secondary_reload (bool in_p,
 	fprintf (stderr, ", reload func = %s, extra cost = %d",
 		 insn_data[sri->icode].name, sri->extra_cost);
 
+      else if (sri->extra_cost > 0)
+	fprintf (stderr, ", extra cost = %d", sri->extra_cost);
+
       fputs ("\n", stderr);
       debug_rtx (x);
     }
@@ -19242,6 +19265,16 @@ rs6000_preferred_reload_class (rtx x, en
   machine_mode mode = GET_MODE (x);
   bool is_constant = CONSTANT_P (x);
 
+  /* If a mode can't go in FPR/ALTIVEC/VSX registers, don't return a preferred
+     reload class for it.  */
+  if ((rclass == ALTIVEC_REGS || rclass == VSX_REGS)
+      && (reg_addr[mode].addr_mask[RELOAD_REG_VMX] & RELOAD_REG_VALID) == 0)
+    return NO_REGS;
+
+  if ((rclass == FLOAT_REGS || rclass == VSX_REGS)
+      && (reg_addr[mode].addr_mask[RELOAD_REG_FPR] & RELOAD_REG_VALID) == 0)
+    return NO_REGS;
+
   /* For VSX, see if we should prefer FLOAT_REGS or ALTIVEC_REGS.  Do not allow
      the reloading of address expressions using PLUS into floating point
      registers.  */
@@ -19291,6 +19324,25 @@ rs6000_preferred_reload_class (rtx x, en
       return NO_REGS;
     }
 
+  /* If we haven't picked a register class, and the type is a vector or
+     floating point type, prefer to use the VSX, FPR, or Altivec register
+     classes.  */
+  if (rclass == NO_REGS)
+    {
+      if (TARGET_VSX && VECTOR_MEM_VSX_OR_P8_VECTOR_P (mode))
+	return VSX_REGS;
+
+      if (TARGET_ALTIVEC && VECTOR_MEM_ALTIVEC_P (mode))
+	return ALTIVEC_REGS;
+
+      if (DECIMAL_FLOAT_MODE_P (mode))
+	return TARGET_DFP ? FLOAT_REGS : NO_REGS;
+
+      if (TARGET_FPRS && TARGET_HARD_FLOAT && FLOAT_MODE_P (mode)
+	  && (reg_addr[mode].addr_mask[RELOAD_REG_FPR] & RELOAD_REG_VALID) == 0)
+	return FLOAT_REGS;
+    }
+
   if (GET_MODE_CLASS (mode) == MODE_INT && rclass == NON_SPECIAL_REGS)
     return GENERAL_REGS;
 
@@ -34066,8 +34118,14 @@ rs6000_complex_function_value (machine_m
   machine_mode inner = GET_MODE_INNER (mode);
   unsigned int inner_bytes = GET_MODE_UNIT_SIZE (mode);
 
-  if (FLOAT_MODE_P (mode) && TARGET_HARD_FLOAT && TARGET_FPRS)
+  if (TARGET_FLOAT128
+      && (mode == KCmode
+	  || (mode == TCmode && TARGET_IEEEQUAD)))
+    regno = ALTIVEC_ARG_RETURN;
+
+  else if (FLOAT_MODE_P (mode) && TARGET_HARD_FLOAT && TARGET_FPRS)
     regno = FP_ARG_RETURN;
+
   else
     {
       regno = GP_ARG_RETURN;
Index: gcc/config/rs6000/rs6000.h
===================================================================
--- gcc/config/rs6000/rs6000.h	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/config/rs6000)	(revision 235776)
+++ gcc/config/rs6000/rs6000.h	(.../gcc/config/rs6000)	(working copy)
@@ -418,12 +418,12 @@ extern const char *host_detect_local_cpu
    Similarly IFmode is the IBM long double format even if the default is IEEE
    128-bit.  */
 #define FLOAT128_IEEE_P(MODE)						\
-  (((MODE) == TFmode && TARGET_IEEEQUAD)				\
-   || ((MODE) == KFmode))
+  ((TARGET_IEEEQUAD && ((MODE) == TFmode || (MODE) == TCmode))		\
+   || ((MODE) == KFmode) || ((MODE) == KCmode))
 
 #define FLOAT128_IBM_P(MODE)						\
-  (((MODE) == TFmode && !TARGET_IEEEQUAD)				\
-   || ((MODE) == IFmode))
+  ((!TARGET_IEEEQUAD && ((MODE) == TFmode || (MODE) == TCmode))		\
+   || ((MODE) == IFmode) || ((MODE) == ICmode))
 
 /* Helper macros to say whether a 128-bit floating point type can go in a
    single vector register, or whether it needs paired scalar values.  */
@@ -1775,7 +1775,9 @@ extern enum reg_class rs6000_constraints
 #define ALTIVEC_ARG_RETURN (FIRST_ALTIVEC_REGNO + 2)
 #define FP_ARG_MAX_RETURN (DEFAULT_ABI != ABI_ELFv2 ? FP_ARG_RETURN	\
 			   : (FP_ARG_RETURN + AGGR_ARG_NUM_REG - 1))
-#define ALTIVEC_ARG_MAX_RETURN (DEFAULT_ABI != ABI_ELFv2 ? ALTIVEC_ARG_RETURN \
+#define ALTIVEC_ARG_MAX_RETURN (DEFAULT_ABI != ABI_ELFv2		\
+				? (ALTIVEC_ARG_RETURN			\
+				   + (TARGET_FLOAT128 ? 1 : 0))		\
 			        : (ALTIVEC_ARG_RETURN + AGGR_ARG_NUM_REG - 1))
 
 /* Flags for the call/call_value rtl operations set up by function_arg */
Index: gcc/doc/extend.texi
===================================================================
--- gcc/doc/extend.texi	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/doc)	(revision 235776)
+++ gcc/doc/extend.texi	(.../gcc/doc)	(working copy)
@@ -962,8 +962,13 @@ complex @code{__float128} type.  When th
 would use the following syntax to declare @code{_Complex128} to be a
 complex @code{__float128} type:
 
+On the PowerPC Linux VSX targets, you can declare complex types using
+the corresponding internal complex type, @code{KCmode} for
+@code{__float128} type and @code{ICmode} for @code{__ibm128} type:
+
 @smallexample
-typedef _Complex float __attribute__((mode(KC))) _Complex128;
+typedef _Complex float __attribute__((mode(KC))) _Complex_float128;
+typedef _Complex float __attribute__((mode(IC))) _Complex_ibm128;
 @end smallexample
 
 Not all targets support additional floating-point types.
Index: gcc/testsuite/gcc.target/powerpc/float128-complex-1.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/float128-complex-1.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/testsuite/gcc.target/powerpc)	(revision 0)
+++ gcc/testsuite/gcc.target/powerpc/float128-complex-1.c	(.../gcc/testsuite/gcc.target/powerpc)	(revision 235777)
@@ -0,0 +1,157 @@
+/* { dg-do compile { target { powerpc*-*-linux* } } } */
+/* { dg-require-effective-target powerpc_float128_sw_ok } */
+/* { dg-skip-if "do not override -mcpu" { powerpc*-*-* } { "-mcpu=*" } { "-mcpu=power7" } } */
+/* { dg-options "-O2 -mcpu=power7 -mfloat128" } */
+
+#ifndef NO_FLOAT
+typedef _Complex float	float_complex;
+extern float_complex cfloat1 (void);
+extern float_complex cfloat2 (void);
+
+#define FLOAT_ARG(NAME, OP)	ARG_OP(float, float_complex, NAME, OP)
+#define FLOAT_PTR(NAME, OP)	PTR_OP(float, float_complex, NAME, OP)
+#define FLOAT_CALL()		CALL_OP(float, float_complex, cfloat1, cfloat2)
+
+#else
+#define FLOAT_ARG(NAME, OP)
+#define FLOAT_PTR(NAME, OP)
+#define FLOAT_CALL()
+#endif
+
+#ifndef NO_DOUBLE
+typedef _Complex double	double_complex;
+extern double_complex cdouble1 (void);
+extern double_complex cdouble2 (void);
+
+#define DOUBLE_ARG(NAME, OP)	ARG_OP(double, double_complex, NAME, OP)
+#define DOUBLE_PTR(NAME, OP)	PTR_OP(double, double_complex, NAME, OP)
+#define DOUBLE_CALL()		CALL_OP(double, double_complex, cdouble1, cdouble2)
+
+#else
+#define DOUBLE_ARG(NAME, OP)
+#define DOUBLE_PTR(NAME, OP)
+#define DOUBLE_CALL()
+#endif
+
+#ifndef NO_FLOAT128
+#ifdef __VSX__
+typedef _Complex float __attribute__((mode(KC)))	float128_complex;
+#else
+typedef _Complex float __attribute__((mode(TC)))	float128_complex;
+#endif
+
+extern float128_complex cfloat128_1 (void);
+extern float128_complex cfloat128_2 (void);
+
+#define FLOAT128_ARG(NAME, OP)	ARG_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_PTR(NAME, OP)	PTR_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_CALL()		CALL_OP(float128, float128_complex, cfloat128_1, cfloat128_2)
+
+#else
+#define FLOAT128_ARG(NAME, OP)
+#define FLOAT128_PTR(NAME, OP)
+#define FLOAT128_CALL()
+#endif
+
+#ifndef NO_LDOUBLE
+typedef _Complex long double ldouble_complex;
+extern ldouble_complex cldouble1 (void);
+extern ldouble_complex cldouble2 (void);
+
+#define LDOUBLE_ARG(NAME, OP)	ARG_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)	PTR_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_CALL()		CALL_OP(ldouble, ldouble_complex, cldouble1, cldouble2)
+
+#else
+#define LDOUBLE_ARG(NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)
+#define LDOUBLE_CALL()
+#endif
+
+
+#define ARG_OP(SUFFIX, TYPE, NAME, OP)					\
+TYPE arg_ ## NAME ## _ ## SUFFIX (TYPE a, TYPE b)			\
+{									\
+  return a OP b;							\
+}
+
+#define PTR_OP(SUFFIX, TYPE, NAME, OP)					\
+void ptr_ ## NAME ## _ ## SUFFIX (TYPE *p, TYPE *a, TYPE *b)		\
+{									\
+  *p = *a OP *b;							\
+}
+
+#define CALL_OP(SUFFIX, TYPE, FUNC1, FUNC2)				\
+TYPE call_ ## SUFFIX (void)						\
+{									\
+  TYPE value1 = FUNC1 ();						\
+  TYPE value2 = FUNC2 ();						\
+  return value1 + value2;						\
+}
+
+#ifndef NO_ARG
+#ifndef NO_ADD
+FLOAT_ARG    (add, +)
+DOUBLE_ARG   (add, +)
+FLOAT128_ARG (add, +)
+LDOUBLE_ARG  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_ARG    (sub, -)
+DOUBLE_ARG   (sub, -)
+FLOAT128_ARG (sub, -)
+LDOUBLE_ARG  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_ARG    (mul, *)
+DOUBLE_ARG   (mul, *)
+FLOAT128_ARG (mul, *)
+LDOUBLE_ARG  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_ARG    (div, /)
+DOUBLE_ARG   (div, /)
+FLOAT128_ARG (div, /)
+LDOUBLE_ARG  (div, /)
+#endif
+#endif
+
+#ifndef NO_PTR
+#ifndef NO_ADD
+FLOAT_PTR    (add, +)
+DOUBLE_PTR   (add, +)
+FLOAT128_PTR (add, +)
+LDOUBLE_PTR  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_PTR    (sub, -)
+DOUBLE_PTR   (sub, -)
+FLOAT128_PTR (sub, -)
+LDOUBLE_PTR  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_PTR    (mul, *)
+DOUBLE_PTR   (mul, *)
+FLOAT128_PTR (mul, *)
+LDOUBLE_PTR  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_PTR    (div, /)
+DOUBLE_PTR   (div, /)
+FLOAT128_PTR (div, /)
+LDOUBLE_PTR  (div, /)
+#endif
+#endif
+
+#ifndef NO_CALL
+FLOAT_CALL    ()
+DOUBLE_CALL   ()
+FLOAT128_CALL ()
+LDOUBLE_CALL  ()
+#endif
Index: gcc/testsuite/gcc.target/powerpc/float128-complex-2.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/float128-complex-2.c	(.../svn+ssh://meissner@gcc.gnu.org/svn/gcc/trunk/gcc/testsuite/gcc.target/powerpc)	(revision 0)
+++ gcc/testsuite/gcc.target/powerpc/float128-complex-2.c	(.../gcc/testsuite/gcc.target/powerpc)	(revision 235777)
@@ -0,0 +1,160 @@
+/* { dg-do compile { target { powerpc*-*-linux* } } } */
+/* { dg-require-effective-target powerpc_float128_hw_ok } */
+/* { dg-skip-if "do not override -mcpu" { powerpc*-*-* } { "-mcpu=*" } { "-mcpu=power9" } } */
+/* { dg-options "-O2 -mcpu=power9 -mfloat128 -mfloat128-hardware" } */
+
+#ifndef NO_FLOAT
+typedef _Complex float	float_complex;
+extern float_complex cfloat1 (void);
+extern float_complex cfloat2 (void);
+
+#define FLOAT_ARG(NAME, OP)	ARG_OP(float, float_complex, NAME, OP)
+#define FLOAT_PTR(NAME, OP)	PTR_OP(float, float_complex, NAME, OP)
+#define FLOAT_CALL()		CALL_OP(float, float_complex, cfloat1, cfloat2)
+
+#else
+#define FLOAT_ARG(NAME, OP)
+#define FLOAT_PTR(NAME, OP)
+#define FLOAT_CALL()
+#endif
+
+#ifndef NO_DOUBLE
+typedef _Complex double	double_complex;
+extern double_complex cdouble1 (void);
+extern double_complex cdouble2 (void);
+
+#define DOUBLE_ARG(NAME, OP)	ARG_OP(double, double_complex, NAME, OP)
+#define DOUBLE_PTR(NAME, OP)	PTR_OP(double, double_complex, NAME, OP)
+#define DOUBLE_CALL()		CALL_OP(double, double_complex, cdouble1, cdouble2)
+
+#else
+#define DOUBLE_ARG(NAME, OP)
+#define DOUBLE_PTR(NAME, OP)
+#define DOUBLE_CALL()
+#endif
+
+#ifndef NO_FLOAT128
+#ifdef __VSX__
+typedef _Complex float __attribute__((mode(KC)))	float128_complex;
+#else
+typedef _Complex float __attribute__((mode(TC)))	float128_complex;
+#endif
+
+extern float128_complex cfloat128_1 (void);
+extern float128_complex cfloat128_2 (void);
+
+#define FLOAT128_ARG(NAME, OP)	ARG_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_PTR(NAME, OP)	PTR_OP(float128, float128_complex, NAME, OP)
+#define FLOAT128_CALL()		CALL_OP(float128, float128_complex, cfloat128_1, cfloat128_2)
+
+#else
+#define FLOAT128_ARG(NAME, OP)
+#define FLOAT128_PTR(NAME, OP)
+#define FLOAT128_CALL()
+#endif
+
+#ifndef NO_LDOUBLE
+typedef _Complex long double ldouble_complex;
+extern ldouble_complex cldouble1 (void);
+extern ldouble_complex cldouble2 (void);
+
+#define LDOUBLE_ARG(NAME, OP)	ARG_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)	PTR_OP(ldouble, ldouble_complex, NAME, OP)
+#define LDOUBLE_CALL()		CALL_OP(ldouble, ldouble_complex, cldouble1, cldouble2)
+
+#else
+#define LDOUBLE_ARG(NAME, OP)
+#define LDOUBLE_PTR(NAME, OP)
+#define LDOUBLE_CALL()
+#endif
+
+
+#define ARG_OP(SUFFIX, TYPE, NAME, OP)					\
+TYPE arg_ ## NAME ## _ ## SUFFIX (TYPE a, TYPE b)			\
+{									\
+  return a OP b;							\
+}
+
+#define PTR_OP(SUFFIX, TYPE, NAME, OP)					\
+void ptr_ ## NAME ## _ ## SUFFIX (TYPE *p, TYPE *a, TYPE *b)		\
+{									\
+  *p = *a OP *b;							\
+}
+
+#define CALL_OP(SUFFIX, TYPE, FUNC1, FUNC2)				\
+TYPE call_ ## SUFFIX (void)						\
+{									\
+  TYPE value1 = FUNC1 ();						\
+  TYPE value2 = FUNC2 ();						\
+  return value1 + value2;						\
+}
+
+#ifndef NO_ARG
+#ifndef NO_ADD
+FLOAT_ARG    (add, +)
+DOUBLE_ARG   (add, +)
+FLOAT128_ARG (add, +)
+LDOUBLE_ARG  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_ARG    (sub, -)
+DOUBLE_ARG   (sub, -)
+FLOAT128_ARG (sub, -)
+LDOUBLE_ARG  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_ARG    (mul, *)
+DOUBLE_ARG   (mul, *)
+FLOAT128_ARG (mul, *)
+LDOUBLE_ARG  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_ARG    (div, /)
+DOUBLE_ARG   (div, /)
+FLOAT128_ARG (div, /)
+LDOUBLE_ARG  (div, /)
+#endif
+#endif
+
+#ifndef NO_PTR
+#ifndef NO_ADD
+FLOAT_PTR    (add, +)
+DOUBLE_PTR   (add, +)
+FLOAT128_PTR (add, +)
+LDOUBLE_PTR  (add, +)
+#endif
+
+#ifndef NO_SUB
+FLOAT_PTR    (sub, -)
+DOUBLE_PTR   (sub, -)
+FLOAT128_PTR (sub, -)
+LDOUBLE_PTR  (sub, -)
+#endif
+
+#ifndef NO_MUL
+FLOAT_PTR    (mul, *)
+DOUBLE_PTR   (mul, *)
+FLOAT128_PTR (mul, *)
+LDOUBLE_PTR  (mul, *)
+#endif
+
+#ifndef NO_DIV
+FLOAT_PTR    (div, /)
+DOUBLE_PTR   (div, /)
+FLOAT128_PTR (div, /)
+LDOUBLE_PTR  (div, /)
+#endif
+#endif
+
+#ifndef NO_CALL
+FLOAT_CALL    ()
+DOUBLE_CALL   ()
+FLOAT128_CALL ()
+LDOUBLE_CALL  ()
+#endif
+
+/* { dg-final { scan-assembler "xsaddqp"  } } */
+/* { dg-final { scan-assembler "xssubqp"  } } */