public inbox for gcc-patches@gcc.gnu.org
 help / color / mirror / Atom feed
* [og8] OpenACC 'kernels' construct changes: splitting of the construct into several regions
@ 2019-02-01  0:00 Thomas Schwinge
  2019-02-01 19:48 ` Thomas Schwinge
                   ` (2 more replies)
  0 siblings, 3 replies; 33+ messages in thread
From: Thomas Schwinge @ 2019-02-01  0:00 UTC (permalink / raw)
  To: gcc-patches; +Cc: Gergö Barany, Tom de Vries

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

Hi!

I've just pushed the attached nine patches to openacc-gcc-8-branch:
OpenACC 'kernels' construct changes: splitting of the construct into
several regions.

There's more work to be done there, and we're aware of a number of TODO
items, but nevertheless: it's a good first step.


(Tom, CCed you just for your information, as you've been working on the
OpenACC 'kernels' construct before.)


Grüße
 Thomas



[-- Attachment #2: 0001-Use-fopenacc-kernels-parloops-to-document-parloops-t.patch --]
[-- Type: text/x-diff, Size: 77711 bytes --]

From 07b7c5d9de8a3e77adca53a8eb3f4235903ea368 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Wed, 30 Jan 2019 10:32:10 +0100
Subject: [PATCH 1/9] Use "-fopenacc-kernels=parloops" to document "parloops"
 test cases

	gcc/
	* flag-types.h (enum openacc_kernels): New type.
	gcc/c-family/
	* c.opt (fopenacc-kernels): New flag.
	gcc/fortran/
	* lang.opt (fopenacc-kernels): New flag.
	gcc/testsuite/
	* c-c++-common/goacc/kernels-1.c: Add
	"-fopenacc-kernels=parloops".
	* c-c++-common/goacc/kernels-acc-loop-reduction.c: Likewise.
	* c-c++-common/goacc/kernels-acc-loop-smaller-equal.c: Likewise.
	* c-c++-common/goacc/kernels-alias-2.c: Likewise.
	* c-c++-common/goacc/kernels-alias-3.c: Likewise.
	* c-c++-common/goacc/kernels-alias-4.c: Likewise.
	* c-c++-common/goacc/kernels-alias-5.c: Likewise.
	* c-c++-common/goacc/kernels-alias-6.c: Likewise.
	* c-c++-common/goacc/kernels-alias-7.c: Likewise.
	* c-c++-common/goacc/kernels-alias-8.c: Likewise.
	* c-c++-common/goacc/kernels-alias-ipa-pta-2.c: Likewise.
	* c-c++-common/goacc/kernels-alias-ipa-pta-3.c: Likewise.
	* c-c++-common/goacc/kernels-alias-ipa-pta-4.c: Likewise.
	* c-c++-common/goacc/kernels-alias-ipa-pta.c: Likewise.
	* c-c++-common/goacc/kernels-alias.c: Likewise.
	* c-c++-common/goacc/kernels-counter-var-redundant-load.c:
	Likewise.
	* c-c++-common/goacc/kernels-counter-vars-function-scope.c:
	Likewise.
	* c-c++-common/goacc/kernels-double-reduction-n.c: Likewise.
	* c-c++-common/goacc/kernels-double-reduction.c: Likewise.
	* c-c++-common/goacc/kernels-loop-2-acc-loop.c: Likewise.
	* c-c++-common/goacc/kernels-loop-2.c: Likewise.
	* c-c++-common/goacc/kernels-loop-3-acc-loop.c: Likewise.
	* c-c++-common/goacc/kernels-loop-3.c: Likewise.
	* c-c++-common/goacc/kernels-loop-acc-loop.c: Likewise.
	* c-c++-common/goacc/kernels-loop-data-2.c: Likewise.
	* c-c++-common/goacc/kernels-loop-data-enter-exit-2.c: Likewise.
	* c-c++-common/goacc/kernels-loop-data-enter-exit.c: Likewise.
	* c-c++-common/goacc/kernels-loop-data-update.c: Likewise.
	* c-c++-common/goacc/kernels-loop-data.c: Likewise.
	* c-c++-common/goacc/kernels-loop-g.c: Likewise.
	* c-c++-common/goacc/kernels-loop-mod-not-zero.c: Likewise.
	* c-c++-common/goacc/kernels-loop-n-acc-loop.c: Likewise.
	* c-c++-common/goacc/kernels-loop-n.c: Likewise.
	* c-c++-common/goacc/kernels-loop-nest.c: Likewise.
	* c-c++-common/goacc/kernels-loop-offload-alias-none.c: Likewise.
	* c-c++-common/goacc/kernels-loop-offload-alias-ptr.c: Likewise.
	* c-c++-common/goacc/kernels-loop.c: Likewise.
	* c-c++-common/goacc/kernels-offload-alias-2.c: Likewise.
	* c-c++-common/goacc/kernels-offload-alias-3.c: Likewise.
	* c-c++-common/goacc/kernels-offload-alias-4.c: Likewise.
	* c-c++-common/goacc/kernels-offload-alias-5.c: Likewise.
	* c-c++-common/goacc/kernels-offload-alias-6.c: Likewise.
	* c-c++-common/goacc/kernels-offload-alias.c: Likewise.
	* c-c++-common/goacc/kernels-one-counter-var.c: Likewise.
	* c-c++-common/goacc/kernels-parallel-loop-data-enter-exit.c:
	Likewise.
	* c-c++-common/goacc/kernels-reduction.c: Likewise.
	* gcc.dg/goacc/kern-1.c: Likewise.
	* gfortran.dg/goacc/kernels-alias-2.f95: Likewise.
	* gfortran.dg/goacc/kernels-alias-3.f95: Likewise.
	* gfortran.dg/goacc/kernels-alias-4.f95: Likewise.
	* gfortran.dg/goacc/kernels-alias.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-2.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-data-2.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-data-enter-exit-2.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-data-enter-exit.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-data-update.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-data.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-inner.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop-n.f95: Likewise.
	* gfortran.dg/goacc/kernels-loop.f95: Likewise.
	* gfortran.dg/goacc/kernels-loops-adjacent.f95: Likewise.
	* gfortran.dg/goacc/kernels-parallel-loop-data-enter-exit.f95:
	Likewise.
	libgomp/
	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c:
	Add "-fopenacc-kernels=parloops".
	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-empty.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-2.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-3.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-2.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-3.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-4.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-5.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-6.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-collapse.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-2.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit-2.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-update.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-g.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-mod-not-zero.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-n.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-nest.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-loop.c: Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-parallel-loop-data-enter-exit.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-reduction-1.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-reduction.c:
	Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop-2.f95: Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop-data-2.f95:
	Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit-2.f95:
	Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit.f95:
	Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop-data-update.f95:
	Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop-data.f95: Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-loop.f95: Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-parallel-loop-data-enter-exit.f95:
	Likewise.
	* testsuite/libgomp.oacc-fortran/kernels-reduction-1.f90:
	Likewise.
---
 gcc/ChangeLog.openacc                         |  4 ++
 gcc/c-family/ChangeLog.openacc                |  4 ++
 gcc/c-family/c.opt                            |  9 +++
 gcc/flag-types.h                              |  6 ++
 gcc/fortran/ChangeLog.openacc                 |  4 ++
 gcc/fortran/lang.opt                          |  3 +
 gcc/testsuite/ChangeLog.openacc               | 68 +++++++++++++++++++
 gcc/testsuite/c-c++-common/goacc/kernels-1.c  |  2 +
 .../goacc/kernels-acc-loop-reduction.c        |  2 +
 .../goacc/kernels-acc-loop-smaller-equal.c    |  2 +
 .../c-c++-common/goacc/kernels-alias-2.c      |  2 +
 .../c-c++-common/goacc/kernels-alias-3.c      |  2 +
 .../c-c++-common/goacc/kernels-alias-4.c      |  2 +
 .../c-c++-common/goacc/kernels-alias-5.c      |  2 +
 .../c-c++-common/goacc/kernels-alias-6.c      |  2 +
 .../c-c++-common/goacc/kernels-alias-7.c      |  2 +
 .../c-c++-common/goacc/kernels-alias-8.c      |  2 +
 .../goacc/kernels-alias-ipa-pta-2.c           |  2 +
 .../goacc/kernels-alias-ipa-pta-3.c           |  2 +
 .../goacc/kernels-alias-ipa-pta-4.c           |  2 +
 .../goacc/kernels-alias-ipa-pta.c             |  2 +
 .../c-c++-common/goacc/kernels-alias.c        |  2 +
 .../kernels-counter-var-redundant-load.c      |  2 +
 .../kernels-counter-vars-function-scope.c     |  2 +
 .../goacc/kernels-double-reduction-n.c        |  2 +
 .../goacc/kernels-double-reduction.c          |  2 +
 .../goacc/kernels-loop-2-acc-loop.c           |  2 +
 .../c-c++-common/goacc/kernels-loop-2.c       |  2 +
 .../goacc/kernels-loop-3-acc-loop.c           |  2 +
 .../c-c++-common/goacc/kernels-loop-3.c       |  2 +
 .../goacc/kernels-loop-acc-loop.c             |  2 +
 .../c-c++-common/goacc/kernels-loop-data-2.c  |  2 +
 .../goacc/kernels-loop-data-enter-exit-2.c    |  2 +
 .../goacc/kernels-loop-data-enter-exit.c      |  2 +
 .../goacc/kernels-loop-data-update.c          |  2 +
 .../c-c++-common/goacc/kernels-loop-data.c    |  2 +
 .../c-c++-common/goacc/kernels-loop-g.c       |  2 +
 .../goacc/kernels-loop-mod-not-zero.c         |  2 +
 .../goacc/kernels-loop-n-acc-loop.c           |  2 +
 .../c-c++-common/goacc/kernels-loop-n.c       |  2 +
 .../c-c++-common/goacc/kernels-loop-nest.c    |  2 +
 .../goacc/kernels-loop-offload-alias-none.c   |  2 +
 .../goacc/kernels-loop-offload-alias-ptr.c    |  2 +
 .../c-c++-common/goacc/kernels-loop.c         |  2 +
 .../goacc/kernels-offload-alias-2.c           |  2 +
 .../goacc/kernels-offload-alias-3.c           |  2 +
 .../goacc/kernels-offload-alias-4.c           |  2 +
 .../goacc/kernels-offload-alias-5.c           |  2 +
 .../goacc/kernels-offload-alias-6.c           |  2 +
 .../goacc/kernels-offload-alias.c             |  2 +
 .../goacc/kernels-one-counter-var.c           |  2 +
 .../kernels-parallel-loop-data-enter-exit.c   |  2 +
 .../c-c++-common/goacc/kernels-reduction.c    |  2 +
 gcc/testsuite/gcc.dg/goacc/kern-1.c           |  2 +
 .../gfortran.dg/goacc/kernels-alias-2.f95     |  2 +
 .../gfortran.dg/goacc/kernels-alias-3.f95     |  2 +
 .../gfortran.dg/goacc/kernels-alias-4.f95     |  2 +
 .../gfortran.dg/goacc/kernels-alias.f95       |  2 +
 .../gfortran.dg/goacc/kernels-loop-2.f95      |  2 +
 .../gfortran.dg/goacc/kernels-loop-data-2.f95 |  2 +
 .../goacc/kernels-loop-data-enter-exit-2.f95  |  2 +
 .../goacc/kernels-loop-data-enter-exit.f95    |  2 +
 .../goacc/kernels-loop-data-update.f95        |  2 +
 .../gfortran.dg/goacc/kernels-loop-data.f95   |  2 +
 .../gfortran.dg/goacc/kernels-loop-inner.f95  |  2 +
 .../gfortran.dg/goacc/kernels-loop-n.f95      |  2 +
 .../gfortran.dg/goacc/kernels-loop.f95        |  2 +
 .../goacc/kernels-loops-adjacent.f95          |  2 +
 .../kernels-parallel-loop-data-enter-exit.f95 |  2 +
 libgomp/ChangeLog.openacc                     | 62 +++++++++++++++++
 .../kernels-alias-ipa-pta-2.c                 |  2 +
 .../kernels-alias-ipa-pta-3.c                 |  2 +
 .../kernels-alias-ipa-pta.c                   |  2 +
 .../libgomp.oacc-c-c++-common/kernels-empty.c |  2 +
 .../kernels-loop-2.c                          |  3 +
 .../kernels-loop-3.c                          |  3 +
 .../kernels-loop-and-seq-2.c                  |  2 +
 .../kernels-loop-and-seq-3.c                  |  3 +
 .../kernels-loop-and-seq-4.c                  |  3 +
 .../kernels-loop-and-seq-5.c                  |  2 +
 .../kernels-loop-and-seq-6.c                  |  2 +
 .../kernels-loop-and-seq.c                    |  2 +
 .../kernels-loop-collapse.c                   |  2 +
 .../kernels-loop-data-2.c                     |  3 +
 .../kernels-loop-data-enter-exit-2.c          |  3 +
 .../kernels-loop-data-enter-exit.c            |  3 +
 .../kernels-loop-data-update.c                |  3 +
 .../kernels-loop-data.c                       |  3 +
 .../kernels-loop-g.c                          |  2 +
 .../kernels-loop-mod-not-zero.c               |  3 +
 .../kernels-loop-n.c                          |  3 +
 .../kernels-loop-nest.c                       |  3 +
 .../libgomp.oacc-c-c++-common/kernels-loop.c  |  3 +
 .../kernels-parallel-loop-data-enter-exit.c   |  3 +
 .../kernels-reduction-1.c                     |  2 +
 .../kernels-reduction.c                       |  3 +
 .../libgomp.oacc-fortran/kernels-loop-2.f95   |  2 +
 .../kernels-loop-data-2.f95                   |  2 +
 .../kernels-loop-data-enter-exit-2.f95        |  2 +
 .../kernels-loop-data-enter-exit.f95          |  2 +
 .../kernels-loop-data-update.f95              |  2 +
 .../kernels-loop-data.f95                     |  2 +
 .../libgomp.oacc-fortran/kernels-loop.f95     |  2 +
 .../kernels-parallel-loop-data-enter-exit.f95 |  2 +
 .../kernels-reduction-1.f90                   |  2 +
 105 files changed, 369 insertions(+)

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 58258fd7a1d..9316130243c 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,7 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+
+	* flag-types.h (enum openacc_kernels): New type.
+
 2019-01-31  Kwok Cheung Yeung  <kcy@codesourcery.com>
 
 	* omp-low.c (lower_omp_target): For use_device clauses, generate
diff --git a/gcc/c-family/ChangeLog.openacc b/gcc/c-family/ChangeLog.openacc
index 08dc56107bb..5b60c3a0dee 100644
--- a/gcc/c-family/ChangeLog.openacc
+++ b/gcc/c-family/ChangeLog.openacc
@@ -1,3 +1,7 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+
+	* c.opt (fopenacc-kernels): New flag.
+
 2019-01-09  Julian Brown  <julian@codesourcery.com>
 
 	* c-cppbuiltin.c (c_cpp_builtins): Update _OPENACC define to 201711.
diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
index a4c8c8ffcb3..73b01598377 100644
--- a/gcc/c-family/c.opt
+++ b/gcc/c-family/c.opt
@@ -1617,6 +1617,15 @@ fopenacc-dim=
 C ObjC C++ ObjC++ LTO Joined Var(flag_openacc_dims)
 Specify default OpenACC compute dimensions.
 
+fopenacc-kernels=
+C ObjC C++ ObjC++ RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS) Undocumented
+
+Enum
+Name(openacc_kernels) Type(enum openacc_kernels)
+
+EnumValue
+Enum(openacc_kernels) String(parloops) Value(OPENACC_KERNELS_PARLOOPS)
+
 fopenmp
 C ObjC C++ ObjC++ LTO Var(flag_openmp)
 Enable OpenMP (implies -frecursive in Fortran).
diff --git a/gcc/flag-types.h b/gcc/flag-types.h
index 6261f6e1106..910be7c7fd4 100644
--- a/gcc/flag-types.h
+++ b/gcc/flag-types.h
@@ -343,4 +343,10 @@ enum cf_protection_level
   CF_FULL = CF_BRANCH | CF_RETURN,
   CF_SET = 1 << 2
 };
+
+/* OpenACC 'kernels' constructs handling.  */
+enum openacc_kernels
+{
+  OPENACC_KERNELS_PARLOOPS
+};
 #endif /* ! GCC_FLAG_TYPES_H */
diff --git a/gcc/fortran/ChangeLog.openacc b/gcc/fortran/ChangeLog.openacc
index 05462a0173c..acb2177f22f 100644
--- a/gcc/fortran/ChangeLog.openacc
+++ b/gcc/fortran/ChangeLog.openacc
@@ -1,3 +1,7 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+
+	* lang.opt (fopenacc-kernels): New flag.
+
 2019-01-29  Gergö Barany  <gergo@codesourcery.com>
 
 	* trans-openmp.c (gfc_privatize_nodesc_array_clauses): Renamed from
diff --git a/gcc/fortran/lang.opt b/gcc/fortran/lang.opt
index 1cb7b6b4f84..097c623eb50 100644
--- a/gcc/fortran/lang.opt
+++ b/gcc/fortran/lang.opt
@@ -642,6 +642,9 @@ fopenacc-dim=
 Fortran LTO Joined Var(flag_openacc_dims)
 ; Documented in C
 
+fopenacc-kernels=
+Fortran RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS) Undocumented
+
 fopenmp
 Fortran LTO
 ; Documented in C
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 2479367dce4..4acd174dca9 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,5 +1,73 @@
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
+	* c-c++-common/goacc/kernels-1.c: Add
+	"-fopenacc-kernels=parloops".
+	* c-c++-common/goacc/kernels-acc-loop-reduction.c: Likewise.
+	* c-c++-common/goacc/kernels-acc-loop-smaller-equal.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-2.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-3.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-4.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-5.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-6.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-7.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-8.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-ipa-pta-2.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-ipa-pta-3.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-ipa-pta-4.c: Likewise.
+	* c-c++-common/goacc/kernels-alias-ipa-pta.c: Likewise.
+	* c-c++-common/goacc/kernels-alias.c: Likewise.
+	* c-c++-common/goacc/kernels-counter-var-redundant-load.c:
+	Likewise.
+	* c-c++-common/goacc/kernels-counter-vars-function-scope.c:
+	Likewise.
+	* c-c++-common/goacc/kernels-double-reduction-n.c: Likewise.
+	* c-c++-common/goacc/kernels-double-reduction.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-2-acc-loop.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-2.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-3-acc-loop.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-3.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-acc-loop.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-data-2.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-data-enter-exit-2.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-data-enter-exit.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-data-update.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-data.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-g.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-mod-not-zero.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-n-acc-loop.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-n.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-nest.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-offload-alias-none.c: Likewise.
+	* c-c++-common/goacc/kernels-loop-offload-alias-ptr.c: Likewise.
+	* c-c++-common/goacc/kernels-loop.c: Likewise.
+	* c-c++-common/goacc/kernels-offload-alias-2.c: Likewise.
+	* c-c++-common/goacc/kernels-offload-alias-3.c: Likewise.
+	* c-c++-common/goacc/kernels-offload-alias-4.c: Likewise.
+	* c-c++-common/goacc/kernels-offload-alias-5.c: Likewise.
+	* c-c++-common/goacc/kernels-offload-alias-6.c: Likewise.
+	* c-c++-common/goacc/kernels-offload-alias.c: Likewise.
+	* c-c++-common/goacc/kernels-one-counter-var.c: Likewise.
+	* c-c++-common/goacc/kernels-parallel-loop-data-enter-exit.c:
+	Likewise.
+	* c-c++-common/goacc/kernels-reduction.c: Likewise.
+	* gcc.dg/goacc/kern-1.c: Likewise.
+	* gfortran.dg/goacc/kernels-alias-2.f95: Likewise.
+	* gfortran.dg/goacc/kernels-alias-3.f95: Likewise.
+	* gfortran.dg/goacc/kernels-alias-4.f95: Likewise.
+	* gfortran.dg/goacc/kernels-alias.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-2.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-data-2.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-data-enter-exit-2.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-data-enter-exit.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-data-update.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-data.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-inner.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop-n.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loop.f95: Likewise.
+	* gfortran.dg/goacc/kernels-loops-adjacent.f95: Likewise.
+	* gfortran.dg/goacc/kernels-parallel-loop-data-enter-exit.f95:
+	Likewise.
+
 	* lib/target-supports.exp
 	(check_effective_target_opt_levels_2_plus)
 	(check_effective_target_opt_levels_size): New.
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-1.c b/gcc/testsuite/c-c++-common/goacc/kernels-1.c
index 0a4bd854611..8512f349aeb 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-1.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-1.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-fopt-info-optimized-omp" } */
 
 int
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-reduction.c b/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-reduction.c
index 4824e530925..19739ade492 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-reduction.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-reduction.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-smaller-equal.c b/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-smaller-equal.c
index d70afb0e662..0dd95b32ed8 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-smaller-equal.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-acc-loop-smaller-equal.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-2.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-2.c
index d437c47779d..99682498037 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-3.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-3.c
index b051481cdfd..fc492a27e95 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-3.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-3.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-4.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-4.c
index 1f626750495..7fa84e5f0eb 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-4.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-4.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-5.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-5.c
index ff0044df683..19b2a6705a3 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-5.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-5.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-6.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-6.c
index 861dc2c75b4..f4205fb6d3c 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-6.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-6.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-7.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-7.c
index d39128eec90..9356f436e8e 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-7.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-7.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-8.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-8.c
index 0896b732235..5f451a56846 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-8.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-8.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-2.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-2.c
index f16d698af0d..7a57477ca25 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fipa-pta -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-3.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-3.c
index e177abfabed..31ba223aacc 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-3.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-3.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fipa-pta -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-4.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-4.c
index 20b21dcb577..41f901585cf 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-4.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta-4.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fipa-pta -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta.c
index 969b466e8a8..e587f96efe8 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias-ipa-pta.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fipa-pta -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-alias.c b/gcc/testsuite/c-c++-common/goacc/kernels-alias.c
index 25821ab2aea..21285839dad 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-alias.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-alias.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-ealias-all" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-counter-var-redundant-load.c b/gcc/testsuite/c-c++-common/goacc/kernels-counter-var-redundant-load.c
index 1fd355b487c..642040f2c83 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-counter-var-redundant-load.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-counter-var-redundant-load.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=all" } */
 /* { dg-additional-options "-fdump-tree-dom3" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-counter-vars-function-scope.c b/gcc/testsuite/c-c++-common/goacc/kernels-counter-vars-function-scope.c
index c475333f1ae..f40de679fe7 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-counter-vars-function-scope.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-counter-vars-function-scope.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction-n.c b/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction-n.c
index dd3b7c8b144..4c1984ecbc7 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction-n.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction-n.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fopt-info-optimized-omp" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction.c b/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction.c
index 0175434a20c..1da2af4dddc 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-double-reduction.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fopt-info-optimized-omp" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-2-acc-loop.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-2-acc-loop.c
index 7b127cb6fd9..c6e66fcfc93 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-2-acc-loop.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-2-acc-loop.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-2.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-2.c
index acef6a1a179..238956c51ef 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-3-acc-loop.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-3-acc-loop.c
index a040e096fc1..4d041492a05 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-3-acc-loop.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-3-acc-loop.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-3.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-3.c
index 75e2bb78cea..2bbb0711abe 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-3.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-3.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-acc-loop.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-acc-loop.c
index 070a5b5bf3d..54b5946d85e 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-acc-loop.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-acc-loop.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-2.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-2.c
index 71800217991..e7830b6b80a 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit-2.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit-2.c
index 0c9f8331240..b5c26705338 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit.c
index 0bd21b68d31..84f92a901ab 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-enter-exit.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-update.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-update.c
index dd5a84146a8..dbdce4547c1 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-update.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data-update.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data.c
index a658182de90..23f4e221f54 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-data.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-data.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-g.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-g.c
index 73b469d7061..2cbd6becfbf 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-g.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-g.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-g" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-mod-not-zero.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-mod-not-zero.c
index 55926230d57..28480aead92 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-mod-not-zero.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-mod-not-zero.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-n-acc-loop.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-n-acc-loop.c
index 1f25e63fbbb..164c3949789 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-n-acc-loop.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-n-acc-loop.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-n.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-n.c
index e86be1b1cdc..26bc3e011bf 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-n.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-n.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-nest.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-nest.c
index 2b0e186ae29..b3fdde197b6 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-nest.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-nest.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-none.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-none.c
index db5f94c96dd..288c4dedde4 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-none.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-none.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
 /* { dg-additional-options "-fdump-tree-alias-all" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-ptr.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-ptr.c
index 30ad2c8ee9a..4256656932d 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-ptr.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop-offload-alias-ptr.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
 /* { dg-additional-options "-fdump-tree-alias-all" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-loop.c b/gcc/testsuite/c-c++-common/goacc/kernels-loop.c
index 9619d53b43d..d0423a88ed1 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-loop.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-loop.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-2.c b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-2.c
index 554c2b72667..a7d6a6baa0f 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=pointer" } */
 /* { dg-additional-options "-fdump-tree-ealias-all -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-3.c b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-3.c
index 7236be95635..de8ffd9b0f1 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-3.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-3.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=pointer" } */
 /* { dg-additional-options "-fdump-tree-ealias-all -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-4.c b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-4.c
index 797575bab7b..9901cc8d9cf 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-4.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-4.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=pointer" } */
 /* { dg-additional-options "-fdump-tree-ealias-all -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-5.c b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-5.c
index 746494c4b61..30d95ebf371 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-5.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-5.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=pointer" } */
 /* { dg-additional-options "-fdump-tree-ealias-all -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-6.c b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-6.c
index 10d2897fb3d..f1df43f9037 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-6.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias-6.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=pointer" } */
 /* { dg-additional-options "-fdump-tree-ealias-all -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias.c b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias.c
index 65095b72d90..bdaff2ef7b0 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-offload-alias.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2 -foffload-alias=pointer" } */
 /* { dg-additional-options "-fdump-tree-ealias-all -fdump-tree-optimized" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-one-counter-var.c b/gcc/testsuite/c-c++-common/goacc/kernels-one-counter-var.c
index 69539b24a78..15a8d3732fc 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-one-counter-var.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-one-counter-var.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-parallel-loop-data-enter-exit.c b/gcc/testsuite/c-c++-common/goacc/kernels-parallel-loop-data-enter-exit.c
index 81b0fee5a44..457a79abf79 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-parallel-loop-data-enter-exit.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-parallel-loop-data-enter-exit.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-reduction.c b/gcc/testsuite/c-c++-common/goacc/kernels-reduction.c
index 5921b88920f..76039888b55 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-reduction.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-reduction.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 /* { dg-additional-options "-fdump-tree-parloops1-all" } */
 /* { dg-additional-options "-fdump-tree-optimized" } */
diff --git a/gcc/testsuite/gcc.dg/goacc/kern-1.c b/gcc/testsuite/gcc.dg/goacc/kern-1.c
index c48e826e306..948eb507825 100644
--- a/gcc/testsuite/gcc.dg/goacc/kern-1.c
+++ b/gcc/testsuite/gcc.dg/goacc/kern-1.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-O2" } */
 
 /* The reduction on sum could cause an ICE with a non-simple latch loop.   */
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-alias-2.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-alias-2.f95
index 7e348dde2bd..2ca68a1a8e5 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-alias-2.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-alias-2.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-ealias-all" }
 
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-alias-3.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-alias-3.f95
index 36b06d34e58..21f9ea58e63 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-alias-3.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-alias-3.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-foffload-alias=all" }
 ! { dg-additional-options "-fdump-tree-ealias-all" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-alias-4.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-alias-4.f95
index e41da824550..da0c5311c6a 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-alias-4.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-alias-4.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-foffload-alias=all" }
 ! { dg-additional-options "-fdump-tree-ealias-all" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-alias.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-alias.f95
index 8d6ccb338b9..f1a2ece47b9 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-alias.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-alias.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-ealias-all" }
 
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-2.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-2.f95
index ef53324dd2a..59001e4734d 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-2.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-2.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-2.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-2.f95
index 2f1dcd603a1..b6f50cbd2d6 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-2.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-2.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit-2.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit-2.f95
index 447e85d6448..779073a1f3b 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit-2.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit-2.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit.f95
index 4edb2889b7b..30ae2cb0f2a 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-enter-exit.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-update.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-update.f95
index fc113e1f660..b68945a91ac 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-update.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data-update.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data.f95
index 94522f58636..f5c6688a05b 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-data.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-inner.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-inner.f95
index 16a64e6d76d..b93c89ab9c1 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-inner.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-inner.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fopt-info-optimized-omp" }
 
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-n.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-n.f95
index aaef42974a6..1bb363ce880 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop-n.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop-n.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loop.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loop.f95
index 6dc7b2e0f28..4da70409c8f 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loop.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loop.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-loops-adjacent.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-loops-adjacent.f95
index fb92da8c08b..a83ff951457 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-loops-adjacent.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-loops-adjacent.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 
 program main
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-parallel-loop-data-enter-exit.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-parallel-loop-data-enter-exit.f95
index 16c9b80b295..260b11446ad 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-parallel-loop-data-enter-exit.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-parallel-loop-data-enter-exit.f95
@@ -1,3 +1,5 @@
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! { dg-additional-options "-O2" }
 ! { dg-additional-options "-fdump-tree-parloops1-all" }
 ! { dg-additional-options "-fdump-tree-optimized" }
diff --git a/libgomp/ChangeLog.openacc b/libgomp/ChangeLog.openacc
index edb7d3f76eb..bf572ee8922 100644
--- a/libgomp/ChangeLog.openacc
+++ b/libgomp/ChangeLog.openacc
@@ -1,5 +1,67 @@
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
+	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c:
+	Add "-fopenacc-kernels=parloops".
+	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-empty.c: Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-2.c: Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-3.c: Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-2.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-3.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-4.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-5.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-6.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-collapse.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-2.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit-2.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-update.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-data.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-g.c: Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-mod-not-zero.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-n.c: Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop-nest.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-loop.c: Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-parallel-loop-data-enter-exit.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-reduction-1.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-reduction.c:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop-2.f95: Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop-data-2.f95:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit-2.f95:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit.f95:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop-data-update.f95:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop-data.f95: Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-loop.f95: Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-parallel-loop-data-enter-exit.f95:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/kernels-reduction-1.f90:
+	Likewise.
+
 	* testsuite/libgomp.oacc-c-c++-common/lib-43.c: Remove.
 	* testsuite/libgomp.oacc-c-c++-common/lib-47.c: Likewise.
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c
index 0a280d58d8d..1bb2da110b0 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-fipa-pta" } */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c
index 6a50e626c02..42e27fb4aa9 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-foffload-alias=all -fipa-pta" } */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta.c
index 2a89cd3a4a1..38f34810105 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-fipa-pta" } */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-empty.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-empty.c
index d527e1497d7..cb1dd42afbc 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-empty.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-empty.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-2.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-2.c
index b84088879c6..4aeeed1f674 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-2.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-2.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-3.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-3.c
index 31114ac86d7..9cbace156a1 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-3.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-3.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-2.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-2.c
index 6a85bf5b50f..fe344903904 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-2.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-2.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-3.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-3.c
index e62297129fd..d53e39304a0 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-3.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-3.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N 32
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-4.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-4.c
index c73127897a9..7435c854172 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-4.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-4.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N 32
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-5.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-5.c
index 98a017fc55d..58c6402b3a7 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-5.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-5.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-6.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-6.c
index 131aa40b2c8..6361001b358 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-6.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq-6.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq.c
index 377f9dcde76..41984e3d354 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-and-seq.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-collapse.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-collapse.c
index bdc10dbe5bd..97658c0b8c5 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-collapse.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-collapse.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-2.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-2.c
index 607c35018df..337ad91507f 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-2.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-2.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit-2.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit-2.c
index 8b9dd5f815a..214dd7e3e69 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit-2.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit-2.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit.c
index 5d5da6fcc01..7d097dadb62 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-enter-exit.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-update.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-update.c
index c111c8f56e7..661cb286a0d 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-update.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data-update.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data.c
index 947bcdac452..2f4f699195d 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-data.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-g.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-g.c
index 88258be16bd..e5a556b36b7 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-g.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-g.c
@@ -1,3 +1,5 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* { dg-additional-options "-g" } */
 
 #include "kernels-loop.c"
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-mod-not-zero.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-mod-not-zero.c
index 147ebb59945..eeb318e7bda 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-mod-not-zero.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-mod-not-zero.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N ((1024 * 512) + 1)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-n.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-n.c
index 9a3eaca1380..eeccc1ddb06 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-n.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-n.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N ((1024 * 512) + 1)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-nest.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-nest.c
index 28c725a61a2..c59c47ecd4a 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-nest.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop-nest.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N 1000
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop.c
index 355123c6088..36eabb959ca 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-loop.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-parallel-loop-data-enter-exit.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-parallel-loop-data-enter-exit.c
index 374014a1e86..eeb08106dc8 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-parallel-loop-data-enter-exit.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-parallel-loop-data-enter-exit.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define N (1024 * 512)
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction-1.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction-1.c
index 23ed7e9b801..d988d9f4277 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction-1.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction-1.c
@@ -1,6 +1,8 @@
 /* Verify that a simple, explicit acc loop reduction works inside
  a kernels region.  */
 
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction.c
index 8647a9432fc..e67340cca20 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-reduction.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+   specifically testing "parloops" handling.  */
+
 #include <stdlib.h>
 
 #define n 10000
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-2.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-2.f95
index 8becc159dd1..14055a36284 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-2.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-2.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-2.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-2.f95
index 2191ebedee3..ec3e74a6c51 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-2.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-2.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit-2.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit-2.f95
index 75fb8a32beb..f6e6e21ef24 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit-2.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit-2.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit.f95
index 8ea34bf7bf8..b6b3a7aa3da 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-enter-exit.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-update.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-update.f95
index 710068a707a..686dfddcf3c 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-update.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data-update.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data.f95
index c1dec2c89c5..b2aa0e7f3db 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop-data.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop.f95
index c9d3c4adc95..96cf48b14f1 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-loop.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-parallel-loop-data-enter-exit.f95 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-parallel-loop-data-enter-exit.f95
index 99300ec88b1..ea752c348dc 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-parallel-loop-data-enter-exit.f95
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-parallel-loop-data-enter-exit.f95
@@ -1,4 +1,6 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 
 program main
   implicit none
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/kernels-reduction-1.f90 b/libgomp/testsuite/libgomp.oacc-fortran/kernels-reduction-1.f90
index a25e8a80f05..a2cdb2a79e8 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/kernels-reduction-1.f90
+++ b/libgomp/testsuite/libgomp.oacc-fortran/kernels-reduction-1.f90
@@ -1,6 +1,8 @@
 ! Test a simple acc loop reduction inside a kernels region. 
 
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=parloops" } as this is
+! specifically testing "parloops" handling.
 ! Override the compiler's "avoid offloading" decision.
 ! { dg-additional-options "-foffload-force" }
 
-- 
2.17.1


[-- Attachment #3: 0002-Add-OpenACC-target-kinds-for-decomposed-kernels-regi.patch --]
[-- Type: text/x-diff, Size: 16731 bytes --]

From 60cee29a41b1c42b8a621d436b0c843447ce4dfa Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Wed, 23 Jan 2019 06:56:52 -0800
Subject: [PATCH 2/9] Add OpenACC target kinds for decomposed kernels regions

This patch is in preparation for changes that will cut up OpenACC kernels
regions into individual parts. For the new sub-regions that will be
generated, this adds the following new kinds of OpenACC regions for internal
use:

- GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED for parts of kernels
  regions to be executed in gang-redundant mode
- GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE for parts of kernels
  regions to be executed in gang-single mode
- GF_OMP_TARGET_KIND_OACC_DATA_KERNELS for data regions generated around the
  body of a kernels region

    gcc/
    * gimple.h (enum gf_mask): Add new target kinds
    GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED,
    GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE, and
    GF_OMP_TARGET_KIND_OACC_DATA_KERNELS.
    (is_gimple_omp_oacc): Handle new target kinds.
    (is_gimple_omp_offloaded): Likewise.
    * gimple-pretty-print.c (dump_gimple_omp_target): Likewise.
    * omp-expand.c (expand_omp_target): Likewise.
    (build_omp_regions_1): Likewise.
    (omp_make_gimple_edges): Likewise.
    * omp-low.c (is_oacc_parallel_or_serial): Likewise.
    (was_originally_oacc_kernels): New function.
    (scan_omp_for): Update check for illegal nesting.
    (check_omp_nesting_restrictions): Handle new target kinds.
    (lower_oacc_reductions): Likewise.
    (lower_omp_target): Likewise.
    * omp-offload.c (execute_oacc_device_lower): Likewise.
---
 gcc/ChangeLog.openacc     | 21 ++++++++++++++++++
 gcc/gimple-pretty-print.c |  9 ++++++++
 gcc/gimple.h              | 14 ++++++++++++
 gcc/omp-expand.c          | 31 +++++++++++++++++++++-----
 gcc/omp-low.c             | 46 ++++++++++++++++++++++++++++++++++++---
 gcc/omp-offload.c         | 20 +++++++++++++++++
 6 files changed, 132 insertions(+), 9 deletions(-)

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 9316130243c..80f7efe18b1 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,24 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* gimple.h (enum gf_mask): Add new target kinds
+	GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED,
+	GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE, and
+	GF_OMP_TARGET_KIND_OACC_DATA_KERNELS.
+	(is_gimple_omp_oacc): Handle new target kinds.
+	(is_gimple_omp_offloaded): Likewise.
+	* gimple-pretty-print.c (dump_gimple_omp_target): Likewise.
+	* omp-expand.c (expand_omp_target): Likewise.
+	(build_omp_regions_1): Likewise.
+	(omp_make_gimple_edges): Likewise.
+	* omp-low.c (is_oacc_parallel_or_serial): Likewise.
+	(was_originally_oacc_kernels): New function.
+	(scan_omp_for): Update check for illegal nesting.
+	(check_omp_nesting_restrictions): Handle new target kinds.
+	(lower_oacc_reductions): Likewise.
+	(lower_omp_target): Likewise.
+	* omp-offload.c (execute_oacc_device_lower): Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
 	* flag-types.h (enum openacc_kernels): New type.
diff --git a/gcc/gimple-pretty-print.c b/gcc/gimple-pretty-print.c
index d2e3fddd7d8..cc6ee1860fc 100644
--- a/gcc/gimple-pretty-print.c
+++ b/gcc/gimple-pretty-print.c
@@ -1623,6 +1623,15 @@ dump_gimple_omp_target (pretty_printer *buffer, gomp_target *gs,
     case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
       kind = " oacc_host_data";
       break;
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+      kind = " oacc_parallel_kernels_parallelized";
+      break;
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+      kind = " oacc_parallel_kernels_gang_single";
+      break;
+    case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
+      kind = " oacc_data_kernels";
+      break;
     default:
       gcc_unreachable ();
     }
diff --git a/gcc/gimple.h b/gcc/gimple.h
index 072b09b1fad..6d126aab3fd 100644
--- a/gcc/gimple.h
+++ b/gcc/gimple.h
@@ -184,6 +184,15 @@ enum gf_mask {
     GF_OMP_TARGET_KIND_OACC_DECLARE = 10,
     GF_OMP_TARGET_KIND_OACC_HOST_DATA = 11,
     GF_OMP_TARGET_KIND_OACC_SERIAL = 12,
+    /* A GF_OMP_TARGET_KIND_OACC_PARALLEL that originates from a 'kernels'
+       construct, parallelized.  */
+    GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED = 13,
+    /* A GF_OMP_TARGET_KIND_OACC_PARALLEL that originates from a 'kernels'
+       construct, "gang-single".  */
+    GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE = 14,
+    /* A GF_OMP_TARGET_KIND_OACC_DATA that originates from a 'kernels'
+       construct.  */
+    GF_OMP_TARGET_KIND_OACC_DATA_KERNELS = 15,
     GF_OMP_TEAMS_GRID_PHONY	= 1 << 0,
 
     /* True on an GIMPLE_OMP_RETURN statement if the return does not require
@@ -6306,6 +6315,9 @@ is_gimple_omp_oacc (const gimple *stmt)
 	case GF_OMP_TARGET_KIND_OACC_ENTER_EXIT_DATA:
 	case GF_OMP_TARGET_KIND_OACC_DECLARE:
 	case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
+	case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+	case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+	case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
 	  return true;
 	default:
 	  return false;
@@ -6331,6 +6343,8 @@ is_gimple_omp_offloaded (const gimple *stmt)
 	case GF_OMP_TARGET_KIND_OACC_PARALLEL:
 	case GF_OMP_TARGET_KIND_OACC_KERNELS:
 	case GF_OMP_TARGET_KIND_OACC_SERIAL:
+	case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+	case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
 	  return true;
 	default:
 	  return false;
diff --git a/gcc/omp-expand.c b/gcc/omp-expand.c
index 42c491099fc..9de47de8a84 100644
--- a/gcc/omp-expand.c
+++ b/gcc/omp-expand.c
@@ -6960,6 +6960,8 @@ expand_omp_target (struct omp_region *region)
     {
     case GF_OMP_TARGET_KIND_OACC_PARALLEL:
     case GF_OMP_TARGET_KIND_OACC_SERIAL:
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
       oacc_parallel = true;
       gcc_fallthrough ();
     case GF_OMP_TARGET_KIND_REGION:
@@ -6975,6 +6977,7 @@ expand_omp_target (struct omp_region *region)
     case GF_OMP_TARGET_KIND_DATA:
     case GF_OMP_TARGET_KIND_OACC_DATA:
     case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
+    case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
       data_region = true;
       break;
     default:
@@ -6997,26 +7000,33 @@ expand_omp_target (struct omp_region *region)
   entry_bb = region->entry;
   exit_bb = region->exit;
 
+  /* Further down, all OpenACC compute constructs will be mapped to
+     BUILT_IN_GOACC_PARALLEL, and to distinguish between them, we now attach
+     attributes.  */
   switch (gimple_omp_target_kind (entry_stmt))
     {
     case GF_OMP_TARGET_KIND_OACC_KERNELS:
       mark_loops_in_oacc_kernels_region (region->entry, region->exit);
 
-      /* Further down, all OpenACC compute constructs will be mapped to
-	 BUILT_IN_GOACC_PARALLEL, and to distinguish between them, there
-	 is an "oacc kernels" attribute set for OpenACC kernels.  */
       DECL_ATTRIBUTES (child_fn)
 	= tree_cons (get_identifier ("oacc kernels"),
 		     NULL_TREE, DECL_ATTRIBUTES (child_fn));
       break;
     case GF_OMP_TARGET_KIND_OACC_SERIAL:
-      /* Further down, all OpenACC compute constructs will be mapped to
-	 BUILT_IN_GOACC_PARALLEL, and to distinguish between them, there
-	 is an "oacc serial" attribute set for OpenACC serial.  */
       DECL_ATTRIBUTES (child_fn)
 	= tree_cons (get_identifier ("oacc serial"),
 		     NULL_TREE, DECL_ATTRIBUTES (child_fn));
       break;
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+      DECL_ATTRIBUTES (child_fn)
+	= tree_cons (get_identifier ("oacc parallel_kernels_parallelized"),
+		     NULL_TREE, DECL_ATTRIBUTES (child_fn));
+      break;
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+      DECL_ATTRIBUTES (child_fn)
+	= tree_cons (get_identifier ("oacc parallel_kernels_gang_single"),
+		     NULL_TREE, DECL_ATTRIBUTES (child_fn));
+      break;
     default:
       break;
     }
@@ -7228,10 +7238,13 @@ expand_omp_target (struct omp_region *region)
     case GF_OMP_TARGET_KIND_OACC_KERNELS:
     case GF_OMP_TARGET_KIND_OACC_PARALLEL:
     case GF_OMP_TARGET_KIND_OACC_SERIAL:
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
       start_ix = BUILT_IN_GOACC_PARALLEL;
       break;
     case GF_OMP_TARGET_KIND_OACC_DATA:
     case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
+    case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
       start_ix = BUILT_IN_GOACC_DATA_START;
       break;
     case GF_OMP_TARGET_KIND_OACC_UPDATE:
@@ -8072,6 +8085,9 @@ build_omp_regions_1 (basic_block bb, struct omp_region *parent,
 		case GF_OMP_TARGET_KIND_OACC_SERIAL:
 		case GF_OMP_TARGET_KIND_OACC_DATA:
 		case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
+		case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+		case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+		case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
 		  if (is_gimple_omp_oacc (stmt))
 		    region->kind = gimple_omp_target_kind (stmt);
 		  break;
@@ -8321,6 +8337,9 @@ omp_make_gimple_edges (basic_block bb, struct omp_region **region,
 	case GF_OMP_TARGET_KIND_OACC_SERIAL:
 	case GF_OMP_TARGET_KIND_OACC_DATA:
 	case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
+	case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+	case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+	case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
 	  break;
 	case GF_OMP_TARGET_KIND_UPDATE:
 	case GF_OMP_TARGET_KIND_ENTER_DATA:
diff --git a/gcc/omp-low.c b/gcc/omp-low.c
index 3ae39c33c9d..636a4a00307 100644
--- a/gcc/omp-low.c
+++ b/gcc/omp-low.c
@@ -166,7 +166,11 @@ is_oacc_parallel_or_serial (omp_context *ctx)
 	  && ((gimple_omp_target_kind (ctx->stmt)
 	       == GF_OMP_TARGET_KIND_OACC_PARALLEL)
 	      || (gimple_omp_target_kind (ctx->stmt)
-		  == GF_OMP_TARGET_KIND_OACC_SERIAL)));
+		  == GF_OMP_TARGET_KIND_OACC_SERIAL)
+	      || (gimple_omp_target_kind (ctx->stmt)
+		  == GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED)
+	      || (gimple_omp_target_kind (ctx->stmt)
+		  == GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE)));
 }
 
 /* Return true if CTX corresponds to an oacc kernels region.  */
@@ -180,6 +184,22 @@ is_oacc_kernels (omp_context *ctx)
 	      == GF_OMP_TARGET_KIND_OACC_KERNELS));
 }
 
+/* Return true if CTX corresponds to an oacc region that was generated from
+   an original kernels region that has been lowered to parallel regions.  */
+
+static bool
+was_originally_oacc_kernels (omp_context *ctx)
+{
+  enum gimple_code outer_type = gimple_code (ctx->stmt);
+  return ((outer_type == GIMPLE_OMP_TARGET)
+	  && ((gimple_omp_target_kind (ctx->stmt)
+	       == GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED)
+	      || (gimple_omp_target_kind (ctx->stmt)
+		  == GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE)
+	      || (gimple_omp_target_kind (ctx->stmt)
+		  == GF_OMP_TARGET_KIND_OACC_DATA_KERNELS)));
+}
+
 /* If DECL is the artificial dummy VAR_DECL created for non-static
    data member privatization, return the underlying "this" parameter,
    otherwise return NULL.  */
@@ -2421,7 +2441,8 @@ scan_omp_for (gomp_for *stmt, omp_context *outer_ctx)
     {
       omp_context *tgt = enclosing_target_ctx (outer_ctx);
 
-      if (!tgt || is_oacc_parallel_or_serial (tgt))
+      if (!tgt || (is_oacc_parallel_or_serial (tgt)
+                    && !was_originally_oacc_kernels (tgt)))
 	for (tree c = clauses; c; c = OMP_CLAUSE_CHAIN (c))
 	  {
 	    char const *check = NULL;
@@ -2908,6 +2929,8 @@ check_omp_nesting_restrictions (gimple *stmt, omp_context *ctx)
 		  case GF_OMP_TARGET_KIND_OACC_PARALLEL:
 		  case GF_OMP_TARGET_KIND_OACC_KERNELS:
 		  case GF_OMP_TARGET_KIND_OACC_SERIAL:
+		  case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+		  case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
 		    ok = true;
 		    break;
 
@@ -3331,6 +3354,11 @@ check_omp_nesting_restrictions (gimple *stmt, omp_context *ctx)
 	      stmt_name = "enter/exit data"; break;
 	    case GF_OMP_TARGET_KIND_OACC_HOST_DATA: stmt_name = "host_data";
 	      break;
+	    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+	    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+	    case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
+	      /* These three cases arise from kernels conversion.  */
+	      stmt_name = "kernels"; break;
 	    default: gcc_unreachable ();
 	    }
 	  switch (gimple_omp_target_kind (ctx->stmt))
@@ -3346,6 +3374,11 @@ check_omp_nesting_restrictions (gimple *stmt, omp_context *ctx)
 	    case GF_OMP_TARGET_KIND_OACC_DATA: ctx_stmt_name = "data"; break;
 	    case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
 	      ctx_stmt_name = "host_data"; break;
+	    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+	    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
+	    case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
+	      /* These three cases arise from kernels conversion.  */
+	      ctx_stmt_name = "kernels"; break;
 	    default: gcc_unreachable ();
 	    }
 
@@ -5374,7 +5407,11 @@ lower_oacc_reductions (location_t loc, tree clauses, tree level, bool inner,
 		    if ((gimple_omp_target_kind (probe->stmt)
 			 != GF_OMP_TARGET_KIND_OACC_PARALLEL)
 			&& (gimple_omp_target_kind (probe->stmt)
-			    != GF_OMP_TARGET_KIND_OACC_SERIAL))
+			    != GF_OMP_TARGET_KIND_OACC_SERIAL)
+			&& (gimple_omp_target_kind (probe->stmt)
+			    != GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED)
+			&& (gimple_omp_target_kind (probe->stmt)
+			    != GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE))
 		      goto do_lookup;
 
 		    cls = gimple_omp_target_clauses (probe->stmt);
@@ -8149,11 +8186,14 @@ lower_omp_target (gimple_stmt_iterator *gsi_p, omp_context *ctx)
     case GF_OMP_TARGET_KIND_OACC_UPDATE:
     case GF_OMP_TARGET_KIND_OACC_ENTER_EXIT_DATA:
     case GF_OMP_TARGET_KIND_OACC_DECLARE:
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED:
+    case GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE:
       data_region = false;
       break;
     case GF_OMP_TARGET_KIND_DATA:
     case GF_OMP_TARGET_KIND_OACC_DATA:
     case GF_OMP_TARGET_KIND_OACC_HOST_DATA:
+    case GF_OMP_TARGET_KIND_OACC_DATA_KERNELS:
       data_region = true;
       break;
     default:
diff --git a/gcc/omp-offload.c b/gcc/omp-offload.c
index 02b366ed4f7..2d265c22c3c 100644
--- a/gcc/omp-offload.c
+++ b/gcc/omp-offload.c
@@ -1604,6 +1604,20 @@ execute_oacc_device_lower ()
   bool is_oacc_kernels_parallelized
     = (lookup_attribute ("oacc kernels parallelized",
 			 DECL_ATTRIBUTES (current_function_decl)) != NULL);
+  if (is_oacc_kernels_parallelized)
+    gcc_checking_assert (is_oacc_kernels);
+  bool is_oacc_parallel_kernels_parallelized
+    = (lookup_attribute ("oacc parallel_kernels_parallelized",
+			 DECL_ATTRIBUTES (current_function_decl)) != NULL);
+  if (is_oacc_parallel_kernels_parallelized)
+    gcc_checking_assert (!is_oacc_kernels);
+  bool is_oacc_parallel_kernels_gang_single
+    = (lookup_attribute ("oacc parallel_kernels_gang_single",
+			 DECL_ATTRIBUTES (current_function_decl)) != NULL);
+  if (is_oacc_parallel_kernels_gang_single)
+    gcc_checking_assert (!is_oacc_kernels);
+  gcc_checking_assert (!(is_oacc_parallel_kernels_parallelized
+			 && is_oacc_parallel_kernels_gang_single));
 
   /* Unparallelized OpenACC kernels constructs must get launched as 1 x 1 x 1
      kernels, so remove the parallelism dimensions function attributes
@@ -1627,6 +1641,12 @@ execute_oacc_device_lower ()
 	fprintf (dump_file, "Function is %s OpenACC kernels offload\n",
 		 (is_oacc_kernels_parallelized
 		  ? "parallelized" : "unparallelized"));
+      else if (is_oacc_parallel_kernels_parallelized)
+	fprintf (dump_file, "Function is %s OpenACC kernels offload\n",
+		 "parallel_kernels_parallelized");
+      else if (is_oacc_parallel_kernels_gang_single)
+	fprintf (dump_file, "Function is %s OpenACC kernels offload\n",
+		 "parallel_kernels_gang_single");
       else
 	fprintf (dump_file, "Function is OpenACC parallel offload\n");
     }
-- 
2.17.1


[-- Attachment #4: 0003-Separate-OpenACC-kernels-regions-in-data-and-paralle.patch --]
[-- Type: text/x-diff, Size: 24144 bytes --]

From b900846645ddacbe957cf0f6acdab03bcde68caa Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerg=C3=B6=20Barany?= <gergo@codesourcery.com>
Date: Mon, 21 Jan 2019 05:28:20 -0800
Subject: [PATCH 3/9] Separate OpenACC kernels regions in data and parallel
 parts

This is the first in a series of patches that completely rework the handling
of the OpenACC "kernels" directive. In the future, kernels regions will be
transformed into data regions containing a sequence of serial and parallel
offloaded regions. This first patch sets up a new pass that is responsible
for this transformation, and in a first step constructs the new data region
containing a parallel region with the original kernels region's body.

	gcc/
	* Makefile.in: Add...
	* omp-oacc-kernels.c: ... this new file for the kernels conversion
	pass.
	* flag-types.h (enum openacc_kernels): Add "split" style.  Adjust
	all users.
	* doc/invoke.texi (-fopenacc-kernels): Update.
	* passes.def: Add pass_convert_oacc_kernels to pipeline.
	* tree-pass.h (make_pass_convert_oacc_kernels): Add declaration.
	gcc/testsuite/
	* c-c++-common/goacc/kernels-conversion.c: New test.
	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
	* c-c++-common/goacc/if-clause-2.c: Update.
	* gfortran.dg/goacc/kernels-tree.f95: Likewise.
	libgomp/
	* testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90:
	Update.
---
 gcc/ChangeLog.openacc                         |  12 +
 gcc/Makefile.in                               |   2 +
 gcc/c-family/c.opt                            |   6 +-
 gcc/doc/invoke.texi                           |  13 +-
 gcc/flag-types.h                              |   1 +
 gcc/fortran/lang.opt                          |   3 +-
 gcc/omp-oacc-kernels.c                        | 245 ++++++++++++++++++
 gcc/passes.def                                |   1 +
 gcc/testsuite/ChangeLog.openacc               |   8 +
 .../c-c++-common/goacc/if-clause-2.c          |   7 +
 .../c-c++-common/goacc/kernels-conversion.c   |  36 +++
 .../gfortran.dg/goacc/kernels-conversion.f95  |  33 +++
 .../gfortran.dg/goacc/kernels-tree.f95        |   6 +
 gcc/tree-pass.h                               |   1 +
 libgomp/ChangeLog.openacc                     |   3 +
 .../initialize_kernels_loops.f90              |   8 +-
 16 files changed, 379 insertions(+), 6 deletions(-)
 create mode 100644 gcc/omp-oacc-kernels.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
 create mode 100644 gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 80f7efe18b1..a330410f41d 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,15 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* Makefile.in: Add...
+	* omp-oacc-kernels.c: ... this new file for the kernels conversion
+	pass.
+	* flag-types.h (enum openacc_kernels): Add "split" style.  Adjust
+	all users.
+	* doc/invoke.texi (-fopenacc-kernels): Update.
+	* passes.def: Add pass_convert_oacc_kernels to pipeline.
+	* tree-pass.h (make_pass_convert_oacc_kernels): Add declaration.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index 20bee0494b1..09685481787 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1420,6 +1420,7 @@ OBJS = \
 	omp-general.o \
 	omp-grid.o \
 	omp-low.o \
+	omp-oacc-kernels.o \
 	omp-simd-clone.o \
 	optabs.o \
 	optabs-libfuncs.o \
@@ -2565,6 +2566,7 @@ GTFILES = $(CPP_ID_DATA_H) $(srcdir)/input.h $(srcdir)/coretypes.h \
   $(srcdir)/omp-offload.c \
   $(srcdir)/omp-expand.c \
   $(srcdir)/omp-low.c \
+  $(srcdir)/omp-oacc-kernels.c \
   $(srcdir)/targhooks.c $(out_file) $(srcdir)/passes.c $(srcdir)/cgraphunit.c \
   $(srcdir)/cgraphclones.c \
   $(srcdir)/tree-phinodes.c \
diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
index 73b01598377..12f8f55c50f 100644
--- a/gcc/c-family/c.opt
+++ b/gcc/c-family/c.opt
@@ -1618,11 +1618,15 @@ C ObjC C++ ObjC++ LTO Joined Var(flag_openacc_dims)
 Specify default OpenACC compute dimensions.
 
 fopenacc-kernels=
-C ObjC C++ ObjC++ RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS) Undocumented
+C ObjC C++ ObjC++ RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS)
+-fopenacc-kernels=[split|parloops]	Configure OpenACC 'kernels' constructs handling.
 
 Enum
 Name(openacc_kernels) Type(enum openacc_kernels)
 
+EnumValue
+Enum(openacc_kernels) String(split) Value(OPENACC_KERNELS_SPLIT)
+
 EnumValue
 Enum(openacc_kernels) String(parloops) Value(OPENACC_KERNELS_PARLOOPS)
 
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 59421b84bac..3bbeb8c6839 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -183,7 +183,7 @@ in the following sections.
 -aux-info @var{filename}  -fallow-parameterless-variadic-functions @gol
 -fno-asm  -fno-builtin  -fno-builtin-@var{function}  -fgimple@gol
 -fhosted  -ffreestanding @gol
--fopenacc  -fopenacc-dim=@var{geom} @gol
+-fopenacc  -fopenacc-dim=@var{geom}  -fopenacc-kernels=@var{style} @gol
 -fopenmp  -fopenmp-simd @gol
 -fms-extensions  -fplan9-extensions  -fsso-struct=@var{endianness} @gol
 -fallow-single-precision  -fcond-mismatch  -flax-vector-conversions @gol
@@ -2158,6 +2158,17 @@ to runtime, the environment variable @var{GOMP_OPENACC_DIM} can be set.
 It has the same format as the option value, except that '-' is not
 permitted.
 
+@item -fopenacc-kernels=@var{style}
+@opindex fopenacc-kernels
+@cindex OpenACC accelerator programming
+Configure OpenACC 'kernels' constructs handling.
+With @option{-fopenacc-kernels=split}, OpenACC 'kernels' constructs
+are split into a sequence of compute constructs, each then handled
+individually.
+With @option{-fopenacc-kernels=parloops}, the whole OpenACC
+'kernels' constructs is handled by the @samp{parloops} pass.
+This is the default.
+
 @item -fopenmp
 @opindex fopenmp
 @cindex OpenMP parallel
diff --git a/gcc/flag-types.h b/gcc/flag-types.h
index 910be7c7fd4..d5c655f648f 100644
--- a/gcc/flag-types.h
+++ b/gcc/flag-types.h
@@ -347,6 +347,7 @@ enum cf_protection_level
 /* OpenACC 'kernels' constructs handling.  */
 enum openacc_kernels
 {
+  OPENACC_KERNELS_SPLIT,
   OPENACC_KERNELS_PARLOOPS
 };
 #endif /* ! GCC_FLAG_TYPES_H */
diff --git a/gcc/fortran/lang.opt b/gcc/fortran/lang.opt
index 097c623eb50..b3c9cdb425f 100644
--- a/gcc/fortran/lang.opt
+++ b/gcc/fortran/lang.opt
@@ -643,7 +643,8 @@ Fortran LTO Joined Var(flag_openacc_dims)
 ; Documented in C
 
 fopenacc-kernels=
-Fortran RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS) Undocumented
+Fortran RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS)
+; Documented in C
 
 fopenmp
 Fortran LTO
diff --git a/gcc/omp-oacc-kernels.c b/gcc/omp-oacc-kernels.c
new file mode 100644
index 00000000000..d1803774442
--- /dev/null
+++ b/gcc/omp-oacc-kernels.c
@@ -0,0 +1,245 @@
+/* Transformation pass for OpenACC kernels regions.  Converts a kernels
+   region into a series of smaller parallel regions.  There is a parallel
+   region for each parallelizable loop nest, as well as a "gang-single"
+   parallel region for each non-parallelizable piece of code.
+
+   Contributed by Gergö Barany <gergo@codesourcery.com> and
+                  Thomas Schwinge <thomas@codesourcery.com>
+
+   Copyright (C) 2019 Free Software Foundation, Inc.
+
+This file is part of GCC.
+
+GCC is free software; you can redistribute it and/or modify it under
+the terms of the GNU General Public License as published by the Free
+Software Foundation; either version 3, or (at your option) any later
+version.
+
+GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or
+FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+for more details.
+
+You should have received a copy of the GNU General Public License
+along with GCC; see the file COPYING3.  If not see
+<http://www.gnu.org/licenses/>.  */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "backend.h"
+#include "target.h"
+#include "tree.h"
+#include "gimple.h"
+#include "tree-pass.h"
+#include "cgraph.h"
+#include "fold-const.h"
+#include "gimplify.h"
+#include "gimple-iterator.h"
+#include "gimple-walk.h"
+#include "gomp-constants.h"
+
+/* This is a preprocessing pass to be run immediately before lower_omp.  It
+   will convert OpenACC "kernels" regions into sequences of "parallel"
+   regions.
+   For now, the translation is as follows:
+   - The entire kernels region is turned into a data region with clauses
+     taken from the kernels region.  New "create" clauses are added for all
+     variables declared at the top level in the kernels region.  */
+
+/* Transform KERNELS_REGION, which is an OpenACC kernels region, into a data
+   region containing the original kernels region.  */
+
+static gimple *
+transform_kernels_region (gimple *kernels_region)
+{
+  gcc_checking_assert (gimple_omp_target_kind (kernels_region)
+                        == GF_OMP_TARGET_KIND_OACC_KERNELS);
+
+  /* Collect the kernels region's data clauses and create the new data
+     region with those clauses.  */
+  tree kernels_clauses = gimple_omp_target_clauses (kernels_region);
+  tree data_clauses = NULL;
+  for (tree c = kernels_clauses; c; c = OMP_CLAUSE_CHAIN (c))
+    {
+      /* Certain map clauses are copied to the enclosing data region.  Any
+         non-data clause remains on the kernels region.  */
+      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_MAP)
+        {
+          tree decl = OMP_CLAUSE_DECL (c);
+          HOST_WIDE_INT kind = OMP_CLAUSE_MAP_KIND (c);
+          switch (kind)
+            {
+            default:
+              if (kind == GOMP_MAP_ALLOC &&
+                  integer_zerop (OMP_CLAUSE_SIZE (c)))
+                /* ??? This is an alloc clause for mapping a pointer whose
+                   target is already mapped.  We leave these on the inner
+                   parallel regions because moving them to the outer data
+                   region causes runtime errors.  */
+                break;
+
+              /* For non-artificial variables, and for non-declaration
+                 expressions like A[0:n], copy the clause to the data
+                 region.  */
+              if ((DECL_P (decl) && !DECL_ARTIFICIAL (decl))
+                  || !DECL_P (decl))
+                {
+                  tree new_clause = build_omp_clause (OMP_CLAUSE_LOCATION (c),
+                                                      OMP_CLAUSE_MAP);
+                  OMP_CLAUSE_SET_MAP_KIND (new_clause, kind);
+                  /* This must be unshared here to avoid "incorrect sharing
+                     of tree nodes" errors from verify_gimple.  */
+                  OMP_CLAUSE_DECL (new_clause) = unshare_expr (decl);
+                  OMP_CLAUSE_SIZE (new_clause) = OMP_CLAUSE_SIZE (c);
+                  OMP_CLAUSE_CHAIN (new_clause) = data_clauses;
+                  data_clauses = new_clause;
+
+                  /* Now that this data is mapped, the inner data clause on
+                     the kernels region can become a present clause.  */
+                  OMP_CLAUSE_SET_MAP_KIND (c, GOMP_MAP_FORCE_PRESENT);
+                }
+              break;
+
+            case GOMP_MAP_POINTER:
+            case GOMP_MAP_TO_PSET:
+            case GOMP_MAP_FORCE_TOFROM:
+            case GOMP_MAP_FIRSTPRIVATE_POINTER:
+            case GOMP_MAP_FIRSTPRIVATE_REFERENCE:
+              /* ??? Copying these map kinds leads to internal compiler
+                 errors in later passes.  */
+              break;
+            }
+        }
+      else if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_IF)
+        {
+          /* If there is an if clause, it must also be present on the
+             enclosing data region.  Temporarily remove the if clause's
+             chain to avoid copying it.  */
+          tree saved_chain = OMP_CLAUSE_CHAIN (c);
+          OMP_CLAUSE_CHAIN (c) = NULL;
+          tree new_if_clause = unshare_expr (c);
+          OMP_CLAUSE_CHAIN (c) = saved_chain;
+          OMP_CLAUSE_CHAIN (new_if_clause) = data_clauses;
+          data_clauses = new_if_clause;
+        }
+    }
+  /* Restore the original order of the clauses.  */
+  data_clauses = nreverse (data_clauses);
+
+  gimple *data_region
+    = gimple_build_omp_target (NULL, GF_OMP_TARGET_KIND_OACC_DATA_KERNELS,
+                               data_clauses);
+  gimple_set_location (data_region, gimple_location (kernels_region));
+
+  /* For now, just construct a new parallel region inside the data region.  */
+  gimple *inner_region
+    = gimple_build_omp_target (NULL, GF_OMP_TARGET_KIND_OACC_PARALLEL,
+                               kernels_clauses);
+  gimple_set_location (inner_region, gimple_location (kernels_region));
+  gimple_omp_set_body (inner_region, gimple_omp_body (kernels_region));
+
+  gbind *bind = gimple_build_bind (NULL, NULL, NULL);
+  gimple_bind_add_stmt (bind, inner_region);
+
+  /* Put the transformed pieces together.  The entire body of the region is
+     wrapped in a try-finally statement that calls __builtin_GOACC_data_end
+     for cleanup.  */
+  tree data_end_fn = builtin_decl_explicit (BUILT_IN_GOACC_DATA_END);
+  gimple *call = gimple_build_call (data_end_fn, 0);
+  gimple_seq cleanup = NULL;
+  gimple_seq_add_stmt (&cleanup, call);
+  gimple *try_stmt = gimple_build_try (bind, cleanup, GIMPLE_TRY_FINALLY);
+  gimple_omp_set_body (data_region, try_stmt);
+
+  return data_region;
+}
+
+/* Helper function of convert_oacc_kernels for walking the tree, calling
+   transform_kernels_region on each kernels region found.  */
+
+static tree
+scan_kernels (gimple_stmt_iterator *gsi_p, bool *handled_ops_p,
+              struct walk_stmt_info *)
+{
+  gimple *stmt = gsi_stmt (*gsi_p);
+  *handled_ops_p = false;
+
+  int kind;
+  switch (gimple_code (stmt))
+    {
+    case GIMPLE_OMP_TARGET:
+      kind = gimple_omp_target_kind (stmt);
+      if (kind == GF_OMP_TARGET_KIND_OACC_KERNELS)
+        {
+          gimple *new_region = transform_kernels_region (stmt);
+          gsi_replace (gsi_p, new_region, false);
+          *handled_ops_p = true;
+        }
+      break;
+
+    default:
+      break;
+    }
+
+  return NULL;
+}
+
+/* Find and transform OpenACC kernels regions in the current function.  */
+
+static unsigned int
+convert_oacc_kernels (void)
+{
+  struct walk_stmt_info wi;
+  gimple_seq body = gimple_body (current_function_decl);
+
+  memset (&wi, 0, sizeof (wi));
+  walk_gimple_seq_mod (&body, scan_kernels, NULL, &wi);
+
+  gimple_set_body (current_function_decl, body);
+
+  return 0;
+}
+
+namespace {
+
+const pass_data pass_data_convert_oacc_kernels =
+{
+  GIMPLE_PASS, /* type */
+  "convert_oacc_kernels", /* name */
+  OPTGROUP_OMP, /* optinfo_flags */
+  TV_NONE, /* tv_id */
+  PROP_gimple_any, /* properties_required */
+  0, /* properties_provided */
+  0, /* properties_destroyed */
+  0, /* todo_flags_start */
+  0, /* todo_flags_finish */
+};
+
+class pass_convert_oacc_kernels : public gimple_opt_pass
+{
+public:
+  pass_convert_oacc_kernels (gcc::context *ctxt)
+    : gimple_opt_pass (pass_data_convert_oacc_kernels, ctxt)
+  {}
+
+  /* opt_pass methods: */
+  virtual bool gate (function *)
+  {
+    return (flag_openacc
+	    && flag_openacc_kernels == OPENACC_KERNELS_SPLIT);
+  }
+  virtual unsigned int execute (function *)
+  {
+    return convert_oacc_kernels ();
+  }
+
+}; // class pass_convert_oacc_kernels
+
+} // anon namespace
+
+gimple_opt_pass *
+make_pass_convert_oacc_kernels (gcc::context *ctxt)
+{
+  return new pass_convert_oacc_kernels (ctxt);
+}
diff --git a/gcc/passes.def b/gcc/passes.def
index 3ebcfc30349..4840bb6cff7 100644
--- a/gcc/passes.def
+++ b/gcc/passes.def
@@ -34,6 +34,7 @@ along with GCC; see the file COPYING3.  If not see
   NEXT_PASS (pass_warn_unused_result);
   NEXT_PASS (pass_diagnose_omp_blocks);
   NEXT_PASS (pass_diagnose_tm_blocks);
+  NEXT_PASS (pass_convert_oacc_kernels);
   NEXT_PASS (pass_lower_omp);
   NEXT_PASS (pass_lower_cf);
   NEXT_PASS (pass_lower_tm);
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 4acd174dca9..887011e7d1f 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,3 +1,11 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* c-c++-common/goacc/kernels-conversion.c: New test.
+	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
+	* c-c++-common/goacc/if-clause-2.c: Update.
+	* gfortran.dg/goacc/kernels-tree.f95: Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
 	* c-c++-common/goacc/kernels-1.c: Add
diff --git a/gcc/testsuite/c-c++-common/goacc/if-clause-2.c b/gcc/testsuite/c-c++-common/goacc/if-clause-2.c
index 5ab8459d732..e17b5dd1107 100644
--- a/gcc/testsuite/c-c++-common/goacc/if-clause-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/if-clause-2.c
@@ -1,3 +1,6 @@
+/* { dg-additional-options "-fopenacc-kernels=split" } */
+/* { dg-additional-options "-fdump-tree-convert_oacc_kernels" } */
+
 void
 f (short c)
 {
@@ -9,3 +12,7 @@ f (short c)
   ;
 #pragma acc update device(c) if(c)
 }
+
+/* Verify that the 'if' clause gets duplicated.
+   { dg-final { scan-tree-dump-times "#pragma omp target oacc_data_kernels if\\(" 1 "convert_oacc_kernels" } }
+   { dg-final { scan-tree-dump-times "#pragma omp target oacc_parallel_kernels_gang_single .* if\\(" 1 "convert_oacc_kernels" } } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
new file mode 100644
index 00000000000..c75db375f26
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
@@ -0,0 +1,36 @@
+/* { dg-additional-options "-fopenacc-kernels=split" } */
+/* { dg-additional-options "-fdump-tree-convert_oacc_kernels" } */
+
+#define N 1024
+
+unsigned int a[N];
+
+int
+main (void)
+{
+  int i;
+  unsigned int sum = 1;
+
+#pragma acc kernels copyin(a[0:N]) copy(sum)
+  {
+    #pragma acc loop
+    for (i = 0; i < N; ++i)
+      sum += a[i];
+
+    sum++;
+
+    #pragma acc loop
+    for (i = 0; i < N; ++i)
+      sum += a[i];
+  }
+
+  return 0;
+}
+
+/* Check that the kernels region is split into a data region and an enclosed
+   parallel region.  */ 
+/* { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel" 1 "convert_oacc_kernels" } } */
+
+/* Check that the original kernels region is removed.  */
+/* { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } } */
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
new file mode 100644
index 00000000000..8c663302a6f
--- /dev/null
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
@@ -0,0 +1,33 @@
+! { dg-additional-options "-fopenacc-kernels=split" }
+! { dg-additional-options "-fdump-tree-convert_oacc_kernels" }
+
+program main
+  implicit none
+  integer, parameter         :: N = 1024
+  integer, dimension (1:N)   :: a
+  integer                    :: i, sum
+
+  !$acc kernels copyin(a(1:N)) copy(sum)
+
+  !$acc loop
+  do i = 1, N
+    sum = sum + a(i)
+  end do
+
+  sum = sum + 1
+
+  !$acc loop
+  do i = 1, N
+    sum = sum + a(i)
+  end do
+
+  !$acc end kernels
+end program main
+
+! Check that the kernels region is split into a data region and an enclosed
+! parallel region.
+! { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel" 1 "convert_oacc_kernels" } }
+
+! Check that the original kernels region is removed.
+! { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95
index a70f1e737bd..b83ca2d8f06 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95
@@ -1,5 +1,7 @@
 ! { dg-do compile } 
 ! { dg-additional-options "-fdump-tree-original" } 
+! { dg-additional-options "-fopenacc-kernels=split" }
+! { dg-additional-options "-fdump-tree-convert_oacc_kernels" }
 
 program test
   implicit none
@@ -33,3 +35,7 @@ end program test
 ! { dg-final { scan-tree-dump-times "map\\(alloc:t\\)" 1 "original" } } 
 
 ! { dg-final { scan-tree-dump-times "map\\(force_deviceptr:u\\)" 1 "original" } } 
+
+! Verify that the 'if' clause gets duplicated.
+! { dg-final { scan-tree-dump-times "#pragma omp target oacc_data_kernels if\\(" 1 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "#pragma omp target oacc_parallel_kernels_gang_single .* if\\(" 1 "convert_oacc_kernels" } }
diff --git a/gcc/tree-pass.h b/gcc/tree-pass.h
index 93a6a99eb7a..24f2110bb1d 100644
--- a/gcc/tree-pass.h
+++ b/gcc/tree-pass.h
@@ -416,6 +416,7 @@ extern gimple_opt_pass *make_pass_lower_switch (gcc::context *ctxt);
 extern gimple_opt_pass *make_pass_lower_vector (gcc::context *ctxt);
 extern gimple_opt_pass *make_pass_lower_vector_ssa (gcc::context *ctxt);
 extern gimple_opt_pass *make_pass_lower_omp (gcc::context *ctxt);
+extern gimple_opt_pass *make_pass_convert_oacc_kernels (gcc::context *ctxt);
 extern gimple_opt_pass *make_pass_diagnose_omp_blocks (gcc::context *ctxt);
 extern gimple_opt_pass *make_pass_expand_omp (gcc::context *ctxt);
 extern gimple_opt_pass *make_pass_expand_omp_ssa (gcc::context *ctxt);
diff --git a/libgomp/ChangeLog.openacc b/libgomp/ChangeLog.openacc
index bf572ee8922..f2ff2ee32d2 100644
--- a/libgomp/ChangeLog.openacc
+++ b/libgomp/ChangeLog.openacc
@@ -1,5 +1,8 @@
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
+	* testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90:
+	Update.
+
 	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-2.c:
 	Add "-fopenacc-kernels=parloops".
 	* testsuite/libgomp.oacc-c-c++-common/kernels-alias-ipa-pta-3.c:
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90 b/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90
index 8eb02b88d25..35e909f8278 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90
+++ b/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90
@@ -1,16 +1,18 @@
 ! { dg-do run }
+! { dg-additional-options "-fopenacc-kernels=split" }
+! { dg-additional-options "-fopt-info-optimized-omp" }
 
 subroutine kernel(lo, hi, a, b, c)
     implicit none
     integer :: lo, hi, i
     real, dimension(lo:hi) :: a, b, c
 
-!$acc kernels ! { dg-bogus "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "TODO" { xfail { openacc_nvidia_accel_selected && opt_levels_2_plus } } }
-!$acc loop independent
+!$acc kernels
+!$acc loop independent ! { dg-warning "note: parallelized loop nest in OpenACC .kernels. construct" }
     do i = lo, hi
       b(i) = a(i)
     end do
-!$acc loop independent
+!$acc loop independent ! { dg-warning "note: parallelized loop nest in OpenACC .kernels. construct" }
     do i = lo, hi
       c(i) = b(i)
     end do
-- 
2.17.1


[-- Attachment #5: 0004-Turn-OpenACC-kernels-regions-into-a-sequence-of-para.patch --]
[-- Type: text/x-diff, Size: 31628 bytes --]

From 2e63393893fac8e9abcf9665a2055f90dce76e86 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerg=C3=B6=20Barany?= <gergo@codesourcery.com>
Date: Mon, 21 Jan 2019 07:16:06 -0800
Subject: [PATCH 4/9] Turn OpenACC kernels regions into a sequence of parallel
 regions

This patch decomposes each OpenACC kernels region into a sequence of
parallel regions. Each OpenACC loop nest turns into its own region; any code
between such loop nests is gathered up into a region as well. The loop
regions can be distributed across gangs if the original kernels region had a
num_gangs clause, while the other regions are executed in "gang-single"
mode. The implied default "auto" clause on kernels loops is made explicit
unless there is a conflicting clause.

    gcc/
    * omp-oacc-kernels.c (top_level_omp_for_in_stmt): New function.
    (make_gang_single_region): Likewise.
    (transform_kernels_loop_clauses, make_gang_parallel_loop_region):
    Likewise.
    (flatten_binds): Likewise.
    (make_data_region_try_statement): Likewise.
    (maybe_build_inner_data_region): Likewise.
    (decompose_kernels_region_body): Likewise.
    (transform_kernels_region): Delegate to decompose_kernels_region_body
    and make_data_region_try_statement.

    gcc/testsuite/
    * c-c++-common/goacc/kernels-conversion.c: Test for a gang-single
    region.
    * gfortran.dg/goacc/kernels-conversion.f95: Likewise.
---
 gcc/ChangeLog.openacc                         |  14 +
 gcc/omp-oacc-kernels.c                        | 558 +++++++++++++++++-
 gcc/testsuite/ChangeLog.openacc               |   7 +
 .../c-c++-common/goacc/kernels-conversion.c   |  11 +-
 .../gfortran.dg/goacc/kernels-conversion.f95  |  11 +-
 5 files changed, 578 insertions(+), 23 deletions(-)

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index a330410f41d..6fa92ee2731 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,17 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* omp-oacc-kernels.c (top_level_omp_for_in_stmt): New function.
+	(make_gang_single_region): Likewise.
+	(transform_kernels_loop_clauses, make_gang_parallel_loop_region):
+	Likewise.
+	(flatten_binds): Likewise.
+	(make_data_region_try_statement): Likewise.
+	(maybe_build_inner_data_region): Likewise.
+	(decompose_kernels_region_body): Likewise.
+	(transform_kernels_region): Delegate to decompose_kernels_region_body
+	and make_data_region_try_statement.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/omp-oacc-kernels.c b/gcc/omp-oacc-kernels.c
index d1803774442..6e083666a17 100644
--- a/gcc/omp-oacc-kernels.c
+++ b/gcc/omp-oacc-kernels.c
@@ -30,6 +30,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "backend.h"
 #include "target.h"
 #include "tree.h"
+#include "cp/cp-tree.h"
 #include "gimple.h"
 #include "tree-pass.h"
 #include "cgraph.h"
@@ -45,16 +46,548 @@ along with GCC; see the file COPYING3.  If not see
    For now, the translation is as follows:
    - The entire kernels region is turned into a data region with clauses
      taken from the kernels region.  New "create" clauses are added for all
-     variables declared at the top level in the kernels region.  */
+     variables declared at the top level in the kernels region.
+   - Any loop annotated with an OpenACC loop directive is wrapped in a new
+     parallel region.  Gang/worker/vector annotations are copied from the
+     original kernels region if present.
+     * Loops without an explicit "independent" or "seq" annotation get an
+       "auto" annotation; other annotations are preserved on the loop or
+       moved to the new surrounding parallel region.  Which annotations are
+       moved is determined by the constraints in the OpenACC spec; for
+       example, loops in the kernels region may have a gang clause, but
+       such annotations must now be moved to the new parallel region.
+   - Any sequences of other code (non-loops, non-OpenACC loops) are wrapped
+     in new "gang-single" parallel regions: Worker/vector annotations are
+     copied from the original kernels region if present, but num_gangs is
+     explicitly set to 1.  */
+
+/* Helper function for decompose_kernels_region_body.  If STMT contains a
+   "top-level" OMP_FOR statement, returns a pointer to that statement;
+   returns NULL otherwise.
+
+   A "top-level" OMP_FOR statement is one that is possibly accompanied by
+   small snippets of setup code.  Specifically, this function accepts an
+   OMP_FOR possibly wrapped in a singleton bind and a singleton try
+   statement to allow for a local loop variable, but not an OMP_FOR
+   statement nested in any other constructs.  Alternatively, it accepts a
+   non-singleton bind containing only assignments and then an OMP_FOR
+   statement at the very end.  The former style can be generated by the C
+   frontend, the latter by the Fortran frontend.  */
+
+static gimple *
+top_level_omp_for_in_stmt (gimple *stmt)
+{
+  if (gimple_code (stmt) == GIMPLE_OMP_FOR)
+    return stmt;
+
+  if (gimple_code (stmt) == GIMPLE_BIND)
+    {
+      gimple_seq body = gimple_bind_body (as_a <gbind *> (stmt));
+      if (gimple_seq_singleton_p (body))
+        {
+          /* Accept an OMP_FOR statement, or a try statement containing only
+             a single OMP_FOR.  */
+          gimple *maybe_for_or_try = gimple_seq_first_stmt (body);
+          if (gimple_code (maybe_for_or_try) == GIMPLE_OMP_FOR)
+            return maybe_for_or_try;
+          else if (gimple_code (maybe_for_or_try) == GIMPLE_TRY)
+            {
+              gimple_seq try_body = gimple_try_eval (maybe_for_or_try);
+              if (!gimple_seq_singleton_p (try_body))
+                return NULL;
+              gimple *maybe_omp_for_stmt = gimple_seq_first_stmt (try_body);
+              if (gimple_code (maybe_omp_for_stmt) == GIMPLE_OMP_FOR)
+                return maybe_omp_for_stmt;
+            }
+        }
+      else
+        {
+          gimple_stmt_iterator gsi;
+          /* Accept only a block of optional assignments followed by an
+             OMP_FOR at the end.  No other kinds of statements allowed.  */
+          for (gsi = gsi_start (body); !gsi_end_p (gsi); gsi_next (&gsi))
+            {
+              gimple *body_stmt = gsi_stmt (gsi);
+              if (gimple_code (body_stmt) == GIMPLE_ASSIGN)
+                continue;
+              else if (gimple_code (body_stmt) == GIMPLE_OMP_FOR
+                        && gsi_one_before_end_p (gsi))
+                return body_stmt;
+              else
+                return NULL;
+            }
+        }
+    }
+
+  return NULL;
+}
+
+/* Construct a "gang-single" OpenACC parallel region at LOC containing the
+   STMTS.  The newly created region is annotated with CLAUSES, which must
+   not contain a num_gangs clause, and an additional "num_gangs(1)" clause
+   to force gang-single execution.  */
+
+static gimple *
+make_gang_single_region (location_t loc, gimple_seq stmts, tree clauses)
+{
+  /* This correctly unshares the entire clause chain rooted here.  */
+  clauses = unshare_expr (clauses);
+  /* Make a num_gangs(1) clause.  */
+  tree gang_single_clause = build_omp_clause (loc, OMP_CLAUSE_NUM_GANGS);
+  OMP_CLAUSE_OPERAND (gang_single_clause, 0) = integer_one_node;
+  OMP_CLAUSE_CHAIN (gang_single_clause) = clauses;
+
+  /* Build the gang-single region.  */
+  gimple *single_region
+    = gimple_build_omp_target (
+        NULL,
+        GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE,
+        gang_single_clause);
+  gimple_set_location (single_region, loc);
+  gbind *single_body = gimple_build_bind (NULL, stmts, make_node (BLOCK));
+  gimple_omp_set_body (single_region, single_body);
+
+  return single_region;
+}
+
+/* Helper for make_region_loop_nest.  Transform OpenACC 'kernels'/'loop'
+   construct clauses into OpenACC 'parallel'/'loop' construct ones.  */
+
+static tree
+transform_kernels_loop_clauses (gimple *omp_for,
+				tree num_gangs_clause,
+				tree clauses)
+{
+  /* If this loop in a kernels region does not have an explicit
+     "independent", "seq", or "auto" clause, we must give it an explicit
+     "auto" clause. */
+  bool add_auto_clause = true;
+  tree loop_clauses = gimple_omp_for_clauses (omp_for);
+  for (tree c = loop_clauses; c; c = OMP_CLAUSE_CHAIN (c))
+    {
+      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_AUTO
+          || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_INDEPENDENT
+          || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_SEQ)
+        {
+          add_auto_clause = false;
+          break;
+        }
+    }
+  if (add_auto_clause)
+    {
+      tree auto_clause = build_omp_clause (gimple_location (omp_for),
+                                           OMP_CLAUSE_AUTO);
+      OMP_CLAUSE_CHAIN (auto_clause) = loop_clauses;
+      gimple_omp_for_set_clauses (omp_for, auto_clause);
+    }
+
+  /* If the kernels region had a num_gangs clause, add that to this new
+     parallel region.  */
+  if (num_gangs_clause != NULL)
+    {
+      tree parallel_num_gangs_clause = unshare_expr (num_gangs_clause);
+      OMP_CLAUSE_CHAIN (parallel_num_gangs_clause) = clauses;
+      clauses = parallel_num_gangs_clause;
+    }
+
+  return clauses;
+}
+
+/* Construct a possibly gang-parallel OpenACC parallel region containing the
+   STMT, which must be identical to, or a bind containing, the loop OMP_FOR
+   with OpenACC loop annotations.
+
+   The newly created region is annotated with the optional NUM_GANGS_CLAUSE
+   as well as the other CLAUSES, which must not contain a num_gangs clause.  */
+
+static gimple *
+make_gang_parallel_loop_region (gimple *omp_for, gimple *stmt,
+                                tree num_gangs_clause, tree clauses)
+{
+  /* This correctly unshares the entire clause chain rooted here.  */
+  clauses = unshare_expr (clauses);
+
+  clauses = transform_kernels_loop_clauses (omp_for,
+					    num_gangs_clause,
+					    clauses);
+
+  /* Now build the parallel region containing this loop.  */
+  gimple_seq parallel_body = NULL;
+  gimple_seq_add_stmt (&parallel_body, stmt);
+  gimple *parallel_body_bind
+    = gimple_build_bind (NULL, parallel_body, make_node (BLOCK));
+  gimple *parallel_region
+    = gimple_build_omp_target (
+        parallel_body_bind,
+        GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED,
+        clauses);
+  gimple_set_location (parallel_region, gimple_location (stmt));
+
+  return parallel_region;
+}
+
+/* Eliminate any binds directly inside BIND by adding their statements to
+   BIND (i.e., modifying it in place), excluding binds that hold only an
+   OMP_FOR loop and associated setup/cleanup code.  Recurse into binds but
+   not other statements.  Return a chain of the local variables of eliminated
+   binds, i.e., the local variables found in nested binds.  If
+   INCLUDE_TOPLEVEL_VARS is true, this also includes the variables belonging
+   to BIND itself. */
+
+static tree
+flatten_binds (gbind *bind, bool include_toplevel_vars = false)
+{
+  tree vars = NULL, last_var = NULL;
+
+  if (include_toplevel_vars)
+    {
+      vars = gimple_bind_vars (bind);
+      last_var = vars;
+    }
+
+  gimple_seq new_body = NULL;
+  gimple_seq body_sequence = gimple_bind_body (bind);
+  gimple_stmt_iterator gsi, gsi_n;
+  for (gsi = gsi_start (body_sequence); !gsi_end_p (gsi); gsi = gsi_n)
+    {
+      /* Advance the iterator here because otherwise it would be invalidated
+         by moving statements below.  */
+      gsi_n = gsi;
+      gsi_next (&gsi_n);
+
+      gimple *stmt = gsi_stmt (gsi);
+      /* Flatten bind statements, except the ones that contain only an
+         OpenACC for loop.  */
+      if (gimple_code (stmt) == GIMPLE_BIND
+          && !top_level_omp_for_in_stmt (stmt))
+        {
+          gbind *inner_bind = as_a <gbind *> (stmt);
+          /* Flatten recursively, and collect all variables.  */
+          tree inner_vars = flatten_binds (inner_bind, true);
+          gimple_seq inner_sequence = gimple_bind_body (inner_bind);
+          gcc_assert (gimple_code (inner_sequence) != GIMPLE_BIND
+                      || top_level_omp_for_in_stmt (inner_sequence));
+          gimple_seq_add_seq (&new_body, inner_sequence);
+          /* Find the last variable; we will append others to it.  */
+          while (last_var != NULL && TREE_CHAIN (last_var) != NULL)
+            last_var = TREE_CHAIN (last_var);
+          if (last_var != NULL)
+            {
+              TREE_CHAIN (last_var) = inner_vars;
+              last_var = inner_vars;
+            }
+          else
+            {
+              vars = inner_vars;
+              last_var = vars;
+            }
+        }
+      else
+        gimple_seq_add_stmt (&new_body, stmt);
+    }
+
+  /* Put the possibly transformed body back into the bind.  */
+  gimple_bind_set_body (bind, new_body);
+  return vars;
+}
+
+/* Helper function for places where we construct data regions.  Wraps the BODY
+   inside a try-finally construct at LOC that calls __builtin_GOACC_data_end
+   in its cleanup block.  Returns this try statement.  */
+
+static gimple *
+make_data_region_try_statement (location_t loc, gimple *body)
+{
+  tree data_end_fn = builtin_decl_explicit (BUILT_IN_GOACC_DATA_END);
+  gimple *call = gimple_build_call (data_end_fn, 0);
+  gimple_seq cleanup = NULL;
+  gimple_seq_add_stmt (&cleanup, call);
+  gimple *try_stmt = gimple_build_try (body, cleanup, GIMPLE_TRY_FINALLY);
+  gimple_set_location (body, loc);
+  return try_stmt;
+}
+
+/* If INNER_BIND_VARS holds variables, build an OpenACC data region with
+   location LOC containing BODY and having "create(var)" clauses for each
+   variable.  If INNER_CLEANUP is present, add a try-finally statement with
+   this cleanup code in the finally block.  Return the new data region, or
+   the original BODY if no data region was needed.  */
+
+static gimple *
+maybe_build_inner_data_region (location_t loc, gimple *body,
+                               tree inner_bind_vars, gimple *inner_cleanup)
+{
+  /* Build data "create(var)" clauses for these local variables.
+     Below we will add these to a data region enclosing the entire body
+     of the decomposed kernels region.  */
+  tree prev_mapped_var = NULL, next = NULL, artificial_vars = NULL,
+       inner_data_clauses = NULL;
+  for (tree v = inner_bind_vars; v; v = next)
+    {
+      next = TREE_CHAIN (v);
+      if (DECL_ARTIFICIAL (v)
+          || TREE_CODE (v) == CONST_DECL
+          || (DECL_LANG_SPECIFIC (current_function_decl)
+              && DECL_TEMPLATE_INSTANTIATION (current_function_decl)))
+        {
+          /* If this is an artificial temporary, it need not be mapped.  We
+             move its declaration into the bind inside the data region.
+             Also avoid mapping variables if we are inside a template
+             instantiation; the code does not contain all the copies to
+             temporaries that would make this legal.  */
+          TREE_CHAIN (v) = artificial_vars;
+          artificial_vars = v;
+          if (prev_mapped_var != NULL)
+            TREE_CHAIN (prev_mapped_var) = next;
+          else
+            inner_bind_vars = next;
+        }
+      else
+        {
+          /* Otherwise, build the map clause.  */
+          tree new_clause = build_omp_clause (loc, OMP_CLAUSE_MAP);
+          OMP_CLAUSE_SET_MAP_KIND (new_clause, GOMP_MAP_ALLOC);
+          OMP_CLAUSE_DECL (new_clause) = v;
+          OMP_CLAUSE_SIZE (new_clause) = DECL_SIZE_UNIT (v);
+          OMP_CLAUSE_CHAIN (new_clause) = inner_data_clauses;
+          inner_data_clauses = new_clause;
+
+          prev_mapped_var = v;
+        }
+    }
+
+  if (artificial_vars)
+    body = gimple_build_bind (artificial_vars, body, make_node (BLOCK));
+
+  /* If we determined above that there are variables that need to be created
+     on the device, construct a data region for them and wrap the body
+     inside that.  */
+  if (inner_data_clauses != NULL)
+    {
+      gcc_assert (inner_bind_vars != NULL);
+      gimple *inner_data_region
+        = gimple_build_omp_target (NULL, GF_OMP_TARGET_KIND_OACC_DATA_KERNELS,
+                                   inner_data_clauses);
+      gimple_set_location (inner_data_region, loc);
+      /* Make sure __builtin_GOACC_data_end is called at the end.  */
+      gimple *try_stmt = make_data_region_try_statement (loc, body);
+      gimple_omp_set_body (inner_data_region, try_stmt);
+      gimple *bind_body;
+      if (inner_cleanup != NULL)
+          /* Clobber all the inner variables that need to be clobbered.  */
+          bind_body = gimple_build_try (inner_data_region, inner_cleanup,
+                                        GIMPLE_TRY_FINALLY);
+      else
+          bind_body = inner_data_region;
+      body = gimple_build_bind (inner_bind_vars, bind_body, make_node (BLOCK));
+    }
+
+  return body;
+}
+
+/* Decompose the body of the KERNELS_REGION, which was originally annotated
+   with the KERNELS_CLAUSES, into a series of parallel regions.  */
+
+static gimple *
+decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
+{
+  location_t loc = gimple_location (kernels_region);
+
+  /* The kernels clauses will be propagated to the child clauses unmodified,
+     except that that num_gangs clause will only be added to loop regions.
+     The other regions are "gang-single" and get an explicit num_gangs(1)
+     clause.  So separate out the num_gangs clause here.  */
+  tree num_gangs_clause = NULL, prev_clause = NULL;
+  tree parallel_clauses = kernels_clauses;
+  for (tree c = parallel_clauses; c; c = OMP_CLAUSE_CHAIN (c))
+    {
+      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NUM_GANGS)
+        {
+          /* Cut this clause out of the chain.  */
+          num_gangs_clause = c;
+          if (prev_clause != NULL)
+            OMP_CLAUSE_CHAIN (prev_clause) = OMP_CLAUSE_CHAIN (c);
+          else
+            kernels_clauses = OMP_CLAUSE_CHAIN (c);
+          OMP_CLAUSE_CHAIN (num_gangs_clause) = NULL;
+          break;
+        }
+      else
+        prev_clause = c;
+    }
+
+  gimple *kernels_body = gimple_omp_body (kernels_region);
+  gbind *kernels_bind = as_a <gbind *> (kernels_body);
+
+  /* The body of the region may contain other nested binds declaring inner
+     local variables.  Collapse all these binds into one to ensure that we
+     have a single sequence of statements to iterate over; also, collect all
+     inner variables.  */
+  tree inner_bind_vars = flatten_binds (kernels_bind);
+  gimple_seq body_sequence = gimple_bind_body (kernels_bind);
+
+  /* All these inner variables will get allocated on the device (below, by
+     calling maybe_build_inner_data_region).  Here we create "present"
+     clauses for them and add these clauses to the list of clauses to be
+     attached to each inner parallel region.  */
+  tree present_clauses = kernels_clauses;
+  for (tree var = inner_bind_vars; var; var = TREE_CHAIN (var))
+    {
+      if (!DECL_ARTIFICIAL (var) && TREE_CODE (var) != CONST_DECL)
+        {
+          tree present_clause = build_omp_clause (loc, OMP_CLAUSE_MAP);
+          OMP_CLAUSE_SET_MAP_KIND (present_clause, GOMP_MAP_FORCE_PRESENT);
+          OMP_CLAUSE_DECL (present_clause) = var;
+          OMP_CLAUSE_SIZE (present_clause) = DECL_SIZE_UNIT (var);
+          OMP_CLAUSE_CHAIN (present_clause) = present_clauses;
+          present_clauses = present_clause;
+        }
+    }
+  kernels_clauses = present_clauses;
+
+  /* In addition to nested binds, the "real" body of the region may be
+     nested inside a try-finally block.  Find its cleanup block, which
+     contains code to clobber the local variables that must be clobbered.  */
+  gimple *inner_cleanup = NULL;
+  if (body_sequence != NULL && gimple_code (body_sequence) == GIMPLE_TRY)
+    {
+      if (gimple_seq_singleton_p (body_sequence))
+        {
+          /* The try statement is the only thing inside the bind.  */
+          inner_cleanup = gimple_try_cleanup (body_sequence);
+          body_sequence = gimple_try_eval (body_sequence);
+        }
+      else
+        {
+          /* The bind's body starts with a try statement, but it is followed
+             by other things.  */
+          gimple_stmt_iterator gsi = gsi_start (body_sequence);
+          gimple *try_stmt = gsi_stmt (gsi);
+          inner_cleanup = gimple_try_cleanup (try_stmt);
+          gimple *try_body = gimple_try_eval (try_stmt);
+
+          gsi_remove (&gsi, false);
+          /* Now gsi indicates the sequence of statements after the try
+             statement in the bind.  Append the statement in the try body and
+             the trailing statements from gsi.  */
+          gsi_insert_seq_before (&gsi, try_body, GSI_CONTINUE_LINKING);
+          body_sequence = gsi_stmt (gsi);
+        }
+    }
+
+  /* This sequence will collect all the top-level statements in the body of
+     the data region we are about to construct.  */
+  gimple_seq region_body = NULL;
+  /* This sequence will collect consecutive statements to be put into a
+     gang-single region.  */
+  gimple_seq gang_single_seq = NULL;
+  /* Flag recording whether the gang_single_seq only contains copies to
+     local variables.  These may be loop setup code that should not be
+     separated from the loop.  */
+  bool only_simple_assignments = true;
+
+  /* Iterate over the statements in the kernels region's body.  */
+  gimple_stmt_iterator gsi, gsi_n;
+  for (gsi = gsi_start (body_sequence); !gsi_end_p (gsi); gsi = gsi_n)
+    {
+      /* Advance the iterator here because otherwise it would be invalidated
+         by moving statements below.  */
+      gsi_n = gsi;
+      gsi_next (&gsi_n);
+
+      gimple *stmt = gsi_stmt (gsi);
+      gimple *omp_for = top_level_omp_for_in_stmt (stmt);
+      if (omp_for != NULL)
+        {
+          /* This is an OMP for statement, put it into a parallel region.
+             But first, construct a gang-single region containing any
+             complex sequential statements we may have seen.  */
+          if (gang_single_seq != NULL && !only_simple_assignments)
+            {
+              gimple *single_region
+                = make_gang_single_region (loc, gang_single_seq,
+                                           kernels_clauses);
+              gimple_seq_add_stmt (&region_body, single_region);
+            }
+          else if (gang_single_seq != NULL && only_simple_assignments)
+            {
+              /* There is a sequence of sequential statements preceding this
+                 loop, but they are all simple assignments.  This is
+                 probably setup code for the loop; in particular, Fortran DO
+                 loops are preceded by code to copy the loop limit variable
+                 to a temporary.  Group this code together with the loop
+                 itself.  */
+              gimple_seq_add_stmt (&gang_single_seq, stmt);
+              stmt = gimple_build_bind (NULL, gang_single_seq,
+                                        make_node (BLOCK));
+            }
+          gang_single_seq = NULL;
+          only_simple_assignments = true;
+
+          gimple *parallel_region
+            = make_gang_parallel_loop_region (omp_for, stmt,
+                                              num_gangs_clause,
+                                              kernels_clauses);
+          gimple_seq_add_stmt (&region_body, parallel_region);
+        }
+      else
+        {
+          /* This is not an OMP for statement, so it will be put into a
+             gang-single region.  */
+          gimple_seq_add_stmt (&gang_single_seq, stmt);
+          /* Is this a simple assignment? We call it simple if it is an
+             assignment to an artificial local variable.  This captures
+             Fortran loop setup code computing loop bounds and offsets.  */
+          bool is_simple_assignment
+            = (gimple_code (stmt) == GIMPLE_ASSIGN
+                && TREE_CODE (gimple_assign_lhs (stmt)) == VAR_DECL
+                && DECL_ARTIFICIAL (gimple_assign_lhs (stmt)));
+          if (!is_simple_assignment)
+            only_simple_assignments = false;
+        }
+    }
+
+  /* If we did not emit a new region, and are not going to emit one now
+     (that is, the original region was empty), prepare to emit a dummy so as
+     to preserve the original construct, which other processing (at least
+     test cases) depend on.  */
+  if (region_body == NULL && gang_single_seq == NULL)
+    {
+      gimple *stmt = gimple_build_nop ();
+      gimple_set_location (stmt, loc);
+      gimple_seq_add_stmt (&gang_single_seq, stmt);
+    }
+
+  /* Gather up any remaining gang-single statements.  */
+  if (gang_single_seq != NULL)
+    {
+      gimple *single_region
+        = make_gang_single_region (loc, gang_single_seq, kernels_clauses);
+      gimple_seq_add_stmt (&region_body, single_region);
+    }
+
+  tree kernels_locals = gimple_bind_vars (as_a <gbind *> (kernels_body));
+  gimple *body = gimple_build_bind (kernels_locals, region_body,
+                                    make_node (BLOCK));
+
+  /* If we found variables declared in nested scopes, build a data region to
+     map them to the device.  */
+  body = maybe_build_inner_data_region (loc, body, inner_bind_vars,
+                                        inner_cleanup);
+
+  return body;
+}
 
 /* Transform KERNELS_REGION, which is an OpenACC kernels region, into a data
-   region containing the original kernels region.  */
+   region containing the original kernels region's body cut up into a
+   sequence of parallel regions.  */
 
 static gimple *
 transform_kernels_region (gimple *kernels_region)
 {
   gcc_checking_assert (gimple_omp_target_kind (kernels_region)
                         == GF_OMP_TARGET_KIND_OACC_KERNELS);
+  location_t loc = gimple_location (kernels_region);
 
   /* Collect the kernels region's data clauses and create the new data
      region with those clauses.  */
@@ -130,26 +663,17 @@ transform_kernels_region (gimple *kernels_region)
   gimple *data_region
     = gimple_build_omp_target (NULL, GF_OMP_TARGET_KIND_OACC_DATA_KERNELS,
                                data_clauses);
-  gimple_set_location (data_region, gimple_location (kernels_region));
-
-  /* For now, just construct a new parallel region inside the data region.  */
-  gimple *inner_region
-    = gimple_build_omp_target (NULL, GF_OMP_TARGET_KIND_OACC_PARALLEL,
-                               kernels_clauses);
-  gimple_set_location (inner_region, gimple_location (kernels_region));
-  gimple_omp_set_body (inner_region, gimple_omp_body (kernels_region));
+  gimple_set_location (data_region, loc);
 
-  gbind *bind = gimple_build_bind (NULL, NULL, NULL);
-  gimple_bind_add_stmt (bind, inner_region);
+  /* Transform the body of the kernels region into a sequence of parallel
+     regions.  */
+  gimple *body = decompose_kernels_region_body (kernels_region,
+                                                kernels_clauses);
 
   /* Put the transformed pieces together.  The entire body of the region is
      wrapped in a try-finally statement that calls __builtin_GOACC_data_end
      for cleanup.  */
-  tree data_end_fn = builtin_decl_explicit (BUILT_IN_GOACC_DATA_END);
-  gimple *call = gimple_build_call (data_end_fn, 0);
-  gimple_seq cleanup = NULL;
-  gimple_seq_add_stmt (&cleanup, call);
-  gimple *try_stmt = gimple_build_try (bind, cleanup, GIMPLE_TRY_FINALLY);
+  gimple *try_stmt = make_data_region_try_statement (loc, body);
   gimple_omp_set_body (data_region, try_stmt);
 
   return data_region;
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 887011e7d1f..7000b099bad 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,3 +1,10 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* c-c++-common/goacc/kernels-conversion.c: Test for a gang-single
+	region.
+	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
index c75db375f26..ec5db0201f3 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
@@ -18,6 +18,7 @@ main (void)
       sum += a[i];
 
     sum++;
+    a[0]++;
 
     #pragma acc loop
     for (i = 0; i < N; ++i)
@@ -27,10 +28,14 @@ main (void)
   return 0;
 }
 
-/* Check that the kernels region is split into a data region and an enclosed
-   parallel region.  */ 
+/* Check that the kernels region is split into a data region and enclosed
+   parallel regions.  */ 
 /* { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } } */
-/* { dg-final { scan-tree-dump-times "oacc_parallel" 1 "convert_oacc_kernels" } } */
+
+/* The two loop regions are parallelized, the sequential part in between is
+   made gang-single.  */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 2 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } } */
 
 /* Check that the original kernels region is removed.  */
 /* { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } } */
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
index 8c663302a6f..4aba2b1beeb 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
@@ -15,6 +15,7 @@ program main
   end do
 
   sum = sum + 1
+  a(1) = a(1) + 1
 
   !$acc loop
   do i = 1, N
@@ -24,10 +25,14 @@ program main
   !$acc end kernels
 end program main
 
-! Check that the kernels region is split into a data region and an enclosed
-! parallel region.
+! Check that the kernels region is split into a data region and enclosed
+! parallel regions.
 ! { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } }
-! { dg-final { scan-tree-dump-times "oacc_parallel" 1 "convert_oacc_kernels" } }
+
+! The two loop regions are parallelized, the sequential part in between is
+! made gang-single.
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 2 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } }
 
 ! Check that the original kernels region is removed.
 ! { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } }
-- 
2.17.1


[-- Attachment #6: 0005-Handle-conditional-execution-of-loops-in-OpenACC-ker.patch --]
[-- Type: text/x-diff, Size: 16850 bytes --]

From d66b17791ae1fb458834f0dd5a326acc7a5cd51a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerg=C3=B6=20Barany?= <gergo@codesourcery.com>
Date: Wed, 23 Jan 2019 14:32:57 -0800
Subject: [PATCH 5/9] Handle conditional execution of loops in OpenACC kernels
 regions

Any OpenACC loop controlled by an if statement or a non-OpenACC loop must be
executed in a gang-single region. Detecting such loops is not trivial as
OpenACC kernels expansion is done on GIMPLE but before computation of the
control flow graph. This patch adds an auxiliary analysis for determining
whether a statement is inside a conditionally executed region (relative to
the kernels region's entry).

    gcc/
    * omp-oacc-kernels.c (control_flow_regions): New class.
    (control_flow_regions::control_flow_regions): New constructor.
    (control_flow_regions::is_unconditional_oacc_for_loop): New method.
    (control_flow_regions::find_rep): Likewise.
    (control_flow_regions::union_reps): Likewise.
    (control_flow_regions::compute_regions): Likewise.
    (decompose_kernels_region_body): Use test for conditional execution.

    gcc/testsuite/
    * c-c++-common/goacc/kernels-conversion.c: Add test for conditionally
    executed code.
    * gfortran.dg/goacc/kernels-conversion.f95: Likewise.
---
 gcc/ChangeLog.openacc                         |  11 +
 gcc/omp-oacc-kernels.c                        | 216 +++++++++++++++++-
 gcc/testsuite/ChangeLog.openacc               |   7 +
 .../c-c++-common/goacc/kernels-conversion.c   |  20 +-
 .../gfortran.dg/goacc/kernels-conversion.f95  |  21 +-
 5 files changed, 263 insertions(+), 12 deletions(-)

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 6fa92ee2731..75eea0b1ef3 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,14 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* omp-oacc-kernels.c (control_flow_regions): New class.
+	(control_flow_regions::control_flow_regions): New constructor.
+	(control_flow_regions::is_unconditional_oacc_for_loop): New method.
+	(control_flow_regions::find_rep): Likewise.
+	(control_flow_regions::union_reps): Likewise.
+	(control_flow_regions::compute_regions): Likewise.
+	(decompose_kernels_region_body): Use test for conditional execution.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/omp-oacc-kernels.c b/gcc/omp-oacc-kernels.c
index 6e083666a17..80a82fa6e1d 100644
--- a/gcc/omp-oacc-kernels.c
+++ b/gcc/omp-oacc-kernels.c
@@ -385,6 +385,208 @@ maybe_build_inner_data_region (location_t loc, gimple *body,
   return body;
 }
 
+/* Auxiliary analysis of the body of a kernels region, to determine for each
+   OpenACC loop whether it is control-dependent (i.e., not necessarily
+   executed every time the kernels region is entered) or not.
+   We say that a loop is control-dependent if there is some cond, switch, or
+   goto statement that jumps over it, forwards or backwards.  For example,
+   if the loop is controlled by an if statement, then a jump to the true
+   block, the false block, or from one of those blocks to the control flow
+   join point will necessarily jump over the loop.
+   This analysis implements an ad-hoc union-find data structure classifying
+   statements into "control-flow regions" as follows: Most statements are in
+   the same region as their predecessor, except that each OpenACC loop is in
+   a region of its own, and each OpenACC loop's successor starts a new
+   region.  We then unite the regions of any statements linked by jumps,
+   placing any cond, switch, or goto statement in the same region as its
+   target label(s).
+   In the end, control dependence of OpenACC loops can be determined by
+   comparing their immediate predecessor and successor statements' regions.
+   A jump crosses the loop if and only if the predecessor and successor are
+   in the same region.  (If there is no predecessor or successor, the loop
+   is executed unconditionally.)
+   The methods in this class identify statements by their index in the
+   kernels region's body.  */
+
+class control_flow_regions
+{
+  public:
+    /* Initialize an instance and pre-compute the control-flow region
+       information for the statement sequence SEQ.  */
+    control_flow_regions (gimple_seq seq);
+
+    /* Return true if the STMT with the given index IDX in the analyzed
+       statement sequence is an unconditionally executed OpenACC loop.  */
+    bool is_unconditional_oacc_for_loop (gimple *stmt, size_t idx);
+
+  private:
+    /* Find the region representative for the statement identified by index
+       STMT_IDX.  */
+    size_t find_rep (size_t stmt_idx);
+
+    /* Union the regions containing the statements represented by
+       representatives A and B.  */
+    void union_reps (size_t a, size_t b);
+
+    /* Helper for the constructor.  Performs the actual computation of the
+       control-flow regions in the statement sequence SEQ.  */
+    void compute_regions (gimple_seq seq);
+
+    /* The mapping from statement indices to region representatives.  */
+    vec <size_t> representatives;
+
+    /* A cache mapping statement indices to a flag indicating whether the
+       statement is a top level OpenACC for loop.  */
+    vec <bool> omp_for_loops;
+};
+
+control_flow_regions::control_flow_regions (gimple_seq seq)
+{
+  representatives.create (1);
+  omp_for_loops.create (1);
+  compute_regions (seq);
+}
+
+bool
+control_flow_regions::is_unconditional_oacc_for_loop (gimple *stmt, size_t idx)
+{
+  if (top_level_omp_for_in_stmt (stmt) == NULL)
+    /* Not an OpenACC for loop.  */
+    return false;
+  if (idx == 0 || idx == representatives.length () - 1)
+    /* The first or last statement in the kernels region.  This means that
+       there is no room before or after it for a jump or a label.  Thus
+       there cannot be a jump across it, so it is unconditional.  */
+    return true;
+  /* Otherwise, the loop is unconditional if the statements before and after
+     it are in different control flow regions.  Scan forward and backward,
+     skipping over neighboring OpenACC for loops, to find these preceding
+     statements.  */
+  size_t prev_index = idx - 1;
+  while (prev_index > 0 && omp_for_loops [prev_index] == true)
+    prev_index--;
+  /* If all preceding statements are also OpenACC loops, all of these are
+     unconditional.  */
+  if (prev_index == 0)
+    return true;
+  size_t succ_index = idx + 1;
+  while (succ_index < omp_for_loops.length ()
+         && omp_for_loops [succ_index] == true)
+    succ_index++;
+  /* If all following statements are also OpenACC loops, all of these are
+     unconditional.  */
+  if (succ_index == omp_for_loops.length ())
+    return true;
+  return (find_rep (prev_index) != find_rep (succ_index));
+}
+
+size_t
+control_flow_regions::find_rep (size_t stmt_idx)
+{
+  size_t rep = stmt_idx, aux = stmt_idx;
+  /* Find the root representative of this statement.  */
+  while (representatives[rep] != rep)
+    rep = representatives[rep];
+  /* Compress the path from the original statement to the representative.  */
+  while (representatives[aux] != rep)
+    {
+      size_t tmp = representatives[aux];
+      representatives[aux] = rep;
+      aux = tmp;
+    }
+  return rep;
+}
+
+void
+control_flow_regions::union_reps (size_t a, size_t b)
+{
+  a = find_rep (a);
+  b = find_rep (b);
+  representatives[b] = a;
+}
+
+void
+control_flow_regions::compute_regions (gimple_seq seq)
+{
+  hash_map <gimple *, size_t> control_flow_reps;
+  hash_map <tree, size_t> label_reps;
+  size_t current_region = 0, idx = 0;
+
+  /* In a first pass, assign an initial region to each statement.  Except in
+     the case of OpenACC loops, each statement simply gets the same region
+     representative as its predecessor.  */
+  for (gimple_stmt_iterator gsi = gsi_start (seq);
+       !gsi_end_p (gsi);
+       gsi_next (&gsi))
+    {
+      gimple *stmt = gsi_stmt (gsi);
+      gimple *omp_for = top_level_omp_for_in_stmt (stmt);
+      omp_for_loops.safe_push (omp_for != NULL);
+      if (omp_for != NULL)
+        {
+          /* Assign a new region to this loop and to its successor.  */
+          current_region = idx;
+          representatives.safe_push (current_region);
+          current_region++;
+        }
+      else
+        {
+          representatives.safe_push (current_region);
+          /* Remember any jumps and labels for the second pass below.  */
+          if (gimple_code (stmt) == GIMPLE_COND
+              || gimple_code (stmt) == GIMPLE_SWITCH
+              || gimple_code (stmt) == GIMPLE_GOTO)
+            control_flow_reps.put (stmt, current_region);
+          else if (gimple_code (stmt) == GIMPLE_LABEL)
+            label_reps.put (gimple_label_label (as_a <glabel *> (stmt)),
+                            current_region);
+        }
+      idx++;
+    }
+  gcc_assert (representatives.length () == omp_for_loops.length ());
+
+  /* Revisit all the control flow statements and union the region of each
+     cond, switch, or goto statement with the target labels' regions.  */
+  for (hash_map <gimple *, size_t>::iterator it = control_flow_reps.begin ();
+       it != control_flow_reps.end ();
+       ++it)
+    {
+      gimple *stmt = (*it).first;
+      size_t stmt_rep = (*it).second;
+      switch (gimple_code (stmt))
+        {
+          tree label;
+          unsigned int n;
+
+        case GIMPLE_COND:
+          label = gimple_cond_true_label (as_a <gcond *> (stmt));
+          union_reps (stmt_rep, *label_reps.get (label));
+          label = gimple_cond_false_label (as_a <gcond *> (stmt));
+          union_reps (stmt_rep, *label_reps.get (label));
+          break;
+
+        case GIMPLE_SWITCH:
+          n = gimple_switch_num_labels (as_a <gswitch *> (stmt));
+          for (unsigned int i = 0; i < n; i++)
+            {
+              tree switch_case
+                = gimple_switch_label (as_a <gswitch *> (stmt), i);
+              label = CASE_LABEL (switch_case);
+              union_reps (stmt_rep, *label_reps.get (label));
+            }
+          break;
+
+        case GIMPLE_GOTO:
+          label = gimple_goto_dest (stmt);
+          union_reps (stmt_rep, *label_reps.get (label));
+          break;
+
+        default:
+          gcc_unreachable ();
+        }
+    }
+}
+
 /* Decompose the body of the KERNELS_REGION, which was originally annotated
    with the KERNELS_CLAUSES, into a series of parallel regions.  */
 
@@ -486,9 +688,14 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
      separated from the loop.  */
   bool only_simple_assignments = true;
 
+  /* Precompute the control flow region information to determine whether an
+     OpenACC loop is executed conditionally or unconditionally.  */
+  control_flow_regions cf_regions (body_sequence);
+
   /* Iterate over the statements in the kernels region's body.  */
+  size_t idx = 0;
   gimple_stmt_iterator gsi, gsi_n;
-  for (gsi = gsi_start (body_sequence); !gsi_end_p (gsi); gsi = gsi_n)
+  for (gsi = gsi_start (body_sequence); !gsi_end_p (gsi); gsi = gsi_n, idx++)
     {
       /* Advance the iterator here because otherwise it would be invalidated
          by moving statements below.  */
@@ -497,7 +704,8 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
 
       gimple *stmt = gsi_stmt (gsi);
       gimple *omp_for = top_level_omp_for_in_stmt (stmt);
-      if (omp_for != NULL)
+      if (omp_for != NULL
+          && cf_regions.is_unconditional_oacc_for_loop (stmt, idx))
         {
           /* This is an OMP for statement, put it into a parallel region.
              But first, construct a gang-single region containing any
@@ -532,8 +740,8 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
         }
       else
         {
-          /* This is not an OMP for statement, so it will be put into a
-             gang-single region.  */
+          /* This is not an unconditional OMP for statement, so it will be
+             put into a gang-single region.  */
           gimple_seq_add_stmt (&gang_single_seq, stmt);
           /* Is this a simple assignment? We call it simple if it is an
              assignment to an artificial local variable.  This captures
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 7000b099bad..7091f321ffd 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,3 +1,10 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* c-c++-common/goacc/kernels-conversion.c: Add test for conditionally
+	executed code.
+	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
index ec5db0201f3..ed4d6429c65 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
@@ -12,6 +12,7 @@ main (void)
   unsigned int sum = 1;
 
 #pragma acc kernels copyin(a[0:N]) copy(sum)
+  /* { dg-bogus "region contains gang partitoned code but is not gang partitioned" "gang partitioned" { xfail *-*-* } .-1 } */
   {
     #pragma acc loop
     for (i = 0; i < N; ++i)
@@ -23,6 +24,17 @@ main (void)
     #pragma acc loop
     for (i = 0; i < N; ++i)
       sum += a[i];
+
+    if (sum > 10)
+      { 
+        #pragma acc loop
+        for (i = 0; i < N; ++i)
+          sum += a[i];
+      }
+
+    #pragma acc loop
+    for (i = 0; i < N; ++i)
+      sum += a[i];
   }
 
   return 0;
@@ -32,10 +44,10 @@ main (void)
    parallel regions.  */ 
 /* { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } } */
 
-/* The two loop regions are parallelized, the sequential part in between is
-   made gang-single.  */
-/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 2 "convert_oacc_kernels" } } */
-/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } } */
+/* The three unconditional loop regions are parallelized, the sequential
+   part in between and the conditional loop are made gang-single.  */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 3 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 2 "convert_oacc_kernels" } } */
 
 /* Check that the original kernels region is removed.  */
 /* { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } } */
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
index 4aba2b1beeb..f89e46b4d3b 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
@@ -22,6 +22,19 @@ program main
     sum = sum + a(i)
   end do
 
+  if (sum .gt. 10) then
+    !$acc loop
+    do i = 1, N
+      sum = sum + a(i)
+    end do
+  end if
+
+  !$acc loop
+  ! { dg-bogus "region contains gang partitoned code but is not gang partitioned" "gang partitioned" { xfail *-*-* } .-1 }
+  do i = 1, N
+    sum = sum + a(i)
+  end do
+
   !$acc end kernels
 end program main
 
@@ -29,10 +42,10 @@ end program main
 ! parallel regions.
 ! { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } }
 
-! The two loop regions are parallelized, the sequential part in between is
-! made gang-single.
-! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 2 "convert_oacc_kernels" } }
-! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } }
+! The three unconditional loop regions are parallelized, the sequential part
+! in between and the conditional loop are made gang-single.
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 3 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 2 "convert_oacc_kernels" } }
 
 ! Check that the original kernels region is removed.
 ! { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } }
-- 
2.17.1


[-- Attachment #7: 0006-Adjust-parallelism-of-loops-in-gang-single-parts-of-.patch --]
[-- Type: text/x-diff, Size: 26194 bytes --]

From c7713be32fc5eace2b1e9c20447da849d23f6076 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerg=C3=B6=20Barany?= <gergo@codesourcery.com>
Date: Wed, 23 Jan 2019 22:11:11 -0800
Subject: [PATCH 6/9] Adjust parallelism of loops in gang-single parts of
 OpenACC kernels regions

Loops in gang-single parts of kernels regions cannot be executed in
gang-redundant mode. If the user specified gang clauses on such loops, emit
an error and remove these clauses. Adjust automatic partitioning to exclude
gang partitioning in gang-single regions.

    gcc/
    * omp-oacc-kernels.c (add_parent_or_loop_num_clause): New function.
    (adjust_nested_loop_clauses): Likewise.
    (transform_kernels_loop_clauses, make_gang_parallel_loop_region):
    Add worker and vector clause parameters, emit error on illegal
    nesting.
    (visit_loops_in_gang_single_region): Emit warning on conditionally
    executed code with a gang clause.
    (make_loops_gang_single): New function.
    (decompose_kernels_region_body): Separate out gang/worker/vector clauses
    for separate handling; add call to make_loops_gang_single.
    * omp-offload.c (oacc_loop_auto_partitions): Add and propagate
    is_oacc_gang_single parameter.
    (oacc_loop_partition): Likewise.
    (execute_oacc_device_lower): Adjust call to oacc_loop_partition.
---
 gcc/ChangeLog.openacc  |  18 ++
 gcc/omp-oacc-kernels.c | 375 +++++++++++++++++++++++++++++++++++++----
 gcc/omp-offload.c      |  22 ++-
 3 files changed, 377 insertions(+), 38 deletions(-)

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 75eea0b1ef3..f3fcbc88831 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,21 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* omp-oacc-kernels.c (add_parent_or_loop_num_clause): New function.
+	(adjust_nested_loop_clauses): Likewise.
+	(transform_kernels_loop_clauses, make_gang_parallel_loop_region):
+	Add worker and vector clause parameters, emit error on illegal
+	nesting.
+	(visit_loops_in_gang_single_region): Emit warning on conditionally
+	executed code with a gang clause.
+	(make_loops_gang_single): New function.
+	(decompose_kernels_region_body): Separate out gang/worker/vector clauses
+	for separate handling; add call to make_loops_gang_single.
+	* omp-offload.c (oacc_loop_auto_partitions): Add and propagate
+	is_oacc_gang_single parameter.
+	(oacc_loop_partition): Likewise.
+	(execute_oacc_device_lower): Adjust call to oacc_loop_partition.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/omp-oacc-kernels.c b/gcc/omp-oacc-kernels.c
index 80a82fa6e1d..c334502972c 100644
--- a/gcc/omp-oacc-kernels.c
+++ b/gcc/omp-oacc-kernels.c
@@ -59,7 +59,14 @@ along with GCC; see the file COPYING3.  If not see
    - Any sequences of other code (non-loops, non-OpenACC loops) are wrapped
      in new "gang-single" parallel regions: Worker/vector annotations are
      copied from the original kernels region if present, but num_gangs is
-     explicitly set to 1.  */
+     explicitly set to 1.
+   - Both points above only apply at the topmost level in the region, i.e.,
+     the transformation does not introduce new parallel regions inside
+     nested statement bodies.  In particular, this means that a
+     gang-parallelizable loop inside an if statement is "gang-serialized" by
+     the transformation.
+     The transformation visits loops inside such new gang-single-regions and
+     removes and warns about any gang annotations.  */
 
 /* Helper function for decompose_kernels_region_body.  If STMT contains a
    "top-level" OMP_FOR statement, returns a pointer to that statement;
@@ -122,6 +129,67 @@ top_level_omp_for_in_stmt (gimple *stmt)
   return NULL;
 }
 
+/* Helper function for make_loops_gang_single for walking the tree.  If the
+   statement indicated by GSI_P is an OpenACC for loop with a gang clause,
+   issue a warning and remove the clause.  */
+
+static tree
+visit_loops_in_gang_single_region (gimple_stmt_iterator *gsi_p,
+                                   bool *handled_ops_p,
+                                   struct walk_stmt_info *)
+{
+  gimple *stmt = gsi_stmt (*gsi_p);
+  tree clauses = NULL, prev_clause = NULL;
+  *handled_ops_p = false;
+
+  switch (gimple_code (stmt))
+    {
+    case GIMPLE_OMP_FOR:
+      clauses = gimple_omp_for_clauses (stmt);
+      for (tree clause = clauses; clause; clause = OMP_CLAUSE_CHAIN (clause))
+        {
+          if (OMP_CLAUSE_CODE (clause) == OMP_CLAUSE_GANG)
+            {
+              /* It makes no sense to have a gang clause in a gang-single
+                 region, so remove it and warn.  */
+              warning_at (gimple_location (stmt), 0,
+                          "conditionally executed loop in kernels region"
+                          " will be executed in a single gang;"
+                          " ignoring %<gang%> clause");
+              if (prev_clause != NULL)
+                OMP_CLAUSE_CHAIN (prev_clause) = OMP_CLAUSE_CHAIN (clause);
+              else
+                clauses = OMP_CLAUSE_CHAIN (clause);
+
+              break;
+            }
+          prev_clause = clause;
+        }
+      gimple_omp_for_set_clauses (stmt, clauses);
+      /* No need to recurse into nested statements; no loop nested inside
+         this loop can be gang-partitioned.  */
+      *handled_ops_p = true;
+      break;
+
+    default:
+      break;
+    }
+
+  return NULL;
+}
+
+/* Visit all nested OpenACC loops in the statement indicated by GSI.  This
+   statement is expected to be inside a gang-single region.  Issue a warning
+   for any loops inside it that have gang clauses and remove the clauses.  */
+
+static void
+make_loops_gang_single (gimple_stmt_iterator gsi)
+{
+  struct walk_stmt_info wi;
+  memset (&wi, 0, sizeof (wi));
+  walk_gimple_stmt (&gsi, visit_loops_in_gang_single_region, NULL, &wi);
+}
+
 /* Construct a "gang-single" OpenACC parallel region at LOC containing the
    STMTS.  The newly created region is annotated with CLAUSES, which must
    not contain a num_gangs clause, and an additional "num_gangs(1)" clause
@@ -150,45 +218,248 @@ make_gang_single_region (location_t loc, gimple_seq stmts, tree clauses)
   return single_region;
 }
 
+/* Helper function for make_gang_parallel_loop_region.  Adds a num_gangs
+   (num_workers, vector_length) clause to the given CLAUSES, either the one
+   from the parent region (PARENT_CLAUSE) or a new one based on the loop's
+   own LOOP_CLAUSE ("gang(num: N)" or similar for workers or vectors) with
+   the given CLAUSE_CODE.  Does nothing if neither PARENT_CLAUSE nor
+   LOOP_CLAUSE exist.  Returns the new clauses.  */
+
+static tree
+add_parent_or_loop_num_clause (tree parent_clause, tree loop_clause,
+                               omp_clause_code clause_code, tree clauses)
+{
+  if (parent_clause != NULL)
+    {
+      tree num_clause = unshare_expr (parent_clause);
+      OMP_CLAUSE_CHAIN (num_clause) = clauses;
+      clauses = num_clause;
+    }
+  else if (loop_clause != NULL)
+    {
+      /* The kernels region does not have a "num_gangs" clause, but the loop
+         itself had a "gang(num: N)" clause.  Honor it by adding a
+         "num_gangs(N)" clause on the parallel region.  */
+      tree num = OMP_CLAUSE_OPERAND (loop_clause, 0);
+      tree new_num_clause
+        = build_omp_clause (OMP_CLAUSE_LOCATION (loop_clause), clause_code);
+      OMP_CLAUSE_OPERAND (new_num_clause, 0) = num;
+      OMP_CLAUSE_CHAIN (new_num_clause) = clauses;
+      clauses = new_num_clause;
+    }
+  return clauses;
+}
+
+/* Helper for make_gang_parallel_loop_region, looking for "worker(num: N)"
+   or "vector(length: N)" clauses in nested loops.  Removes the numeric
+   argument, transferring it to the enclosing parallel region (via
+   WI->INFO).  If numeric arguments within the same loop nest conflict,
+   emits a warning.
+
+   This function also decides whether to add an auto clause on each of these
+   nested loops.  It adds an auto clause unless there is already an
+   independent/seq/auto clause or a gang/worker/vector annotation.  */
+
+static tree
+adjust_nested_loop_clauses (gimple_stmt_iterator *gsi_p, bool *,
+                            struct walk_stmt_info *wi)
+{
+  tree **clauses = (tree **) wi->info;
+  tree *gang_num_clause = clauses[GOMP_DIM_GANG];
+  tree *worker_num_clause = clauses[GOMP_DIM_WORKER];
+  tree *vector_length_clause = clauses[GOMP_DIM_VECTOR];
+  gimple *stmt = gsi_stmt (*gsi_p);
+
+  if (gimple_code (stmt) == GIMPLE_OMP_FOR)
+    {
+      bool add_auto_clause = true;
+      tree loop_clauses = gimple_omp_for_clauses (stmt);
+      tree loop_clause = loop_clauses;
+      for (; loop_clause; loop_clause = OMP_CLAUSE_CHAIN (loop_clause))
+        {
+          tree *outer_clause_ptr = NULL;
+          switch (OMP_CLAUSE_CODE (loop_clause))
+            {
+              case OMP_CLAUSE_GANG:
+                outer_clause_ptr = gang_num_clause;
+                break;
+              case OMP_CLAUSE_WORKER:
+                outer_clause_ptr = worker_num_clause;
+                break;
+              case OMP_CLAUSE_VECTOR:
+                outer_clause_ptr = vector_length_clause;
+                break;
+              case OMP_CLAUSE_INDEPENDENT:
+              case OMP_CLAUSE_SEQ:
+              case OMP_CLAUSE_AUTO:
+                add_auto_clause = false;
+              default:
+                break;
+            }
+          if (outer_clause_ptr != NULL)
+            {
+              if (OMP_CLAUSE_OPERAND (loop_clause, 0) != NULL
+                  && *outer_clause_ptr == NULL)
+                {
+                  /* Transfer the clause to the enclosing parallel region
+                     and remove the numerical argument from the loop.  */
+                  *outer_clause_ptr = unshare_expr (loop_clause);
+                  OMP_CLAUSE_OPERAND (loop_clause, 0) = NULL;
+                }
+              else if (OMP_CLAUSE_OPERAND (loop_clause, 0) != NULL &&
+                       OMP_CLAUSE_OPERAND (*outer_clause_ptr, 0) != NULL)
+                {
+                  /* See if both of these are the same constant.  If they
+                     aren't, emit a warning.  */
+                  tree old_op = OMP_CLAUSE_OPERAND (*outer_clause_ptr, 0);
+                  tree new_op = OMP_CLAUSE_OPERAND (loop_clause, 0);
+                  if (!(cst_and_fits_in_hwi (old_op) &&
+                        cst_and_fits_in_hwi (new_op) &&
+                        int_cst_value (old_op) == int_cst_value (new_op)))
+                    {
+                      const char *clause_name
+                        = omp_clause_code_name[OMP_CLAUSE_CODE (loop_clause)];
+                      error_at (gimple_location (stmt),
+                                "cannot honor conflicting %qs annotation",
+                                clause_name);
+                      inform (OMP_CLAUSE_LOCATION (*outer_clause_ptr),
+                              "location of the previous annotation "
+                              "in the same loop nest");
+                    }
+                  OMP_CLAUSE_OPERAND (loop_clause, 0) = NULL;
+                }
+            }
+        }
+      if (add_auto_clause)
+        {
+          tree auto_clause
+            = build_omp_clause (gimple_location (stmt), OMP_CLAUSE_AUTO);
+          OMP_CLAUSE_CHAIN (auto_clause) = loop_clauses;
+          gimple_omp_for_set_clauses (stmt, auto_clause);
+        }
+    }
+
+  return NULL;
+}
+
 /* Helper for make_region_loop_nest.  Transform OpenACC 'kernels'/'loop'
    construct clauses into OpenACC 'parallel'/'loop' construct ones.  */
 
 static tree
 transform_kernels_loop_clauses (gimple *omp_for,
 				tree num_gangs_clause,
+				tree num_workers_clause,
+				tree vector_length_clause,
 				tree clauses)
 {
   /* If this loop in a kernels region does not have an explicit
      "independent", "seq", or "auto" clause, we must give it an explicit
-     "auto" clause. */
+     "auto" clause.
+     We also check for "gang(num: N)" clauses.  These must not appear in
+     kernels regions that have their own "num_gangs" clause.  Otherwise, they
+     must be converted and put on the region; similarly for workers and
+     vectors.  */
   bool add_auto_clause = true;
+  tree loop_gang_clause = NULL, loop_worker_clause = NULL,
+       loop_vector_clause = NULL;
   tree loop_clauses = gimple_omp_for_clauses (omp_for);
-  for (tree c = loop_clauses; c; c = OMP_CLAUSE_CHAIN (c))
+  for (tree loop_clause = loop_clauses;
+       loop_clause;
+       loop_clause = OMP_CLAUSE_CHAIN (loop_clause))
     {
-      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_AUTO
-          || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_INDEPENDENT
-          || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_SEQ)
-        {
-          add_auto_clause = false;
-          break;
+      /* Look for gang, worker, and vector clauses.  */
+      bool found_num_clause = false;
+      tree *clause_ptr, clause_to_check;
+      switch (OMP_CLAUSE_CODE (loop_clause))
+         {
+          case OMP_CLAUSE_GANG:
+            found_num_clause = true;
+            clause_ptr = &loop_gang_clause;
+            clause_to_check = num_gangs_clause;
+            break;
+          case OMP_CLAUSE_WORKER:
+            found_num_clause = true;
+            clause_ptr = &loop_worker_clause;
+            clause_to_check = num_workers_clause;
+            break;
+          case OMP_CLAUSE_VECTOR:
+            found_num_clause = true;
+            clause_ptr = &loop_vector_clause;
+            clause_to_check = vector_length_clause;
+            break;
+          case OMP_CLAUSE_INDEPENDENT:
+          case OMP_CLAUSE_SEQ:
+          case OMP_CLAUSE_AUTO:
+            add_auto_clause = false;
+          default:
+            break;
         }
-    }
+      if (found_num_clause && OMP_CLAUSE_OPERAND (loop_clause, 0) != NULL)
+        {
+          if (clause_to_check)
+            {
+              const char *clause_name
+                = omp_clause_code_name[OMP_CLAUSE_CODE (loop_clause)];
+              const char *parent_clause_name
+                = omp_clause_code_name[OMP_CLAUSE_CODE (clause_to_check)];
+              error_at (OMP_CLAUSE_LOCATION (loop_clause),
+                        "argument not permitted on %qs clause"
+                        " in OpenACC %<kernels%> region with a %qs clause",
+                        clause_name, parent_clause_name);
+              inform (OMP_CLAUSE_LOCATION (clause_to_check),
+                      "location of OpenACC %<kernels%> region");
+            }
+          /* Copy the gang(N)/worker(N)/vector(N) clause to the enclosing
+             parallel region.  */
+          *clause_ptr = unshare_expr (loop_clause);
+          OMP_CLAUSE_CHAIN (*clause_ptr) = NULL;
+          /* Leave a gang/worker/vector clause on the loop, but without a
+             numeric argument.  */
+          OMP_CLAUSE_OPERAND (loop_clause, 0) = NULL;
+         }
+     }
   if (add_auto_clause)
     {
       tree auto_clause = build_omp_clause (gimple_location (omp_for),
                                            OMP_CLAUSE_AUTO);
       OMP_CLAUSE_CHAIN (auto_clause) = loop_clauses;
-      gimple_omp_for_set_clauses (omp_for, auto_clause);
-    }
-
-  /* If the kernels region had a num_gangs clause, add that to this new
-     parallel region.  */
-  if (num_gangs_clause != NULL)
-    {
-      tree parallel_num_gangs_clause = unshare_expr (num_gangs_clause);
-      OMP_CLAUSE_CHAIN (parallel_num_gangs_clause) = clauses;
-      clauses = parallel_num_gangs_clause;
+      loop_clauses = auto_clause;
     }
+  gimple_omp_for_set_clauses (omp_for, loop_clauses);
+  /* We must also recurse into the loop; it might contain nested loops
+     having their own "worker(num: W)" or "vector(length: V)" annotations.
+     Turn these into worker/vector annotations on the parallel region.  */
+  struct walk_stmt_info wi;
+  memset (&wi, 0, sizeof (wi));
+  tree *num_clauses[GOMP_DIM_MAX]
+    = { [GOMP_DIM_GANG] = &loop_gang_clause,
+        [GOMP_DIM_WORKER] = &loop_worker_clause,
+        [GOMP_DIM_VECTOR] = &loop_vector_clause };
+  wi.info = num_clauses;
+  gimple *body = gimple_omp_body (omp_for);
+  walk_gimple_seq (body, adjust_nested_loop_clauses, NULL, &wi);
+  /* Check if there were conflicting numbers of workers or vector lanes.  */
+  if (loop_gang_clause != NULL &&
+      OMP_CLAUSE_OPERAND (loop_gang_clause, 0) == NULL)
+    loop_gang_clause = NULL;
+  if (loop_worker_clause != NULL &&
+      OMP_CLAUSE_OPERAND (loop_worker_clause, 0) == NULL)
+    loop_worker_clause = NULL;
+  if (loop_vector_clause != NULL &&
+      OMP_CLAUSE_OPERAND (loop_vector_clause, 0) == NULL)
+    vector_length_clause = NULL;
+
+  /* If the kernels region had num_gangs, num_worker, vector_length clauses,
+     add these to this new parallel region.  */
+  clauses
+    = add_parent_or_loop_num_clause (num_gangs_clause, loop_gang_clause,
+				     OMP_CLAUSE_NUM_GANGS, clauses);
+  clauses
+    = add_parent_or_loop_num_clause (num_workers_clause, loop_worker_clause,
+				     OMP_CLAUSE_NUM_WORKERS, clauses);
+  clauses
+    = add_parent_or_loop_num_clause (vector_length_clause, loop_vector_clause,
+				     OMP_CLAUSE_VECTOR_LENGTH, clauses);
 
   return clauses;
 }
@@ -197,18 +468,33 @@ transform_kernels_loop_clauses (gimple *omp_for,
    STMT, which must be identical to, or a bind containing, the loop OMP_FOR
    with OpenACC loop annotations.
 
-   The newly created region is annotated with the optional NUM_GANGS_CLAUSE
-   as well as the other CLAUSES, which must not contain a num_gangs clause.  */
+   The NUM_GANGS_CLAUSE, NUM_WORKERS_CLAUSE, and VECTOR_LENGTH_CLAUSE are
+   optional clauses from the original kernels region and must not be
+   contained in the other CLAUSES. The newly created region is annotated
+   with the optional NUM_GANGS_CLAUSE as well as the other CLAUSES. If there
+   is no NUM_GANGS_CLAUSE but the loop has a "gang(num: N)" clause, that is
+   converted to a "num_gangs(N)" clause on the new region, and similarly for
+   workers and vectors.
+
+   The outermost loop gets an auto clause unless there already is an
+   independent/seq/auto clause or a gang/worker/vector annotation.  Nested
+   loops inside OMP_FOR are treated similarly by the
+   adjust_nested_loop_clauses function.  */
 
 static gimple *
 make_gang_parallel_loop_region (gimple *omp_for, gimple *stmt,
-                                tree num_gangs_clause, tree clauses)
+                                tree num_gangs_clause,
+                                tree num_workers_clause,
+                                tree vector_length_clause,
+                                tree clauses)
 {
   /* This correctly unshares the entire clause chain rooted here.  */
   clauses = unshare_expr (clauses);
 
   clauses = transform_kernels_loop_clauses (omp_for,
 					    num_gangs_clause,
+					    num_workers_clause,
+					    vector_length_clause,
 					    clauses);
 
   /* Now build the parallel region containing this loop.  */
@@ -596,23 +882,43 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
   location_t loc = gimple_location (kernels_region);
 
   /* The kernels clauses will be propagated to the child clauses unmodified,
-     except that that num_gangs clause will only be added to loop regions.
-     The other regions are "gang-single" and get an explicit num_gangs(1)
-     clause.  So separate out the num_gangs clause here.  */
-  tree num_gangs_clause = NULL, prev_clause = NULL;
+     except that the num_gangs, num_workers, and vector_length clauses will
+     only be added to loop regions.  The other regions are "gang-single" and
+     get an explicit num_gangs(1) clause.  So separate out the num_gangs,
+     num_workers, and vector_length clauses here.  */
+  tree num_gangs_clause = NULL, num_workers_clause = NULL,
+       vector_length_clause = NULL;
+  tree prev_clause = NULL, next_clause = NULL;
   tree parallel_clauses = kernels_clauses;
-  for (tree c = parallel_clauses; c; c = OMP_CLAUSE_CHAIN (c))
+  for (tree c = parallel_clauses; c; c = next_clause)
     {
-      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NUM_GANGS)
+      /* Preserve this here, as we might NULL it later.  */
+      next_clause = OMP_CLAUSE_CHAIN (c);
+
+      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NUM_GANGS
+          || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NUM_WORKERS
+          || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_VECTOR_LENGTH)
         {
           /* Cut this clause out of the chain.  */
-          num_gangs_clause = c;
           if (prev_clause != NULL)
             OMP_CLAUSE_CHAIN (prev_clause) = OMP_CLAUSE_CHAIN (c);
           else
             kernels_clauses = OMP_CLAUSE_CHAIN (c);
-          OMP_CLAUSE_CHAIN (num_gangs_clause) = NULL;
-          break;
+          OMP_CLAUSE_CHAIN (c) = NULL;
+          switch (OMP_CLAUSE_CODE (c))
+            {
+              case OMP_CLAUSE_NUM_GANGS:
+                num_gangs_clause = c;
+                break;
+              case OMP_CLAUSE_NUM_WORKERS:
+                num_workers_clause = c;
+                break;
+              case OMP_CLAUSE_VECTOR_LENGTH:
+                vector_length_clause = c;
+                break;
+              default:
+                gcc_unreachable ();
+            }
         }
       else
         prev_clause = c;
@@ -735,6 +1041,8 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
           gimple *parallel_region
             = make_gang_parallel_loop_region (omp_for, stmt,
                                               num_gangs_clause,
+                                              num_workers_clause,
+                                              vector_length_clause,
                                               kernels_clauses);
           gimple_seq_add_stmt (&region_body, parallel_region);
         }
@@ -752,6 +1060,9 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
                 && DECL_ARTIFICIAL (gimple_assign_lhs (stmt)));
           if (!is_simple_assignment)
             only_simple_assignments = false;
+          /* Remove and issue warnings about gang clauses on any OpenACC
+             loops nested inside this sequentially executed statement.  */
+          make_loops_gang_single (gsi);
         }
     }
 
diff --git a/gcc/omp-offload.c b/gcc/omp-offload.c
index 2d265c22c3c..a8dc57a92c6 100644
--- a/gcc/omp-offload.c
+++ b/gcc/omp-offload.c
@@ -1339,7 +1339,7 @@ oacc_loop_fixed_partitions (oacc_loop *loop, unsigned outer_mask)
 
 static unsigned
 oacc_loop_auto_partitions (oacc_loop *loop, unsigned outer_mask,
-			   bool outer_assign)
+			   bool outer_assign, bool is_oacc_gang_single)
 {
   bool assign = (loop->flags & OLF_AUTO) && (loop->flags & OLF_INDEPENDENT);
   bool noisy = true;
@@ -1357,6 +1357,10 @@ oacc_loop_auto_partitions (oacc_loop *loop, unsigned outer_mask,
 	 non-innermost available level.  */
       unsigned this_mask = GOMP_DIM_MASK (GOMP_DIM_GANG);
 
+      /* Gang partitioning is not available in a gang-single region.  */
+      if (is_oacc_gang_single)
+        this_mask = GOMP_DIM_MASK (GOMP_DIM_WORKER);
+
       /* Orphan reductions cannot have gang partitioning.  */
       if ((loop->flags & OLF_REDUCTION)
 	  && oacc_get_fn_attrib (current_function_decl)
@@ -1394,7 +1398,8 @@ oacc_loop_auto_partitions (oacc_loop *loop, unsigned outer_mask,
     {
       unsigned tmp_mask = outer_mask | loop->mask | loop->e_mask;
       loop->inner = oacc_loop_auto_partitions (loop->child, tmp_mask,
-					       outer_assign | assign);
+					       outer_assign | assign,
+					       is_oacc_gang_single);
     }
 
   if (assign && (!loop->mask || (tiling && !loop->e_mask) || !outer_assign))
@@ -1455,7 +1460,8 @@ oacc_loop_auto_partitions (oacc_loop *loop, unsigned outer_mask,
 
   if (loop->sibling)
     inner_mask |= oacc_loop_auto_partitions (loop->sibling,
-					     outer_mask, outer_assign);
+					     outer_mask, outer_assign,
+					     is_oacc_gang_single);
 
   inner_mask |= loop->inner | loop->mask | loop->e_mask;
 
@@ -1466,14 +1472,16 @@ oacc_loop_auto_partitions (oacc_loop *loop, unsigned outer_mask,
    axes.  Return mask of partitioning.  */
 
 static unsigned
-oacc_loop_partition (oacc_loop *loop, unsigned outer_mask)
+oacc_loop_partition (oacc_loop *loop, unsigned outer_mask,
+                     bool is_oacc_gang_single)
 {
   unsigned mask_all = oacc_loop_fixed_partitions (loop, outer_mask);
 
   if (mask_all & GOMP_DIM_MASK (GOMP_DIM_MAX))
     {
       mask_all ^= GOMP_DIM_MASK (GOMP_DIM_MAX);
-      mask_all |= oacc_loop_auto_partitions (loop, outer_mask, false);
+      mask_all |= oacc_loop_auto_partitions (loop, outer_mask, false,
+                                             is_oacc_gang_single);
     }
   return mask_all;
 }
@@ -1652,7 +1660,9 @@ execute_oacc_device_lower ()
     }
 
   unsigned outer_mask = fn_level >= 0 ? GOMP_DIM_MASK (fn_level) - 1 : 0;
-  unsigned used_mask = oacc_loop_partition (loops, outer_mask);
+  unsigned used_mask = oacc_loop_partition (loops, outer_mask,
+                                            is_oacc_parallel_kernels_gang_single);
+
   /* OpenACC kernels constructs are special: they currently don't use the
      generic oacc_loop infrastructure and attribute/dimension processing.  */
   if (is_oacc_kernels && is_oacc_kernels_parallelized)
-- 
2.17.1


[-- Attachment #8: 0007-Launch-kernels-asynchronously-in-OpenACC-kernels-reg.patch --]
[-- Type: text/x-diff, Size: 9342 bytes --]

From 5f30851a2d4c1a6e631e5b7870f8533811e50875 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Gerg=C3=B6=20Barany?= <gergo@codesourcery.com>
Date: Mon, 21 Jan 2019 12:50:14 -0800
Subject: [PATCH 7/9] Launch kernels asynchronously in OpenACC kernels regions

Kernels regions are decomposed into one or more smaller regions that are to
be executed in sequence. With this patch, all of these regions are launched
asynchronously, and a wait directive is added after them. This means that
the host only waits once for the kernels to complete, not once per kernel.
If the original kernels region was marked async, that asynchronous behavior
is preserved, and no wait is added.

    gcc/
    * omp-oacc-kernels.c (add_async_clauses_and_wait): New function...
    (decompose_kernels_region_body): ... called from here.

    gcc/testsuite/
    * c-c++-common/goacc/kernels-conversion.c: Test automatically generated
    async clauses.
    * gfortran.dg/goacc/kernels-conversion.f95: Likewise.
---
 gcc/ChangeLog.openacc                         |  6 ++
 gcc/omp-oacc-kernels.c                        | 56 ++++++++++++++++++-
 gcc/testsuite/ChangeLog.openacc               |  7 +++
 .../c-c++-common/goacc/kernels-conversion.c   |  5 ++
 .../gfortran.dg/goacc/kernels-conversion.f95  |  5 ++
 5 files changed, 76 insertions(+), 3 deletions(-)

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index f3fcbc88831..307922a29c0 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,9 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* omp-oacc-kernels.c (add_async_clauses_and_wait): New function...
+	(decompose_kernels_region_body): ... called from here.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/omp-oacc-kernels.c b/gcc/omp-oacc-kernels.c
index c334502972c..f8553f77708 100644
--- a/gcc/omp-oacc-kernels.c
+++ b/gcc/omp-oacc-kernels.c
@@ -66,7 +66,13 @@ along with GCC; see the file COPYING3.  If not see
      gang-parallelizable loop inside an if statement is "gang-serialized" by
      the transformation.
      The transformation visits loops inside such new gang-single-regions and
-     removes and warns about any gang annotations.  */
+     removes and warns about any gang annotations.
+   - In order to make the host wait only once for the whole region instead
+     of once per kernel launch, the new parallel and serial regions are
+     annotated async.  Unless the original kernels region was marked async,
+     the entire region ends with a wait construct.  If the original kernels
+     region was marked async, the generated async statements use the async
+     queue the kernels region was annotated with (possibly implicitly).  */
 
 /* Helper function for decompose_kernels_region_body.  If STMT contains a
    "top-level" OMP_FOR statement, returns a pointer to that statement;
@@ -671,6 +677,38 @@ maybe_build_inner_data_region (location_t loc, gimple *body,
   return body;
 }
 
+/* Helper function of decompose_kernels_region_body.  The statements in
+   REGION_BODY are expected to be decomposed parallel regions; add an
+   "async" clause to each.  Also add a "wait" pragma at the end of the
+   sequence.  */
+
+static void
+add_async_clauses_and_wait (location_t loc, gimple_seq *region_body)
+{
+  tree default_async_queue
+    = build_int_cst (integer_type_node, GOMP_ASYNC_NOVAL);
+  for (gimple_stmt_iterator gsi = gsi_start (*region_body);
+       !gsi_end_p (gsi);
+       gsi_next (&gsi))
+    {
+      gimple *stmt = gsi_stmt (gsi);
+      tree target_clauses = gimple_omp_target_clauses (stmt);
+      tree new_async_clause = build_omp_clause (loc, OMP_CLAUSE_ASYNC);
+      OMP_CLAUSE_OPERAND (new_async_clause, 0) = default_async_queue;
+      OMP_CLAUSE_CHAIN (new_async_clause) = target_clauses;
+      target_clauses = new_async_clause;
+      gimple_omp_target_set_clauses (as_a <gomp_target *> (stmt),
+                                     target_clauses);
+    }
+  /* A "#pragma acc wait" is just a call GOACC_wait (acc_async_sync, 0).  */
+  tree wait_fn = builtin_decl_explicit (BUILT_IN_GOACC_WAIT);
+  tree sync_arg = build_int_cst (integer_type_node, GOMP_ASYNC_SYNC);
+  gimple *wait_call = gimple_build_call (wait_fn, 2,
+                                         sync_arg, integer_zero_node);
+  gimple_set_location (wait_call, loc);
+  gimple_seq_add_stmt (region_body, wait_call);
+}
+
 /* Auxiliary analysis of the body of a kernels region, to determine for each
    OpenACC loop whether it is control-dependent (i.e., not necessarily
    executed every time the kernels region is entered) or not.
@@ -885,10 +923,12 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
      except that the num_gangs, num_workers, and vector_length clauses will
      only be added to loop regions.  The other regions are "gang-single" and
      get an explicit num_gangs(1) clause.  So separate out the num_gangs,
-     num_workers, and vector_length clauses here.  */
+     num_workers, and vector_length clauses here.
+     Also check for the presence of an async clause but do not remove it
+     from the kernels clauses.  */
   tree num_gangs_clause = NULL, num_workers_clause = NULL,
        vector_length_clause = NULL;
-  tree prev_clause = NULL, next_clause = NULL;
+  tree prev_clause = NULL, next_clause = NULL, async_clause = NULL;
   tree parallel_clauses = kernels_clauses;
   for (tree c = parallel_clauses; c; c = next_clause)
     {
@@ -922,6 +962,8 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
         }
       else
         prev_clause = c;
+      if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_ASYNC)
+        async_clause = c;
     }
 
   gimple *kernels_body = gimple_omp_body (kernels_region);
@@ -1085,6 +1127,14 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
       gimple_seq_add_stmt (&region_body, single_region);
     }
 
+  /* We want to launch these kernels asynchronously.  If the original
+     kernels region had an async clause, this is done automatically because
+     that async clause was copied to the individual regions we created.
+     Otherwise, add an async clause to each newly created region, as well as
+     a wait directive at the end.  */
+  if (async_clause == NULL)
+    add_async_clauses_and_wait (loc, &region_body);
+
   tree kernels_locals = gimple_bind_vars (as_a <gbind *> (kernels_body));
   gimple *body = gimple_build_bind (kernels_locals, region_body,
                                     make_node (BLOCK));
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 7091f321ffd..84d345fdecc 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,3 +1,10 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* c-c++-common/goacc/kernels-conversion.c: Test automatically generated
+	async clauses.
+	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
index ed4d6429c65..3e52ec4f16f 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
@@ -49,5 +49,10 @@ main (void)
 /* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 3 "convert_oacc_kernels" } } */
 /* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 2 "convert_oacc_kernels" } } */
 
+/* Each of the parallel regions is async, and there is a final call to
+   __builtin_GOACC_wait.  */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels.* async\(-1\)" 5 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "__builtin_GOACC_wait" 1 "convert_oacc_kernels" } } */
+
 /* Check that the original kernels region is removed.  */
 /* { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } } */
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
index f89e46b4d3b..559916c2325 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
@@ -47,5 +47,10 @@ end program main
 ! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 3 "convert_oacc_kernels" } }
 ! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 2 "convert_oacc_kernels" } }
 
+! Each of the parallel regions is async, and there is a final call to
+! __builtin_GOACC_wait.
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels.* async\(-1\)" 5 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "__builtin_GOACC_wait" 1 "convert_oacc_kernels" } }
+
 ! Check that the original kernels region is removed.
 ! { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } }
-- 
2.17.1


[-- Attachment #9: 0008-New-OpenACC-kernels-region-decompose-algorithm.patch --]
[-- Type: text/x-diff, Size: 39630 bytes --]

From d008a745628e5fc568e34dd959597c48e36e126b Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Thu, 24 Jan 2019 08:40:03 -0800
Subject: [PATCH 8/9] New OpenACC kernels region decompose algorithm

Previously, OpenACC kernels region bodies were decomposed into a sequence of
alternating gang-single and gang-parallel "parallel" regions. The new
algorithm in this patch introduces a third possibility: Loops that look like
they might benefit from the parloops pass are converted into old "kernels"
regions, exposing them to the parloops pass later on. This has the benefit
that loops that cannot be parallelized are not offloaded to the GPU.

	gcc/
	* omp-oacc-kernels.c (adjust_region_code_walk_stmt_fn)
	(adjust_region_code): New functions.
	(make_loops_gang_single): Update.
	(make_gang_single_region): Rename to...
	(make_region_seq): ... this, and update.
	(make_gang_parallel_loop_region): Rename to...
	(make_region_loop_nest): ... this, and update.
	(is_unconditional_oacc_for_loop): Remove stmt parameter and check.
	(decompose_kernels_region_body): Update.
	gcc/testsuite/
	* c-c++-common/goacc/kernels-conversion.c: Adjust test.
	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
	* c-c++-common/goacc/kernels-decompose-1.c: New file.
	* gfortran.dg/goacc/kernels-decompose-1.f95: Likewise.
	libgomp/
	* testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c: New
	file.
---
 gcc/ChangeLog.openacc                         |  13 +
 gcc/omp-oacc-kernels.c                        | 287 +++++++++++++++---
 gcc/testsuite/ChangeLog.openacc               |   8 +
 .../c-c++-common/goacc/kernels-conversion.c   |  19 +-
 .../c-c++-common/goacc/kernels-decompose-1.c  | 123 ++++++++
 .../gfortran.dg/goacc/kernels-conversion.f95  |  22 +-
 .../gfortran.dg/goacc/kernels-decompose-1.f95 | 132 ++++++++
 libgomp/ChangeLog.openacc                     |   3 +
 .../kernels-decompose-1.c                     |  30 ++
 9 files changed, 571 insertions(+), 66 deletions(-)
 create mode 100644 gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c
 create mode 100644 gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95
 create mode 100644 libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 307922a29c0..3ef97adef47 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,16 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* omp-oacc-kernels.c (adjust_region_code_walk_stmt_fn)
+	(adjust_region_code): New functions.
+	(make_loops_gang_single): Update.
+	(make_gang_single_region): Rename to...
+	(make_region_seq): ... this, and update.
+	(make_gang_parallel_loop_region): Rename to...
+	(make_region_loop_nest): ... this, and update.
+	(is_unconditional_oacc_for_loop): Remove stmt parameter and check.
+	(decompose_kernels_region_body): Update.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/omp-oacc-kernels.c b/gcc/omp-oacc-kernels.c
index f8553f77708..a8860c98e11 100644
--- a/gcc/omp-oacc-kernels.c
+++ b/gcc/omp-oacc-kernels.c
@@ -39,6 +39,7 @@ along with GCC; see the file COPYING3.  If not see
 #include "gimple-iterator.h"
 #include "gimple-walk.h"
 #include "gomp-constants.h"
+#include "omp-general.h"
 
 /* This is a preprocessing pass to be run immediately before lower_omp.  It
    will convert OpenACC "kernels" regions into sequences of "parallel"
@@ -135,6 +136,95 @@ top_level_omp_for_in_stmt (gimple *stmt)
   return NULL;
 }
 
+/* Helper for adjust_region_code: evaluate the statement at GSI_P.  */
+
+static tree
+adjust_region_code_walk_stmt_fn (gimple_stmt_iterator *gsi_p,
+				 bool *handled_ops_p,
+				 struct walk_stmt_info *wi)
+{
+  int *region_code = (int *) wi->info;
+
+  gimple *stmt = gsi_stmt (*gsi_p);
+  switch (gimple_code (stmt))
+    {
+    case GIMPLE_OMP_FOR:
+      {
+	tree clauses = gimple_omp_for_clauses (stmt);
+	if (omp_find_clause (clauses, OMP_CLAUSE_INDEPENDENT))
+	  {
+	    /* Explicit 'independent' clause.  */
+	    /* Keep going; recurse into loop body.  */
+	    break;
+	  }
+	else if (omp_find_clause (clauses, OMP_CLAUSE_SEQ))
+	  {
+	    /* Explicit 'seq' clause.  */
+	    /* We'll "parallelize" if at some level a loop construct has been
+	       marked up by the user as unparallelizable ('seq' clause; we'll
+	       respect that in the later processing).  Given that the user has
+	       explicitly marked it up, this loop construct cannot be
+	       performance-critical (and we thus don't have to "avoid
+	       offloading"), and in this case it's also fine to "parallelize"
+	       instead of "gang-single", because any outer or inner loops may
+	       still exploit the available parallelism.  */
+	    /* Keep going; recurse into loop body.  */
+	    break;
+	  }
+	else
+	  {
+	    /* Explicit or implicit 'auto' clause.  */
+	    /* The user would like this loop analyzed ('auto' clause) and
+	       typically parallelized, but we don't have available yet the
+	       compiler logic to analyze this, so can't parallelize it here, so
+	       we'd very likely be running into a performance problem if we
+	       were to execute this unparallelized, thus forward the whole loop
+	       nest to "parloops".  */
+	    *region_code = GF_OMP_TARGET_KIND_OACC_KERNELS;
+	    /* Terminate: final decision for this region.  */
+	    *handled_ops_p = true;
+	    return integer_zero_node;
+	  }
+	gcc_unreachable ();
+      }
+
+    case GIMPLE_COND:
+    case GIMPLE_GOTO:
+    case GIMPLE_SWITCH:
+    case GIMPLE_ASM:
+    case GIMPLE_TRANSACTION:
+    case GIMPLE_RETURN:
+      /* Statement that might constitute some looping/control flow pattern.  */
+      /* The user would like this code analyzed (implicit inside a 'kernels'
+	 region) and typically parallelized, but we don't have available yet
+	 the compiler logic to analyze this, so can't parallelize it here, so
+	 we'd very likely be running into a performance problem if we were to
+	 execute this unparallelized, thus forward the whole thing to
+	 "parloops".  */
+      *region_code = GF_OMP_TARGET_KIND_OACC_KERNELS;
+      /* Terminate: final decision for this region.  */
+      *handled_ops_p = true;
+      return integer_zero_node;
+
+    default:
+      /* Keep going.  */
+      break;
+    }
+
+  return NULL;
+}
+
+/* Adjust the REGION_CODE for the region in GS.  */
+
+static void
+adjust_region_code (gimple_seq gs, int *region_code)
+{
+  struct walk_stmt_info wi;
+  memset (&wi, 0, sizeof (wi));
+  wi.info = region_code;
+  walk_gimple_seq (gs, adjust_region_code_walk_stmt_fn, NULL, &wi);
+}
+
 /* Helper function for make_loops_gang_single for walking the tree.  If the
    statement indicated by GSI_P is an OpenACC for loop with a gang clause,
    issue a warning and remove the clause.  */
@@ -174,6 +264,7 @@ visit_loops_in_gang_single_region (gimple_stmt_iterator *gsi_p,
       gimple_omp_for_set_clauses (stmt, clauses);
       /* No need to recurse into nested statements; no loop nested inside
          this loop can be gang-partitioned.  */
+      sorry ("'gang' loop in \"gang-single\" region");
       *handled_ops_p = true;
       break;
 
@@ -184,16 +275,16 @@ visit_loops_in_gang_single_region (gimple_stmt_iterator *gsi_p,
   return NULL;
 }
 
-/* Visit all nested OpenACC loops in the statement indicated by GSI.  This
+/* Visit all nested OpenACC loops in the sequence indicated by GS.  This
    statement is expected to be inside a gang-single region.  Issue a warning
    for any loops inside it that have gang clauses and remove the clauses.  */
 
 static void
-make_loops_gang_single (gimple_stmt_iterator gsi)
+make_loops_gang_single (gimple_seq gs)
 {
   struct walk_stmt_info wi;
   memset (&wi, 0, sizeof (wi));
-  walk_gimple_stmt (&gsi, visit_loops_in_gang_single_region, NULL, &wi);
+  walk_gimple_seq (gs, visit_loops_in_gang_single_region, NULL, &wi);
 }
 
 /* Construct a "gang-single" OpenACC parallel region at LOC containing the
@@ -202,21 +293,73 @@ make_loops_gang_single (gimple_stmt_iterator gsi)
    to force gang-single execution.  */
 
 static gimple *
-make_gang_single_region (location_t loc, gimple_seq stmts, tree clauses)
+make_region_seq (location_t loc, gimple_seq stmts,
+		 tree num_gangs_clause,
+		 tree num_workers_clause,
+		 tree vector_length_clause,
+		 tree clauses)
 {
   /* This correctly unshares the entire clause chain rooted here.  */
   clauses = unshare_expr (clauses);
-  /* Make a num_gangs(1) clause.  */
-  tree gang_single_clause = build_omp_clause (loc, OMP_CLAUSE_NUM_GANGS);
-  OMP_CLAUSE_OPERAND (gang_single_clause, 0) = integer_one_node;
-  OMP_CLAUSE_CHAIN (gang_single_clause) = clauses;
+
+  location_t loc_stmts_first = gimple_location (gimple_seq_first (stmts));
+
+  /* Figure out the region code for this region.  */
+  /* Optimistic default: assume "setup code", no looping; thus not
+     performance-critical.  */
+  int region_code = GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE;
+  adjust_region_code (stmts, &region_code);
+
+  if (region_code == GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE)
+    {
+      dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, loc_stmts_first,
+		       "beginning \"gang-single\" region in OpenACC 'kernels'"
+		       " construct\n");
+
+      /* Make a num_gangs(1) clause.  */
+      tree gang_single_clause = build_omp_clause (loc, OMP_CLAUSE_NUM_GANGS);
+      OMP_CLAUSE_OPERAND (gang_single_clause, 0) = integer_one_node;
+      OMP_CLAUSE_CHAIN (gang_single_clause) = clauses;
+      clauses = gang_single_clause;
+
+      /* Remove and issue warnings about gang clauses on any OpenACC
+	 loops nested inside this sequentially executed statement.  */
+      make_loops_gang_single (stmts);
+    }
+  else if (region_code == GF_OMP_TARGET_KIND_OACC_KERNELS)
+    {
+      dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, loc_stmts_first,
+		       "beginning \"parloops\" region in OpenACC 'kernels'"
+		       " construct\n");
+
+      /* As we're transforming a "GF_OMP_TARGET_KIND_OACC_KERNELS" into another
+	 "GF_OMP_TARGET_KIND_OACC_KERNELS", this isn't doing any of the clauses
+	 mangling that "make_region_loop_nest" is doing.  */
+      /* Re-assemble the clauses stripped off earlier.  */
+      if (num_gangs_clause != NULL)
+	{
+	  tree c = unshare_expr (num_gangs_clause);
+	  OMP_CLAUSE_CHAIN (c) = clauses;
+	  clauses = c;
+	}
+      if (num_workers_clause != NULL)
+	{
+	  tree c = unshare_expr (num_workers_clause);
+	  OMP_CLAUSE_CHAIN (c) = clauses;
+	  clauses = c;
+	}
+      if (vector_length_clause != NULL)
+	{
+	  tree c = unshare_expr (vector_length_clause);
+	  OMP_CLAUSE_CHAIN (c) = clauses;
+	  clauses = c;
+	}
+    }
+  else
+    gcc_unreachable ();
 
   /* Build the gang-single region.  */
-  gimple *single_region
-    = gimple_build_omp_target (
-        NULL,
-        GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_GANG_SINGLE,
-        gang_single_clause);
+  gimple *single_region = gimple_build_omp_target (NULL, region_code, clauses);
   gimple_set_location (single_region, loc);
   gbind *single_body = gimple_build_bind (NULL, stmts, make_node (BLOCK));
   gimple_omp_set_body (single_region, single_body);
@@ -224,7 +367,7 @@ make_gang_single_region (location_t loc, gimple_seq stmts, tree clauses)
   return single_region;
 }
 
-/* Helper function for make_gang_parallel_loop_region.  Adds a num_gangs
+/* Helper function for make_region_loop_nest.  Adds a num_gangs
    (num_workers, vector_length) clause to the given CLAUSES, either the one
    from the parent region (PARENT_CLAUSE) or a new one based on the loop's
    own LOOP_CLAUSE ("gang(num: N)" or similar for workers or vectors) with
@@ -256,7 +399,7 @@ add_parent_or_loop_num_clause (tree parent_clause, tree loop_clause,
   return clauses;
 }
 
-/* Helper for make_gang_parallel_loop_region, looking for "worker(num: N)"
+/* Helper for make_region_loop_nest, looking for "worker(num: N)"
    or "vector(length: N)" clauses in nested loops.  Removes the numeric
    argument, transferring it to the enclosing parallel region (via
    WI->INFO).  If numeric arguments within the same loop nest conflict,
@@ -488,32 +631,63 @@ transform_kernels_loop_clauses (gimple *omp_for,
    adjust_nested_loop_clauses function.  */
 
 static gimple *
-make_gang_parallel_loop_region (gimple *omp_for, gimple *stmt,
-                                tree num_gangs_clause,
-                                tree num_workers_clause,
-                                tree vector_length_clause,
-                                tree clauses)
+make_region_loop_nest (gimple *omp_for, gimple_seq stmts,
+		       tree num_gangs_clause,
+		       tree num_workers_clause,
+		       tree vector_length_clause,
+		       tree clauses)
 {
   /* This correctly unshares the entire clause chain rooted here.  */
   clauses = unshare_expr (clauses);
 
-  clauses = transform_kernels_loop_clauses (omp_for,
-					    num_gangs_clause,
-					    num_workers_clause,
-					    vector_length_clause,
-					    clauses);
+  /* Figure out the region code for this region.  */
+  /* Optimistic default: assume that the loop nest is parallelizable
+     (essentially, no GIMPLE_OMP_FOR with (explicit or implicit) 'auto' clause,
+     and no un-annotated loops).  */
+  int region_code = GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED;
+  adjust_region_code (stmts, &region_code);
+
+  if (region_code == GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED)
+    {
+      dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, gimple_location (omp_for),
+		       "parallelized loop nest in OpenACC 'kernels'"
+		       " construct\n");
+
+      clauses = transform_kernels_loop_clauses (omp_for,
+						num_gangs_clause,
+						num_workers_clause,
+						vector_length_clause,
+						clauses);
+    }
+  else if (region_code == GF_OMP_TARGET_KIND_OACC_KERNELS)
+    {
+      dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, gimple_location (omp_for),
+		       "forwarded loop nest in OpenACC 'kernels' construct to"
+		       " \"parloops\" for analysis\n");
+
+      /* We're transforming one "GF_OMP_TARGET_KIND_OACC_KERNELS" into another
+	 "GF_OMP_TARGET_KIND_OACC_KERNELS", so don't have to
+	 "transform_kernels_loop_clauses".  */
+      /* Re-assemble the clauses stripped off earlier.  */
+      clauses
+	= add_parent_or_loop_num_clause (num_gangs_clause, NULL,
+					 OMP_CLAUSE_NUM_GANGS, clauses);
+      clauses
+	= add_parent_or_loop_num_clause (num_workers_clause, NULL,
+					 OMP_CLAUSE_NUM_WORKERS, clauses);
+      clauses
+	= add_parent_or_loop_num_clause (vector_length_clause, NULL,
+					 OMP_CLAUSE_VECTOR_LENGTH, clauses);
+    }
+  else
+    gcc_unreachable ();
 
   /* Now build the parallel region containing this loop.  */
-  gimple_seq parallel_body = NULL;
-  gimple_seq_add_stmt (&parallel_body, stmt);
   gimple *parallel_body_bind
-    = gimple_build_bind (NULL, parallel_body, make_node (BLOCK));
+    = gimple_build_bind (NULL, stmts, make_node (BLOCK));
   gimple *parallel_region
-    = gimple_build_omp_target (
-        parallel_body_bind,
-        GF_OMP_TARGET_KIND_OACC_PARALLEL_KERNELS_PARALLELIZED,
-        clauses);
-  gimple_set_location (parallel_region, gimple_location (stmt));
+    = gimple_build_omp_target (parallel_body_bind, region_code, clauses);
+  gimple_set_location (parallel_region, gimple_location (omp_for));
 
   return parallel_region;
 }
@@ -739,9 +913,9 @@ class control_flow_regions
        information for the statement sequence SEQ.  */
     control_flow_regions (gimple_seq seq);
 
-    /* Return true if the STMT with the given index IDX in the analyzed
+    /* Return true if the statement with the given index IDX in the analyzed
        statement sequence is an unconditionally executed OpenACC loop.  */
-    bool is_unconditional_oacc_for_loop (gimple *stmt, size_t idx);
+    bool is_unconditional_oacc_for_loop (size_t idx);
 
   private:
     /* Find the region representative for the statement identified by index
@@ -772,11 +946,8 @@ control_flow_regions::control_flow_regions (gimple_seq seq)
 }
 
 bool
-control_flow_regions::is_unconditional_oacc_for_loop (gimple *stmt, size_t idx)
+control_flow_regions::is_unconditional_oacc_for_loop (size_t idx)
 {
-  if (top_level_omp_for_in_stmt (stmt) == NULL)
-    /* Not an OpenACC for loop.  */
-    return false;
   if (idx == 0 || idx == representatives.length () - 1)
     /* The first or last statement in the kernels region.  This means that
        there is no room before or after it for a jump or a label.  Thus
@@ -912,7 +1083,7 @@ control_flow_regions::compute_regions (gimple_seq seq)
 }
 
 /* Decompose the body of the KERNELS_REGION, which was originally annotated
-   with the KERNELS_CLAUSES, into a series of parallel regions.  */
+   with the KERNELS_CLAUSES, into a series of regions.  */
 
 static gimple *
 decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
@@ -1052,17 +1223,24 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
 
       gimple *stmt = gsi_stmt (gsi);
       gimple *omp_for = top_level_omp_for_in_stmt (stmt);
+      bool is_unconditional_oacc_for_loop = false;
+      if (omp_for != NULL)
+	is_unconditional_oacc_for_loop
+	  = cf_regions.is_unconditional_oacc_for_loop (idx);
       if (omp_for != NULL
-          && cf_regions.is_unconditional_oacc_for_loop (stmt, idx))
+          && is_unconditional_oacc_for_loop)
         {
-          /* This is an OMP for statement, put it into a parallel region.
+          /* This is an OMP for statement, put it into a separate region.
              But first, construct a gang-single region containing any
              complex sequential statements we may have seen.  */
           if (gang_single_seq != NULL && !only_simple_assignments)
             {
               gimple *single_region
-                = make_gang_single_region (loc, gang_single_seq,
-                                           kernels_clauses);
+                = make_region_seq (loc, gang_single_seq,
+				   num_gangs_clause,
+				   num_workers_clause,
+				   vector_length_clause,
+				   kernels_clauses);
               gimple_seq_add_stmt (&region_body, single_region);
             }
           else if (gang_single_seq != NULL && only_simple_assignments)
@@ -1080,8 +1258,10 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
           gang_single_seq = NULL;
           only_simple_assignments = true;
 
+	  gimple_seq parallel_seq = NULL;
+	  gimple_seq_add_stmt (&parallel_seq, stmt);
           gimple *parallel_region
-            = make_gang_parallel_loop_region (omp_for, stmt,
+	    = make_region_loop_nest (omp_for, parallel_seq,
                                               num_gangs_clause,
                                               num_workers_clause,
                                               vector_length_clause,
@@ -1090,6 +1270,14 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
         }
       else
         {
+	  if (omp_for != NULL)
+	    {
+	      gcc_checking_assert (!is_unconditional_oacc_for_loop);
+	      dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, gimple_location (omp_for),
+			       "unparallelized loop nest in OpenACC 'kernels'"
+			       " region: it's executed conditionally\n");
+	    }
+
           /* This is not an unconditional OMP for statement, so it will be
              put into a gang-single region.  */
           gimple_seq_add_stmt (&gang_single_seq, stmt);
@@ -1102,9 +1290,6 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
                 && DECL_ARTIFICIAL (gimple_assign_lhs (stmt)));
           if (!is_simple_assignment)
             only_simple_assignments = false;
-          /* Remove and issue warnings about gang clauses on any OpenACC
-             loops nested inside this sequentially executed statement.  */
-          make_loops_gang_single (gsi);
         }
     }
 
@@ -1123,7 +1308,11 @@ decompose_kernels_region_body (gimple *kernels_region, tree kernels_clauses)
   if (gang_single_seq != NULL)
     {
       gimple *single_region
-        = make_gang_single_region (loc, gang_single_seq, kernels_clauses);
+        = make_region_seq (loc, gang_single_seq,
+			   num_gangs_clause,
+			   num_workers_clause,
+			   vector_length_clause,
+			   kernels_clauses);
       gimple_seq_add_stmt (&region_body, single_region);
     }
 
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 84d345fdecc..3b4a9c8370d 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,3 +1,11 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* c-c++-common/goacc/kernels-conversion.c: Adjust test.
+	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
+	* c-c++-common/goacc/kernels-decompose-1.c: New file.
+	* gfortran.dg/goacc/kernels-decompose-1.f95: Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
index 3e52ec4f16f..ea7eec997fb 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
@@ -12,19 +12,22 @@ main (void)
   unsigned int sum = 1;
 
 #pragma acc kernels copyin(a[0:N]) copy(sum)
-  /* { dg-bogus "region contains gang partitoned code but is not gang partitioned" "gang partitioned" { xfail *-*-* } .-1 } */
   {
+    /* converted to "oacc_kernels" */
     #pragma acc loop
     for (i = 0; i < N; ++i)
       sum += a[i];
 
+    /* converted to "oacc_parallel_kernels_gang_single" */
     sum++;
     a[0]++;
 
-    #pragma acc loop
+    /* converted to "oacc_parallel_kernels_parallelized" */
+    #pragma acc loop independent
     for (i = 0; i < N; ++i)
       sum += a[i];
 
+    /* converted to "oacc_kernels" */
     if (sum > 10)
       { 
         #pragma acc loop
@@ -32,7 +35,8 @@ main (void)
           sum += a[i];
       }
 
-    #pragma acc loop
+    /* converted to "oacc_kernels" */
+    #pragma acc loop auto
     for (i = 0; i < N; ++i)
       sum += a[i];
   }
@@ -44,10 +48,11 @@ main (void)
    parallel regions.  */ 
 /* { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } } */
 
-/* The three unconditional loop regions are parallelized, the sequential
-   part in between and the conditional loop are made gang-single.  */
-/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 3 "convert_oacc_kernels" } } */
-/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 2 "convert_oacc_kernels" } } */
+/* As noted in the comments above, we get one gang-single serial region; one
+   parallelized loop region; and three "old-style" kernel regions. */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 1 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_kernels" 3 "convert_oacc_kernels" } } */
 
 /* Each of the parallel regions is async, and there is a final call to
    __builtin_GOACC_wait.  */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c b/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c
new file mode 100644
index 00000000000..b5d58c37200
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c
@@ -0,0 +1,123 @@
+/* Test OpenACC 'kernels' construct decomposition.  */
+
+/* { dg-additional-options "-fopenacc-kernels=split" } */
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+/* { dg-additional-options "-O2" } for "parloops".  */
+
+/* See also "../../gfortran.dg/goacc/kernels-decompose-1.f95".  */
+
+#pragma acc routine gang
+extern int
+f_g (int);
+
+#pragma acc routine worker
+extern int
+f_w (int);
+
+#pragma acc routine vector
+extern int
+f_v (int);
+
+#pragma acc routine seq
+extern int
+f_s (int);
+
+int
+main ()
+{
+  int x, y, z;
+#define N 10
+  int a[N], b[N], c[N];
+
+#pragma acc kernels
+  {
+    x = 0; /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+    y = x < 10;
+    z = x++;
+    ;
+  }
+
+#pragma acc kernels
+  for (int i = 0; i < N; i++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    a[i] = 0;
+
+#pragma acc kernels loop
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (int i = 0; i < N; i++)
+    b[i] = a[N - i - 1];
+
+#pragma acc kernels
+  {
+#pragma acc loop
+    /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+    for (int i = 0; i < N; i++)
+      b[i] = a[N - i - 1];
+
+#pragma acc loop
+    /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+    for (int i = 0; i < N; i++)
+      c[i] = a[i] * b[i];
+
+    a[z] = 0; /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+
+#pragma acc loop
+    /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+    for (int i = 0; i < N; i++)
+      c[i] += a[i];
+
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+    /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+    for (int i = 0 + 1; i < N; i++)
+      c[i] += c[i - 1];
+  }
+
+#pragma acc kernels
+  {
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+    /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+    for (int i = 0; i < N; ++i)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+      for (int j = 0; j < N; ++j)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+	 /* { dg-warning "insufficient partitioning available to parallelize loop" "" { target *-*-* } .-1 } */
+	for (int k = 0; k < N; ++k)
+	  a[(i + j + k) % N]
+	    = b[j]
+	    + f_v (c[k]); /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+
+    //TODO Should the following turn into "gang-single" instead of "parloops"?
+    //TODO The problem is that the first STMT is "if (y <= 4) goto <D.2547>; else goto <D.2548>;", thus "parloops".
+    if (y < 5) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+#pragma acc loop independent /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" } */
+      for (int j = 0; j < N; ++j)
+	b[j] = f_w (c[j]);
+  }
+
+#pragma acc kernels /* { dg-warning "region contains gang partitoned code but is not gang partitioned" } */
+  {
+    /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" "" { target *-*-* } .+1 } */
+    y = f_g (a[5]); /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+    /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+    for (int j = 0; j < N; ++j)
+      b[j] = y + f_w (c[j]); /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+  }
+
+#pragma acc kernels
+  {
+    y = 3; /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+    /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+    for (int j = 0; j < N; ++j)
+      b[j] = y + f_v (c[j]); /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+
+    z = 2; /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+  }
+
+#pragma acc kernels /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+  ;
+
+  return 0;
+}
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
index 559916c2325..6604727cf13 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
@@ -9,19 +9,23 @@ program main
 
   !$acc kernels copyin(a(1:N)) copy(sum)
 
+  ! converted to "oacc_kernels"
   !$acc loop
   do i = 1, N
     sum = sum + a(i)
   end do
 
+  ! converted to "oacc_parallel_kernels_gang_single"
   sum = sum + 1
   a(1) = a(1) + 1
 
-  !$acc loop
+  ! converted to "oacc_parallel_kernels_parallelized"
+  !$acc loop independent
   do i = 1, N
     sum = sum + a(i)
   end do
 
+  ! converted to "oacc_kernels"
   if (sum .gt. 10) then
     !$acc loop
     do i = 1, N
@@ -29,8 +33,8 @@ program main
     end do
   end if
 
-  !$acc loop
-  ! { dg-bogus "region contains gang partitoned code but is not gang partitioned" "gang partitioned" { xfail *-*-* } .-1 }
+  ! converted to "oacc_kernels"
+  !$acc loop auto
   do i = 1, N
     sum = sum + a(i)
   end do
@@ -42,15 +46,13 @@ end program main
 ! parallel regions.
 ! { dg-final { scan-tree-dump-times "oacc_data_kernels" 1 "convert_oacc_kernels" } }
 
-! The three unconditional loop regions are parallelized, the sequential part
-! in between and the conditional loop are made gang-single.
-! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 3 "convert_oacc_kernels" } }
-! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 2 "convert_oacc_kernels" } }
+! As noted in the comments above, we get one gang-single serial region; one
+! parallelized loop region; and three "old-style" kernel regions.
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 1 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_kernels" 3 "convert_oacc_kernels" } }
 
 ! Each of the parallel regions is async, and there is a final call to
 ! __builtin_GOACC_wait.
 ! { dg-final { scan-tree-dump-times "oacc_parallel_kernels.* async\(-1\)" 5 "convert_oacc_kernels" } }
 ! { dg-final { scan-tree-dump-times "__builtin_GOACC_wait" 1 "convert_oacc_kernels" } }
-
-! Check that the original kernels region is removed.
-! { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95
new file mode 100644
index 00000000000..520bf034ac6
--- /dev/null
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95
@@ -0,0 +1,132 @@
+! Test OpenACC 'kernels' construct decomposition.
+
+! { dg-additional-options "-fopenacc-kernels=split" }
+! { dg-additional-options "-fopt-info-optimized-omp" }
+! { dg-additional-options "-O2" } for "parloops".
+
+! See also "../../c-c++-common/goacc/kernels-decompose-1.c".
+
+program main
+  implicit none
+
+  integer, external :: f_g
+  !$acc routine (f_g) gang
+  integer, external :: f_w
+  !$acc routine (f_w) worker
+  integer, external :: f_v
+  !$acc routine (f_v) vector
+  integer, external :: f_s
+  !$acc routine (f_s) seq
+
+  integer :: i, j, k
+  integer :: x, y, z
+  logical :: y_l
+  integer, parameter :: N = 10
+  integer :: a(N), b(N), c(N)
+
+  !$acc kernels
+  x = 0 ! { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" }
+  y = 0
+  y_l = x < 10
+  z = x
+  x = x + 1
+  ;
+  !$acc end kernels
+
+  !$acc kernels ! { dg-message "note: assigned OpenACC gang loop parallelism" }
+  do i = 1, N ! { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" }
+     a(i) = 0
+  end do
+  !$acc end kernels
+
+  !$acc kernels loop ! { dg-message "note: assigned OpenACC gang loop parallelism" }
+  ! { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 }
+  do i = 1, N
+     b(i) = a(N - i + 1)
+  end do
+
+  !$acc kernels
+  !$acc loop ! { dg-message "note: assigned OpenACC gang loop parallelism" }
+  ! { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 }
+  do i = 1, N
+     b(i) = a(N - i + 1)
+  end do
+
+  !$acc loop ! { dg-message "note: assigned OpenACC gang loop parallelism" }
+  ! { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 }
+  do i = 1, N
+     c(i) = a(i) * b(i)
+  end do
+
+  a(z) = 0 ! { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" }
+
+  !$acc loop ! { dg-message "note: assigned OpenACC gang loop parallelism" }
+  ! { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 }
+  do i = 1, N
+     c(i) = c(i) + a(i)
+  end do
+
+  !$acc loop seq ! { dg-message "note: assigned OpenACC seq loop parallelism" }
+  ! { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 }
+  do i = 1 + 1, N
+     c(i) = c(i) + c(i - 1)
+  end do
+  !$acc end kernels
+
+  !$acc kernels ! { dg-bogus "note: assigned OpenACC seq loop parallelism" "TODO" { xfail *-*-* } }
+  !$acc loop independent ! { dg-message "note: assigned OpenACC gang loop parallelism" }
+  ! { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 }
+  do i = 1, N
+     !$acc loop independent ! { dg-message "note: assigned OpenACC worker loop parallelism" }
+     do j = 1, N
+        !$acc loop independent ! { dg-message "note: assigned OpenACC seq loop parallelism" "TODO" { xfail *-*-* } }
+        ! { dg-warning "insufficient partitioning available to parallelize loop" "TODO" { xfail *-*-* } .-1 }
+        ! { dg-bogus "note: assigned OpenACC vector loop parallelism" "TODO" { xfail *-*-* } .-2 }
+        do k = 1, N
+           a(1 + mod(i + j + k, N)) &
+                = b(j) &
+                + f_v (c(k)) ! { dg-message "note: assigned OpenACC vector loop parallelism" "TODO" { xfail *-*-* } .-1 }
+        end do
+     end do
+  end do
+
+  !TODO Should the following turn into "gang-single" instead of "parloops"?
+  !TODO The problem is that the first STMT is "if (y <= 4) goto <D.2547>; else goto <D.2548>;", thus "parloops".
+  if (y < 5) then ! { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" }
+     !$acc loop independent ! { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" }
+     do j = 1, N
+        b(j) = f_w (c(j))
+     end do
+  end if
+  !$acc end kernels
+
+  !$acc kernels
+  !TODO This refers to the "gang-single" "f_g" call.
+  ! { dg-warning "region contains gang partitoned code but is not gang partitioned" "TODO" { xfail *-*-* } .-2 }
+  ! { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" "" { target *-*-* } .+1 }
+  y = f_g (a(5)) ! { dg-message "note: assigned OpenACC gang worker vector loop parallelism" "TODO" { xfail *-*-* } }
+
+  !$acc loop independent ! { dg-message "note: assigned OpenACC gang loop parallelism" "TODO" { xfail *-*-* } }
+  ! { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 }
+  ! { dg-bogus "note: assigned OpenACC gang vector loop parallelism" "TODO" { xfail *-*-* } .-2 }
+  do j = 1, N
+     b(j) = y + f_w (c(j)) ! { dg-message "note: assigned OpenACC worker vector loop parallelism" "TODO" { xfail *-*-* } }
+  end do
+  !$acc end kernels
+
+  !$acc kernels
+  y = 3 ! { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" }
+
+  !$acc loop independent ! { dg-message "note: assigned OpenACC gang worker loop parallelism" "TODO" { xfail *-*-* } }
+  ! { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 }
+  ! { dg-bogus "note: assigned OpenACC gang vector loop parallelism" "TODO" { xfail *-*-* } .-2 }
+  do j = 1, N
+     b(j) = y + f_v (c(j)) ! { dg-message "note: assigned OpenACC vector loop parallelism" "TODO" { xfail *-*-* } }
+  end do
+
+  z = 2 ! { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" }
+  !$acc end kernels
+
+  !$acc kernels ! { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" }
+  !$acc end kernels  
+end program main
diff --git a/libgomp/ChangeLog.openacc b/libgomp/ChangeLog.openacc
index f2ff2ee32d2..27ce3434e49 100644
--- a/libgomp/ChangeLog.openacc
+++ b/libgomp/ChangeLog.openacc
@@ -1,5 +1,8 @@
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
+	* testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c: New
+	file.
+
 	* testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90:
 	Update.
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c
new file mode 100644
index 00000000000..601e543495e
--- /dev/null
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c
@@ -0,0 +1,30 @@
+/* { dg-additional-options "-fopenacc-kernels=split" } */
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+#undef NDEBUG
+#include <assert.h>
+
+int main()
+{
+  int a = 0;
+#define N 123
+  int b[N] = { 0 };
+
+#pragma acc kernels
+  {
+    int c = 234; /* { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+
+#pragma acc loop independent gang /* { dg-warning "note: assigned OpenACC gang loop parallelism" } */
+    /* { dg-warning "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } 17 } */
+    for (int i = 0; i < N; ++i)
+      b[i] = c;
+
+    a = c; /* { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+  }
+
+  for (int i = 0; i < N; ++i)
+    assert (b[i] == 234);
+  assert (a == 234);
+
+  return 0;
+}
-- 
2.17.1


[-- Attachment #10: 0009-Make-new-OpenACC-kernels-conversion-the-default-adju.patch --]
[-- Type: text/x-diff, Size: 103526 bytes --]

From 53f61640a510d13537709239d523c283881f0755 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Wed, 23 Jan 2019 02:40:08 -0800
Subject: [PATCH 9/9] Make new OpenACC kernels conversion the default; adjust
 and add tests

	gcc/c-family/
	* c.opt (fopenacc-kernels): Default to "split".
	gcc/fortran/
	* lang.opt (fopenacc-kernels): Default to "split".
	gcc/
	* doc/invoke.texi (-fopenacc-kernels): Update.
	gcc/testsuite/
	* c-c++-common/goacc/note-parallelism-1-kernels-conditional-loop-independent_seq.c:
	New file.
	* c-c++-common/goacc/note-parallelism-1-kernels-loop-auto.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-1-kernels-loop-independent_seq.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-1-kernels-loops.c: Likewise.
	* c-c++-common/goacc/note-parallelism-1-kernels-straight-line.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-combined-kernels-loop-auto.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-combined-kernels-loop-independent_seq.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-kernels-conditional-loop-independent_seq.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-kernels-loop-auto.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-kernels-loop-independent_seq.c:
	Likewise.
	* c-c++-common/goacc/note-parallelism-kernels-loops.c: Likewise.
	* c-c++-common/goacc/classify-kernels-unparallelized.c: Update.
	* c-c++-common/goacc/classify-kernels.c: Likewise.
	* c-c++-common/goacc/classify-parallel.c: Likewise.
	* c-c++-common/goacc/classify-routine.c: Likewise.
	* c-c++-common/goacc/dtype-1.c: Likewise.
	* c-c++-common/goacc/if-clause-2.c: Likewise.
	* c-c++-common/goacc/kernels-conversion.c: Likewise.
	* c-c++-common/goacc/kernels-decompose-1.c: Likewise.
	* c-c++-common/goacc/loop-2-kernels.c: Likewise.
	* c-c++-common/goacc/note-parallelism.c: Likewise.
	* c-c++-common/goacc/routine-1.c: Likewise.
	* c-c++-common/goacc/uninit-dim-clause.c: Likewise.
	* gfortran.dg/goacc/dtype-1.f95: Likewise.
	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
	* gfortran.dg/goacc/kernels-decompose-1.f95: Likewise.
	* gfortran.dg/goacc/kernels-tree.f95: Likewise.
	libgomp/
	* testsuite/libgomp.oacc-c-c++-common/acc_prof-kernels-1.c:
	Update.
	* testsuite/libgomp.oacc-c-c++-common/avoid-offloading-1.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/avoid-offloading-2.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/avoid-offloading-3.c:
	Likewise.
	* testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c:
	Likewise.
	* testsuite/libgomp.oacc-fortran/avoid-offloading-1.f: Likewise.
	* testsuite/libgomp.oacc-fortran/avoid-offloading-2.f: Likewise.
	* testsuite/libgomp.oacc-fortran/avoid-offloading-3.f: Likewise.
	* testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90:
	Likewise.
---
 gcc/ChangeLog.openacc                         |   4 +
 gcc/c-family/ChangeLog.openacc                |   4 +
 gcc/c-family/c.opt                            |   2 +-
 gcc/doc/invoke.texi                           |   2 +-
 gcc/fortran/ChangeLog.openacc                 |   4 +
 gcc/fortran/lang.opt                          |   2 +-
 gcc/testsuite/ChangeLog.openacc               |  40 ++++
 .../goacc/classify-kernels-unparallelized.c   |   7 +-
 .../c-c++-common/goacc/classify-kernels.c     |   2 +-
 .../c-c++-common/goacc/classify-parallel.c    |   2 +-
 .../c-c++-common/goacc/classify-routine.c     |   2 +-
 gcc/testsuite/c-c++-common/goacc/dtype-1.c    |   8 +-
 .../c-c++-common/goacc/if-clause-2.c          |   1 -
 .../c-c++-common/goacc/kernels-conversion.c   |  10 +-
 .../c-c++-common/goacc/kernels-decompose-1.c  |   1 -
 .../c-c++-common/goacc/loop-2-kernels.c       |  14 +-
 ...kernels-conditional-loop-independent_seq.c | 129 +++++++++++
 .../note-parallelism-1-kernels-loop-auto.c    | 126 +++++++++++
 ...rallelism-1-kernels-loop-independent_seq.c | 126 +++++++++++
 .../goacc/note-parallelism-1-kernels-loops.c  |  47 ++++
 ...note-parallelism-1-kernels-straight-line.c |  82 +++++++
 ...e-parallelism-combined-kernels-loop-auto.c | 121 +++++++++++
 ...sm-combined-kernels-loop-independent_seq.c | 121 +++++++++++
 ...kernels-conditional-loop-independent_seq.c | 204 ++++++++++++++++++
 .../note-parallelism-kernels-loop-auto.c      | 138 ++++++++++++
 ...parallelism-kernels-loop-independent_seq.c | 138 ++++++++++++
 .../goacc/note-parallelism-kernels-loops.c    |  50 +++++
 .../c-c++-common/goacc/note-parallelism.c     |   3 +-
 gcc/testsuite/c-c++-common/goacc/routine-1.c  |   2 +-
 .../c-c++-common/goacc/uninit-dim-clause.c    |   6 +-
 gcc/testsuite/gfortran.dg/goacc/dtype-1.f95   |   8 +-
 .../gfortran.dg/goacc/kernels-conversion.f95  |   7 +-
 .../gfortran.dg/goacc/kernels-decompose-1.f95 |   1 -
 .../gfortran.dg/goacc/kernels-tree.f95        |   1 -
 libgomp/ChangeLog.openacc                     |  16 ++
 .../acc_prof-kernels-1.c                      |  17 +-
 .../avoid-offloading-1.c                      |  18 +-
 .../avoid-offloading-2.c                      |  17 +-
 .../avoid-offloading-3.c                      |  14 +-
 .../kernels-decompose-1.c                     |   3 +-
 .../libgomp.oacc-fortran/avoid-offloading-1.f |  18 +-
 .../libgomp.oacc-fortran/avoid-offloading-2.f |  18 +-
 .../libgomp.oacc-fortran/avoid-offloading-3.f |  15 +-
 .../initialize_kernels_loops.f90              |   1 -
 44 files changed, 1494 insertions(+), 58 deletions(-)
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-conditional-loop-independent_seq.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-auto.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-independent_seq.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loops.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-straight-line.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-auto.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-independent_seq.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-conditional-loop-independent_seq.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-auto.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-independent_seq.c
 create mode 100644 gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loops.c

diff --git a/gcc/ChangeLog.openacc b/gcc/ChangeLog.openacc
index 3ef97adef47..433653b2b38 100644
--- a/gcc/ChangeLog.openacc
+++ b/gcc/ChangeLog.openacc
@@ -1,3 +1,7 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+
+	* doc/invoke.texi (-fopenacc-kernels): Update.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/c-family/ChangeLog.openacc b/gcc/c-family/ChangeLog.openacc
index 5b60c3a0dee..39d51c53808 100644
--- a/gcc/c-family/ChangeLog.openacc
+++ b/gcc/c-family/ChangeLog.openacc
@@ -1,3 +1,7 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+
+	* c.opt (fopenacc-kernels): Default to "split".
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
 	* c.opt (fopenacc-kernels): New flag.
diff --git a/gcc/c-family/c.opt b/gcc/c-family/c.opt
index 12f8f55c50f..7b7a90e938a 100644
--- a/gcc/c-family/c.opt
+++ b/gcc/c-family/c.opt
@@ -1618,7 +1618,7 @@ C ObjC C++ ObjC++ LTO Joined Var(flag_openacc_dims)
 Specify default OpenACC compute dimensions.
 
 fopenacc-kernels=
-C ObjC C++ ObjC++ RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS)
+C ObjC C++ ObjC++ RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_SPLIT)
 -fopenacc-kernels=[split|parloops]	Configure OpenACC 'kernels' constructs handling.
 
 Enum
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 3bbeb8c6839..569c02d1e96 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -2165,9 +2165,9 @@ Configure OpenACC 'kernels' constructs handling.
 With @option{-fopenacc-kernels=split}, OpenACC 'kernels' constructs
 are split into a sequence of compute constructs, each then handled
 individually.
+This is the default.
 With @option{-fopenacc-kernels=parloops}, the whole OpenACC
 'kernels' constructs is handled by the @samp{parloops} pass.
-This is the default.
 
 @item -fopenmp
 @opindex fopenmp
diff --git a/gcc/fortran/ChangeLog.openacc b/gcc/fortran/ChangeLog.openacc
index acb2177f22f..2715d7aa195 100644
--- a/gcc/fortran/ChangeLog.openacc
+++ b/gcc/fortran/ChangeLog.openacc
@@ -1,3 +1,7 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+
+	* lang.opt (fopenacc-kernels): Default to "split".
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
 	* lang.opt (fopenacc-kernels): New flag.
diff --git a/gcc/fortran/lang.opt b/gcc/fortran/lang.opt
index b3c9cdb425f..19199aeccbc 100644
--- a/gcc/fortran/lang.opt
+++ b/gcc/fortran/lang.opt
@@ -643,7 +643,7 @@ Fortran LTO Joined Var(flag_openacc_dims)
 ; Documented in C
 
 fopenacc-kernels=
-Fortran RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_PARLOOPS)
+Fortran RejectNegative Joined Enum(openacc_kernels) Var(flag_openacc_kernels) Init(OPENACC_KERNELS_SPLIT)
 ; Documented in C
 
 fopenmp
diff --git a/gcc/testsuite/ChangeLog.openacc b/gcc/testsuite/ChangeLog.openacc
index 3b4a9c8370d..5dd6d7d656c 100644
--- a/gcc/testsuite/ChangeLog.openacc
+++ b/gcc/testsuite/ChangeLog.openacc
@@ -1,3 +1,43 @@
+2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
+	    Gergö Barany  <gergo@codesourcery.com>
+
+	* c-c++-common/goacc/note-parallelism-1-kernels-conditional-loop-independent_seq.c:
+	New file.
+	* c-c++-common/goacc/note-parallelism-1-kernels-loop-auto.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-1-kernels-loop-independent_seq.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-1-kernels-loops.c: Likewise.
+	* c-c++-common/goacc/note-parallelism-1-kernels-straight-line.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-combined-kernels-loop-auto.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-combined-kernels-loop-independent_seq.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-kernels-conditional-loop-independent_seq.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-kernels-loop-auto.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-kernels-loop-independent_seq.c:
+	Likewise.
+	* c-c++-common/goacc/note-parallelism-kernels-loops.c: Likewise.
+	* c-c++-common/goacc/classify-kernels-unparallelized.c: Update.
+	* c-c++-common/goacc/classify-kernels.c: Likewise.
+	* c-c++-common/goacc/classify-parallel.c: Likewise.
+	* c-c++-common/goacc/classify-routine.c: Likewise.
+	* c-c++-common/goacc/dtype-1.c: Likewise.
+	* c-c++-common/goacc/if-clause-2.c: Likewise.
+	* c-c++-common/goacc/kernels-conversion.c: Likewise.
+	* c-c++-common/goacc/kernels-decompose-1.c: Likewise.
+	* c-c++-common/goacc/loop-2-kernels.c: Likewise.
+	* c-c++-common/goacc/note-parallelism.c: Likewise.
+	* c-c++-common/goacc/routine-1.c: Likewise.
+	* c-c++-common/goacc/uninit-dim-clause.c: Likewise.
+	* gfortran.dg/goacc/dtype-1.f95: Likewise.
+	* gfortran.dg/goacc/kernels-conversion.f95: Likewise.
+	* gfortran.dg/goacc/kernels-decompose-1.f95: Likewise.
+	* gfortran.dg/goacc/kernels-tree.f95: Likewise.
+
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 	    Gergö Barany  <gergo@codesourcery.com>
 
diff --git a/gcc/testsuite/c-c++-common/goacc/classify-kernels-unparallelized.c b/gcc/testsuite/c-c++-common/goacc/classify-kernels-unparallelized.c
index 64467774037..61caa2df236 100644
--- a/gcc/testsuite/c-c++-common/goacc/classify-kernels-unparallelized.c
+++ b/gcc/testsuite/c-c++-common/goacc/classify-kernels-unparallelized.c
@@ -1,5 +1,5 @@
 /* Check offloaded function's attributes and classification for unparallelized
-   OpenACC kernels.  */
+   OpenACC 'kernels'.  */
 
 /* { dg-additional-options "-O2" }
    { dg-additional-options "-fopt-info-optimized-omp" }
@@ -13,14 +13,15 @@ extern unsigned int *__restrict a;
 extern unsigned int *__restrict b;
 extern unsigned int *__restrict c;
 
-/* An "extern"al mapping of loop iterations/array indices makes the loop
-   unparallelizable.  */
 extern unsigned int f (unsigned int);
+#pragma acc routine (f) seq
 
 void KERNELS ()
 {
 #pragma acc kernels copyin (a[0:N], b[0:N]) copyout (c[0:N]) /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
   for (unsigned int i = 0; i < N; i++)
+    /* An "extern"al mapping of loop iterations/array indices makes the loop
+       unparallelizable.  */
     c[i] = a[f (i)] + b[f (i)];
 }
 
diff --git a/gcc/testsuite/c-c++-common/goacc/classify-kernels.c b/gcc/testsuite/c-c++-common/goacc/classify-kernels.c
index c59a65e1d0f..eb78c8f5e40 100644
--- a/gcc/testsuite/c-c++-common/goacc/classify-kernels.c
+++ b/gcc/testsuite/c-c++-common/goacc/classify-kernels.c
@@ -1,5 +1,5 @@
 /* Check offloaded function's attributes and classification for OpenACC
-   kernels.  */
+   'kernels'.  */
 
 /* { dg-additional-options "-O2" }
    { dg-additional-options "-fopt-info-optimized-omp" }
diff --git a/gcc/testsuite/c-c++-common/goacc/classify-parallel.c b/gcc/testsuite/c-c++-common/goacc/classify-parallel.c
index b345c225aea..baa0bce004d 100644
--- a/gcc/testsuite/c-c++-common/goacc/classify-parallel.c
+++ b/gcc/testsuite/c-c++-common/goacc/classify-parallel.c
@@ -1,5 +1,5 @@
 /* Check offloaded function's attributes and classification for OpenACC
-   parallel.  */
+   'parallel'.  */
 
 /* { dg-additional-options "-O2" }
    { dg-additional-options "-fopt-info-optimized-omp" }
diff --git a/gcc/testsuite/c-c++-common/goacc/classify-routine.c b/gcc/testsuite/c-c++-common/goacc/classify-routine.c
index 5ca2ec9c603..094c9a760fa 100644
--- a/gcc/testsuite/c-c++-common/goacc/classify-routine.c
+++ b/gcc/testsuite/c-c++-common/goacc/classify-routine.c
@@ -1,5 +1,5 @@
 /* Check offloaded function's attributes and classification for OpenACC
-   routine.  */
+   'routine'.  */
 
 /* { dg-additional-options "-O2" }
    { dg-additional-options "-fopt-info-optimized-omp" }
diff --git a/gcc/testsuite/c-c++-common/goacc/dtype-1.c b/gcc/testsuite/c-c++-common/goacc/dtype-1.c
index 6dd6ebd8ae1..ae3de574dfc 100644
--- a/gcc/testsuite/c-c++-common/goacc/dtype-1.c
+++ b/gcc/testsuite/c-c++-common/goacc/dtype-1.c
@@ -96,11 +96,13 @@ test ()
 
 /* { dg-final { scan-tree-dump-times "oacc_parallel device_type\\(\\*\\) \\\[ wait\\(10\\) vector_length\\(10\\) num_workers\\(10\\) num_gangs\\(10\\) async\\(10\\) \\\] device_type\\(nvidia\\) \\\[ wait\\(3\\) vector_length\\(128\\) num_workers\\(300\\) num_gangs\\(300\\) async\\(3\\) \\\] wait\\(1\\) vector_length\\(1\\) num_workers\\(1\\) num_gangs\\(1\\) async\\(1\\)" 1 "omplower" } } */
 
-/* { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(nvidia\\) \\\[ wait\\(-1\\) async\\(-1\\) \\\]" 1 "omplower" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single async\\(-1\\) num_gangs\\(1\\) device_type\\(nvidia\\) \\\[ wait\\(-1\\) async\\(-1\\) \\\]" 1 "omplower" } } */
 
-/* { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(nvidia\\) \\\[ wait\\(1\\) async\\(1\\) \\\] wait\\(-1\\) async\\(-1\\)" 1 "omplower" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single num_gangs\\(1\\) device_type\\(nvidia\\) \\\[ wait\\(1\\) async\\(1\\) \\\] wait\\(-1\\) async\\(-1\\)" 1 "omplower" } } */
 
-/* { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(\\*\\) \\\[ wait\\(0\\) async\\(0\\) \\\] device_type\\(nvidia\\) \\\[ wait\\(2\\) async\\(2\\) \\\] wait\\(-1\\) async\\(-1\\)" 1 "omplower" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single num_gangs\\(1\\) device_type\\(\\*\\) \\\[ wait\\(0\\) async\\(0\\) \\\] device_type\\(nvidia\\) \\\[ wait\\(2\\) async\\(2\\) \\\] wait\\(-1\\) async\\(-1\\)" 1 "omplower" } } */
+
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single num_gangs\\(1\\) device_type\\(\\*\\) \\\[ wait\\(0\\) async\\(0\\) \\\] device_type\\(nvidia_ptx\\) \\\[ wait\\(1\\) async\\(1\\) \\\] wait\\(-1\\) async\\(-1\\)" 1 "omplower" } } */
 
 /* { dg-final { scan-tree-dump-times "acc loop device_type\\(nvidia\\) \\\[ tile\\(1\\) gang \\\] private\\(i1\\.0\\) private\\(i1\\)" 1 "omplower" } } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/if-clause-2.c b/gcc/testsuite/c-c++-common/goacc/if-clause-2.c
index e17b5dd1107..9920b4fd175 100644
--- a/gcc/testsuite/c-c++-common/goacc/if-clause-2.c
+++ b/gcc/testsuite/c-c++-common/goacc/if-clause-2.c
@@ -1,4 +1,3 @@
-/* { dg-additional-options "-fopenacc-kernels=split" } */
 /* { dg-additional-options "-fdump-tree-convert_oacc_kernels" } */
 
 void
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
index ea7eec997fb..8cb63f00444 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-conversion.c
@@ -1,4 +1,3 @@
-/* { dg-additional-options "-fopenacc-kernels=split" } */
 /* { dg-additional-options "-fdump-tree-convert_oacc_kernels" } */
 
 #define N 1024
@@ -52,12 +51,11 @@ main (void)
    parallelized loop region; and three "old-style" kernel regions. */
 /* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } } */
 /* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 1 "convert_oacc_kernels" } } */
-/* { dg-final { scan-tree-dump-times "oacc_kernels" 3 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_kernels " 3 "convert_oacc_kernels" } } */
 
 /* Each of the parallel regions is async, and there is a final call to
    __builtin_GOACC_wait.  */
-/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels.* async\(-1\)" 5 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_kernels async\\(-1\\)" 3 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single async\\(-1\\)" 1 "convert_oacc_kernels" } } */
+/* { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized async\\(-1\\)" 1 "convert_oacc_kernels" } } */
 /* { dg-final { scan-tree-dump-times "__builtin_GOACC_wait" 1 "convert_oacc_kernels" } } */
-
-/* Check that the original kernels region is removed.  */
-/* { dg-final { scan-tree-dump-not "oacc_kernels" "convert_oacc_kernels" } } */
diff --git a/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c b/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c
index b5d58c37200..7255d692e34 100644
--- a/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c
+++ b/gcc/testsuite/c-c++-common/goacc/kernels-decompose-1.c
@@ -1,6 +1,5 @@
 /* Test OpenACC 'kernels' construct decomposition.  */
 
-/* { dg-additional-options "-fopenacc-kernels=split" } */
 /* { dg-additional-options "-fopt-info-optimized-omp" } */
 /* { dg-additional-options "-O2" } for "parloops".  */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/loop-2-kernels.c b/gcc/testsuite/c-c++-common/goacc/loop-2-kernels.c
index 2608c128b43..8b5c5e75c35 100644
--- a/gcc/testsuite/c-c++-common/goacc/loop-2-kernels.c
+++ b/gcc/testsuite/c-c++-common/goacc/loop-2-kernels.c
@@ -35,7 +35,7 @@ void K(void)
 	for (j = 0; j < 10; j++)
 	  { }
       }
-#pragma acc loop seq gang // { dg-error "'seq' overrides" }
+#pragma acc loop seq gang // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
     for (i = 0; i < 10; i++)
       { }
 
@@ -61,7 +61,7 @@ void K(void)
 	for (j = 0; j < 10; j++)
 	  { }
       }
-#pragma acc loop seq worker // { dg-error "'seq' overrides" }
+#pragma acc loop seq worker // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
     for (i = 0; i < 10; i++)
       { }
 #pragma acc loop gang worker
@@ -90,7 +90,7 @@ void K(void)
 	for (j = 1; j < 10; j++)
 	  { }
       }
-#pragma acc loop seq vector // { dg-error "'seq' overrides" }
+#pragma acc loop seq vector // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
     for (i = 0; i < 10; i++)
       { }
 #pragma acc loop gang vector
@@ -103,7 +103,7 @@ void K(void)
 #pragma acc loop auto
     for (i = 0; i < 10; i++)
       { }
-#pragma acc loop seq auto // { dg-error "'seq' overrides" }
+#pragma acc loop seq auto // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
     for (i = 0; i < 10; i++)
       { }
 #pragma acc loop gang auto
@@ -145,7 +145,7 @@ void K(void)
 #pragma acc kernels loop worker(num:5)
   for (i = 0; i < 10; i++)
     { }
-#pragma acc kernels loop seq worker // { dg-error "'seq' overrides" }
+#pragma acc kernels loop seq worker // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
   for (i = 0; i < 10; i++)
     { }
 #pragma acc kernels loop gang worker
@@ -161,7 +161,7 @@ void K(void)
 #pragma acc kernels loop vector(length:5)
   for (i = 0; i < 10; i++)
     { }
-#pragma acc kernels loop seq vector // { dg-error "'seq' overrides" }
+#pragma acc kernels loop seq vector // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
   for (i = 0; i < 10; i++)
     { }
 #pragma acc kernels loop gang vector
@@ -174,7 +174,7 @@ void K(void)
 #pragma acc kernels loop auto
   for (i = 0; i < 10; i++)
     { }
-#pragma acc kernels loop seq auto // { dg-error "'seq' overrides" }
+#pragma acc kernels loop seq auto // { dg-error "'seq' overrides" "TODO" { xfail *-*-* } }
   for (i = 0; i < 10; i++)
     { }
 #pragma acc kernels loop gang auto
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-conditional-loop-independent_seq.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-conditional-loop-independent_seq.c
new file mode 100644
index 00000000000..a81d3559daf
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-conditional-loop-independent_seq.c
@@ -0,0 +1,129 @@
+/* Test the output of "-fopt-info-optimized-omp" for an OpenACC 'kernels'
+   construct containing conditionally executed 'loop' constructs with
+   'independent' or 'seq' clauses.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+extern int c;
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+  if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop seq
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent worker
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang worker
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent worker vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang worker vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent worker
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent vector
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop seq
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop seq
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop seq
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-auto.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-auto.c
new file mode 100644
index 00000000000..22ac5399a9d
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-auto.c
@@ -0,0 +1,126 @@
+/* Test the output of "-fopt-info-optimized-omp" for an OpenACC 'kernels'
+   construct containing 'loop' constructs with explicit or implicit 'auto'
+   clause.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels
+ /* Strangely indented to keep this similar to other test cases.  */
+ {
+#pragma acc loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto gang /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto worker /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto gang vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto gang worker /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto worker vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto gang worker vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto gang /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto worker
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto vector
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-independent_seq.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-independent_seq.c
new file mode 100644
index 00000000000..a436cd3f007
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loop-independent_seq.c
@@ -0,0 +1,126 @@
+/* Test the output of "-fopt-info-optimized-omp" for an OpenACC 'kernels'
+   construct containing 'loop' constructs with 'independent' or 'seq'
+   clauses.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels
+ /* Strangely indented to keep this similar to other test cases.  */
+ {
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent worker /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent vector /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang vector /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang worker /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent worker vector /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang worker vector /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent gang /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent worker /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent vector /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loops.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loops.c
new file mode 100644
index 00000000000..e8b994b5be0
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-loops.c
@@ -0,0 +1,47 @@
+/* Test the output of "-fopt-info-optimized-omp" for an OpenACC 'kernels'
+   construct containing loops.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ {
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    ;
+
+  for (x = 0; x < 10; x++)
+    ;
+
+  for (x = 0; x < 10; x++)
+    for (y = 0; y < 10; y++)
+      for (z = 0; z < 10; z++)
+	;
+
+  for (x = 0; x < 10; x++)
+    ;
+
+  for (x = 0; x < 10; x++)
+    for (y = 0; y < 10; y++)
+      ;
+
+  for (x = 0; x < 10; x++)
+    for (y = 0; y < 10; y++)
+      for (z = 0; z < 10; z++)
+	;
+
+  for (x = 0; x < 10; x++)
+    for (y = 0; y < 10; y++)
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-straight-line.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-straight-line.c
new file mode 100644
index 00000000000..8e40f6217be
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-1-kernels-straight-line.c
@@ -0,0 +1,82 @@
+/* Test the output of "-fopt-info-optimized-omp" for an OpenACC 'kernels'
+   construct containing straight-line code.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+#pragma acc routine gang
+extern int
+f_g (int);
+
+#pragma acc routine worker
+extern int
+f_w (int);
+
+#pragma acc routine vector
+extern int
+f_v (int);
+
+#pragma acc routine seq
+extern int
+f_s (int);
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels /* { dg-warning "region contains gang partitoned code but is not gang partitioned" } */
+  {
+    x = 0; /* { dg-message "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+    y = x < 10;
+    z = x++;
+    ;
+
+    y = 0;
+    z = y < 10;
+    x -= f_g (y++); /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+    ;
+
+    x = f_w (0); /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+    z = f_v (x < 10); /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    y -= f_s (x++); /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+    ;
+
+    x = 0;
+    y = x < 10;
+    z = (x++);
+    y = 0;
+    x = y < 10;
+    z += (y++);
+    ;
+
+    x = 0;
+    y += f_s (x < 10); /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+    x++;
+    y = 0;
+    y += f_v (y < 10); /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    y++;
+    z = 0;
+    y += f_w (z < 10); /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+    z++;
+    ;
+
+    x = 0;
+    y *= f_g ( /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+	      f_w (x < 10) /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+	      + f_g (x < 10) /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+	      );
+    x++;
+    y = 0;
+    y *= y < 10;
+    y++;
+    z = 0;
+    y *= z < 10;
+    z++;
+    ;
+  }
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-auto.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-auto.c
new file mode 100644
index 00000000000..0254036d7af
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-auto.c
@@ -0,0 +1,121 @@
+/* Test the output of "-fopt-info-optimized-omp" for combined OpenACC 'kernels
+   loop' constructs with explicit or implicit 'auto' clause.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto gang /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto worker /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto gang vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto gang worker /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto worker vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto gang worker vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto gang /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto worker
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto vector
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc kernels loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop
+      for (z = 0; z < 10; z++)
+	;
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-independent_seq.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-independent_seq.c
new file mode 100644
index 00000000000..83602a9414d
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-combined-kernels-loop-independent_seq.c
@@ -0,0 +1,121 @@
+/* Test the output of "-fopt-info-optimized-omp" for combined OpenACC 'kernels
+   loop' constructs with 'independent' or 'seq' clauses.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent gang /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent worker /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent vector /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent gang vector /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent gang worker /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent worker vector /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent gang worker vector /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent gang /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent worker /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent vector /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop independent /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc kernels loop independent /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-conditional-loop-independent_seq.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-conditional-loop-independent_seq.c
new file mode 100644
index 00000000000..e12e0fdae52
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-conditional-loop-independent_seq.c
@@ -0,0 +1,204 @@
+/* Test the output of "-fopt-info-optimized-omp" for OpenACC 'kernels'
+   constructs containing conditionally executed 'loop' constructs with
+   'independent' or 'seq' clauses.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+extern int c;
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop seq
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent gang
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent worker
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent gang vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent gang worker
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent worker vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent gang worker vector
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent gang
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent worker
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent vector
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+      ;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop seq
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop seq
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop independent
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+ /* Strangely indented to keep this similar to other test cases.  */
+ if (c) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+ {
+#pragma acc loop seq
+  /* { dg-message "note: unparallelized loop nest in OpenACC .kernels. region: it's executed conditionally" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq
+      for (z = 0; z < 10; z++)
+	;
+ }
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-auto.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-auto.c
new file mode 100644
index 00000000000..d52b2e860c2
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-auto.c
@@ -0,0 +1,138 @@
+/* Test the output of "-fopt-info-optimized-omp" for OpenACC 'kernels'
+   constructs containing 'loop' constructs with explicit or implicit 'auto'
+   clause.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels
+#pragma acc loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto gang /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto worker /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto gang vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto gang worker /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto worker vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto gang worker vector /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto gang /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto worker
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto vector
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc kernels
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop
+    for (y = 0; y < 10; y++)
+#pragma acc loop auto
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop auto /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop auto
+    for (y = 0; y < 10; y++)
+#pragma acc loop
+      for (z = 0; z < 10; z++)
+	;
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-independent_seq.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-independent_seq.c
new file mode 100644
index 00000000000..661f7122a2c
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loop-independent_seq.c
@@ -0,0 +1,138 @@
+/* Test the output of "-fopt-info-optimized-omp" for OpenACC 'kernels'
+   constructs containing 'loop' constructs with 'independent' or 'seq'
+   clauses.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent gang /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent worker /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent vector /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent gang vector /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent gang worker /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent worker vector /* { dg-message "note: assigned OpenACC worker vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent gang worker vector /* { dg-message "note: assigned OpenACC gang worker vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent gang /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent worker /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent vector /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+    ;
+
+#pragma acc kernels
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc kernels
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang worker loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  /* { dg-message "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } .-1 } */
+  for (x = 0; x < 10; x++)
+#pragma acc loop independent /* { dg-message "note: assigned OpenACC gang vector loop parallelism" } */
+    for (y = 0; y < 10; y++)
+#pragma acc loop seq /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+      for (z = 0; z < 10; z++)
+	;
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loops.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loops.c
new file mode 100644
index 00000000000..7587d9d2962
--- /dev/null
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism-kernels-loops.c
@@ -0,0 +1,50 @@
+/* Test the output of "-fopt-info-optimized-omp" for an OpenACC 'kernels'
+   construct containing loops.  */
+
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
+//TODO update accordingly
+/* See also "../../gfortran.dg/goacc/note-parallelism.f90".  */
+
+int
+main ()
+{
+  int x, y, z;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    ;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    ;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    for (y = 0; y < 10; y++)
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    ;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    for (y = 0; y < 10; y++)
+      ;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    for (y = 0; y < 10; y++)
+      for (z = 0; z < 10; z++)
+	;
+
+#pragma acc kernels /* { dg-message "note: assigned OpenACC seq loop parallelism" } */
+  for (x = 0; x < 10; x++) /* { dg-message "note: beginning .parloops. region in OpenACC .kernels. construct" } */
+    for (y = 0; y < 10; y++)
+      for (z = 0; z < 10; z++)
+	;
+
+  return 0;
+}
diff --git a/gcc/testsuite/c-c++-common/goacc/note-parallelism.c b/gcc/testsuite/c-c++-common/goacc/note-parallelism.c
index 7548fb72d14..c514fae2574 100644
--- a/gcc/testsuite/c-c++-common/goacc/note-parallelism.c
+++ b/gcc/testsuite/c-c++-common/goacc/note-parallelism.c
@@ -1,4 +1,5 @@
-/* Test the output of "-fopt-info-optimized-omp".  */
+/* Test the output of "-fopt-info-optimized-omp" for OpenACC 'parallel'
+   constructs.  */
 
 /* { dg-additional-options "-fopt-info-optimized-omp" } */
 
diff --git a/gcc/testsuite/c-c++-common/goacc/routine-1.c b/gcc/testsuite/c-c++-common/goacc/routine-1.c
index 738957501b9..7cf5506e41a 100644
--- a/gcc/testsuite/c-c++-common/goacc/routine-1.c
+++ b/gcc/testsuite/c-c++-common/goacc/routine-1.c
@@ -91,7 +91,7 @@ extern void nohost (void);
 
 int main ()
 {
-#pragma acc kernels num_gangs (32) num_workers (32) vector_length (32)
+#pragma acc kernels num_gangs (32) num_workers (32) vector_length (32) /* { dg-warning "region contains gang partitoned code but is not gang partitioned" } */
   {
     gang ();
     worker ();
diff --git a/gcc/testsuite/c-c++-common/goacc/uninit-dim-clause.c b/gcc/testsuite/c-c++-common/goacc/uninit-dim-clause.c
index 72aacd70f79..6bc35d77765 100644
--- a/gcc/testsuite/c-c++-common/goacc/uninit-dim-clause.c
+++ b/gcc/testsuite/c-c++-common/goacc/uninit-dim-clause.c
@@ -21,12 +21,12 @@ void acc_kernels()
 {
   int i, j, k;
 
-  #pragma acc kernels num_gangs(i) /* { dg-warning "is used uninitialized in this function" } */
+  #pragma acc kernels num_gangs(i) /* { dg-warning "is used uninitialized in this function" "TODO" { xfail *-*-* } } */
   ;
 
-  #pragma acc kernels num_workers(j) /* { dg-warning "is used uninitialized in this function" } */
+  #pragma acc kernels num_workers(j) /* { dg-warning "is used uninitialized in this function" "TODO" { xfail *-*-* } } */
   ;
 
-  #pragma acc kernels vector_length(k) /* { dg-warning "is used uninitialized in this function" } */
+  #pragma acc kernels vector_length(k) /* { dg-warning "is used uninitialized in this function" "TODO" { xfail *-*-* } } */
   ;
 }
diff --git a/gcc/testsuite/gfortran.dg/goacc/dtype-1.f95 b/gcc/testsuite/gfortran.dg/goacc/dtype-1.f95
index 460922a35df..b716d450abf 100644
--- a/gcc/testsuite/gfortran.dg/goacc/dtype-1.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/dtype-1.f95
@@ -175,13 +175,13 @@ end subroutine sr5b
 
 ! { dg-final { scan-tree-dump-times "oacc_parallel device_type\\(\\*\\) \\\[ async\\(10\\) wait\\(10\\) num_gangs\\(10\\) num_workers\\(10\\) vector_length\\(10\\) \\\] device_type\\(nvidia_ptx\\) \\\[ async\\(3\\) wait\\(3\\) num_gangs\\(300\\) num_workers\\(300\\) vector_length\\(128\\) \\\] async\\(1\\) wait\\(1\\) num_gangs\\(1\\) num_workers\\(1\\) vector_length\\(1\\)" 1 "omplower" } }
 
-! { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(nvidia\\) \\\[ async\\(-1\\) wait\\(-1\\) \\\]" 1 "omplower" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single async\\(-1\\) num_gangs\\(1\\) device_type\\(nvidia\\) \\\[ async\\(-1\\) wait\\(-1\\) \\\]" 1 "omplower" } }
 
-! { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(nvidia\\) \\\[ async\\(1\\) wait\\(1\\) \\\]" 1 "omplower" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single num_gangs\\(1\\) device_type\\(nvidia\\) \\\[ async\\(1\\) wait\\(1\\) \\\]" 1 "omplower" } }
 
-! { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(\\*\\) \\\[ async\\(0\\) wait\\(0\\) \\\] device_type\\(nvidia\\) \\\[ async\\(2\\) wait\\(2\\) \\\]" 1 "omplower" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single num_gangs\\(1\\) device_type\\(\\*\\) \\\[ async\\(0\\) wait\\(0\\) \\\] device_type\\(nvidia\\) \\\[ async\\(2\\) wait\\(2\\) \\\]" 1 "omplower" } }
 
-! { dg-final { scan-tree-dump-times "oacc_kernels device_type\\(\\*\\) \\\[ async\\(0\\) wait\\(0\\) \\\] device_type\\(nvidia_ptx\\) \\\[ async\\(1\\) wait\\(1\\) \\\] async\\(-1\\) wait\\(-1\\)" 1 "omplower" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single num_gangs\\(1\\) device_type\\(\\*\\) \\\[ async\\(0\\) wait\\(0\\) \\\] device_type\\(nvidia_ptx\\) \\\[ async\\(1\\) wait\\(1\\) \\\] async\\(-1\\) wait\\(-1\\)" 1 "omplower" } }
 
 ! { dg-final { scan-tree-dump-times "acc loop device_type\\(nvidia\\) \\\[ tile\\(1\\) gang \\\] private\\(i1\\) private\\(i1\\.1\\)" 1 "omplower" } }
 
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
index 6604727cf13..4672d15572e 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-conversion.f95
@@ -1,4 +1,3 @@
-! { dg-additional-options "-fopenacc-kernels=split" }
 ! { dg-additional-options "-fdump-tree-convert_oacc_kernels" }
 
 program main
@@ -50,9 +49,11 @@ end program main
 ! parallelized loop region; and three "old-style" kernel regions.
 ! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single" 1 "convert_oacc_kernels" } }
 ! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized" 1 "convert_oacc_kernels" } }
-! { dg-final { scan-tree-dump-times "oacc_kernels" 3 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_kernels " 3 "convert_oacc_kernels" } }
 
 ! Each of the parallel regions is async, and there is a final call to
 ! __builtin_GOACC_wait.
-! { dg-final { scan-tree-dump-times "oacc_parallel_kernels.* async\(-1\)" 5 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_kernels async\\(-1\\)" 3 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_gang_single async\\(-1\\)" 1 "convert_oacc_kernels" } }
+! { dg-final { scan-tree-dump-times "oacc_parallel_kernels_parallelized async\\(-1\\)" 1 "convert_oacc_kernels" } }
 ! { dg-final { scan-tree-dump-times "__builtin_GOACC_wait" 1 "convert_oacc_kernels" } }
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95
index 520bf034ac6..8173c3651e1 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-decompose-1.f95
@@ -1,6 +1,5 @@
 ! Test OpenACC 'kernels' construct decomposition.
 
-! { dg-additional-options "-fopenacc-kernels=split" }
 ! { dg-additional-options "-fopt-info-optimized-omp" }
 ! { dg-additional-options "-O2" } for "parloops".
 
diff --git a/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95 b/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95
index b83ca2d8f06..bc9bebac969 100644
--- a/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95
+++ b/gcc/testsuite/gfortran.dg/goacc/kernels-tree.f95
@@ -1,6 +1,5 @@
 ! { dg-do compile } 
 ! { dg-additional-options "-fdump-tree-original" } 
-! { dg-additional-options "-fopenacc-kernels=split" }
 ! { dg-additional-options "-fdump-tree-convert_oacc_kernels" }
 
 program test
diff --git a/libgomp/ChangeLog.openacc b/libgomp/ChangeLog.openacc
index 27ce3434e49..96908d1a305 100644
--- a/libgomp/ChangeLog.openacc
+++ b/libgomp/ChangeLog.openacc
@@ -1,5 +1,21 @@
 2019-01-31  Thomas Schwinge  <thomas@codesourcery.com>
 
+	* testsuite/libgomp.oacc-c-c++-common/acc_prof-kernels-1.c:
+	Update.
+	* testsuite/libgomp.oacc-c-c++-common/avoid-offloading-1.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/avoid-offloading-2.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/avoid-offloading-3.c:
+	Likewise.
+	* testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c:
+	Likewise.
+	* testsuite/libgomp.oacc-fortran/avoid-offloading-1.f: Likewise.
+	* testsuite/libgomp.oacc-fortran/avoid-offloading-2.f: Likewise.
+	* testsuite/libgomp.oacc-fortran/avoid-offloading-3.f: Likewise.
+	* testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90:
+	Likewise.
+
 	* testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c: New
 	file.
 
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_prof-kernels-1.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_prof-kernels-1.c
index 24b5718ced6..1a5b5fbaa6d 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_prof-kernels-1.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/acc_prof-kernels-1.c
@@ -34,6 +34,7 @@ static int state = -1;
 static acc_device_t acc_device_type;
 static int acc_device_num;
 static int num_gangs, num_workers, vector_length;
+static int async;
 
 void cb_enqueue_launch_start (acc_prof_info *prof_info, acc_event_info *event_info, acc_api_info *api_info)
 {
@@ -50,7 +51,7 @@ void cb_enqueue_launch_start (acc_prof_info *prof_info, acc_event_info *event_in
   assert (prof_info->device_type == acc_device_type);
   assert (prof_info->device_number == acc_device_num);
   assert (prof_info->thread_id == -1);
-  assert (prof_info->async == acc_async_sync);
+  assert (prof_info->async == async);
   assert (prof_info->async_queue == prof_info->async);
   assert (prof_info->src_file == NULL);
   assert (prof_info->func_name == NULL);
@@ -142,8 +143,10 @@ int main()
   acc_device_num = acc_get_device_num (acc_device_type);
   assert (state == 0);
 
-  /* Parallelism dimensions: compiler/runtime decides.  */
   STATE_OP (state, = 0);
+  /* Implicit async.  */
+  async = acc_async_noval;
+  /* Parallelism dimensions: compiler/runtime decides.  */
   num_gangs = num_workers = vector_length = 0;
   {
 #define N 100
@@ -172,8 +175,10 @@ int main()
 #undef N
   }
 
-  /* Parallelism dimensions: literal.  */
   STATE_OP (state, = 0);
+  /* Explicit async: without argument.  */
+  async = acc_async_noval;
+  /* Parallelism dimensions: literal.  */
   num_gangs = 30;
   num_workers = 3;
   vector_length = 5;
@@ -181,6 +186,7 @@ int main()
 #define N 100
     int x[N];
 #pragma acc kernels \
+  async \
   num_gangs (30) num_workers (3) vector_length (5)
     /* { dg-prune-output "using vector_length \\(32\\), ignoring 5" } */
     {
@@ -206,8 +212,10 @@ int main()
 #undef N
   }
 
-  /* Parallelism dimensions: variable.  */
   STATE_OP (state, = 0);
+  /* Explicit async: variable.  */
+  async = 123;
+  /* Parallelism dimensions: variable.  */
   num_gangs = 22;
   num_workers = 5;
   vector_length = 7;
@@ -215,6 +223,7 @@ int main()
 #define N 100
     int x[N];
 #pragma acc kernels \
+  async (async) \
   num_gangs (num_gangs) num_workers (num_workers) vector_length (vector_length)
     /* { dg-prune-output "using vector_length \\(32\\), ignoring runtime setting" } */
     {
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-1.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-1.c
index 72b9ce0ce02..fb57faef054 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-1.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-1.c
@@ -1,5 +1,7 @@
 /* Test that the compiler decides to "avoid offloading".  */
 
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
 #include <openacc.h>
 
 int main(void)
@@ -7,8 +9,20 @@ int main(void)
   int x, y;
 
 #pragma acc data copyout(x, y)
-#pragma acc kernels /* { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } } */
-  *((volatile int *) &x) = 33, y = acc_on_device (acc_device_host);
+#pragma acc kernels
+  *((volatile int *) &x) = 33, y = acc_on_device (acc_device_host); /* { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+
+  /* The following will trigger "avoid offloading".  */
+#pragma acc kernels
+  {
+#pragma acc loop auto /* { dg-warning "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" } */
+    /* { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } 18 } */
+    /* { dg-warning "note: assigned OpenACC seq loop parallelism" "" { target *-*-* } 18 } */
+    for (int i = 0; i < x; ++i)
+      if (x == 0)
+	x = 1;
+      ;
+  }
 
   if (x != 33)
     __builtin_abort();
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-2.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-2.c
index 9e05d84d792..00912a3d006 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-2.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-2.c
@@ -1,6 +1,8 @@
 /* Test that a user can override the compiler's "avoid offloading"
    decision.  */
 
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
 #include <openacc.h>
 
 int main(void)
@@ -19,8 +21,19 @@ int main(void)
   int x, y;
 
 #pragma acc data copyout(x, y)
-#pragma acc kernels /* { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } } */
-  *((volatile int *) &x) = 33, y = acc_on_device (acc_device_host);
+#pragma acc kernels
+  *((volatile int *) &x) = 33, y = acc_on_device (acc_device_host); /* { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+
+  /* The following will trigger "avoid offloading".  */
+#pragma acc kernels
+  {
+#pragma acc loop auto /* { dg-warning "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" } */
+    /* { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } 30 } */
+    /* { dg-warning "note: assigned OpenACC seq loop parallelism" "" { target *-*-* } 30 } */
+    for (int i = 0; i < x; ++i)
+      if (x == 0)
+	x = 1;
+  }
 
   if (x != 33)
     __builtin_abort();
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-3.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-3.c
index f186482026e..1bb6b103b4e 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-3.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/avoid-offloading-3.c
@@ -1,6 +1,8 @@
 /* Test that a user can override the compiler's "avoid offloading"
    decision.  */
 
+/* { dg-additional-options "-fopt-info-optimized-omp" } */
+
 /* Override the compiler's "avoid offloading" decision.
    { dg-additional-options "-foffload-force" } */
 
@@ -12,7 +14,17 @@ int main(void)
 
 #pragma acc data copyout(x, y)
 #pragma acc kernels
-  *((volatile int *) &x) = 33, y = acc_on_device (acc_device_host);
+  *((volatile int *) &x) = 33, y = acc_on_device (acc_device_host); /* { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
+
+  /* The following would trigger "avoid offloading".  */
+#pragma acc kernels
+  {
+#pragma acc loop auto /* { dg-warning "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" } */
+    /* { dg-warning "note: assigned OpenACC seq loop parallelism" "" { target *-*-* } 22 } */
+    for (int i = 0; i < x; ++i)
+      if (x == 0)
+	x = 1;
+  }
 
   if (x != 33)
     __builtin_abort();
diff --git a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c
index 601e543495e..bf58a139d3a 100644
--- a/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c
+++ b/libgomp/testsuite/libgomp.oacc-c-c++-common/kernels-decompose-1.c
@@ -1,4 +1,3 @@
-/* { dg-additional-options "-fopenacc-kernels=split" } */
 /* { dg-additional-options "-fopt-info-optimized-omp" } */
 
 #undef NDEBUG
@@ -15,7 +14,7 @@ int main()
     int c = 234; /* { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" } */
 
 #pragma acc loop independent gang /* { dg-warning "note: assigned OpenACC gang loop parallelism" } */
-    /* { dg-warning "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } 17 } */
+    /* { dg-warning "note: parallelized loop nest in OpenACC .kernels. construct" "" { target *-*-* } 16 } */
     for (int i = 0; i < N; ++i)
       b[i] = c;
 
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-1.f b/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-1.f
index fb14be19d8d..275c22f90a6 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-1.f
+++ b/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-1.f
@@ -2,6 +2,7 @@
 
 ! { dg-do run }
 ! { dg-additional-options "-cpp" }
+! { dg-additional-options "-fopt-info-optimized-omp" }
 ! As __OPTIMIZE__ is defined for -O1 and higher, we don't have an (easy) way to
 ! distinguish -O1 (where we will offload) from -O2 (where we won't offload), so
 ! for -O1 testing, we expect to abort.
@@ -10,16 +11,29 @@
       IMPLICIT NONE
       INCLUDE "openacc_lib.h"
 
+      INTEGER :: I
       INTEGER, VOLATILE :: X
       LOGICAL :: Y
 
 !$ACC DATA COPYOUT(X, Y)
-!$ACC KERNELS ! { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } }
-      X = 33
+!$ACC KERNELS
+      X = 33 ! { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" }
       Y = ACC_ON_DEVICE (ACC_DEVICE_HOST);
 !$ACC END KERNELS
 !$ACC END DATA
 
+      ! The following will trigger "avoid offloading".
+!$ACC KERNELS
+!$ACC LOOP AUTO ! { dg-warning "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" }
+! { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } 27 }
+! { dg-warning "note: assigned OpenACC seq loop parallelism" "" { target *-*-* } 27 }
+      DO I = 1, X
+         IF (X .EQ. 0) THEN
+            X = 1
+         END IF
+      END DO
+!$ACC END KERNELS
+
       IF (X .NE. 33) CALL ABORT
 #if defined ACC_DEVICE_TYPE_nvidia
 # if !defined __OPTIMIZE__
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-2.f b/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-2.f
index 5a064618e51..a83e1739885 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-2.f
+++ b/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-2.f
@@ -2,11 +2,13 @@
 
 ! { dg-do run }
 ! { dg-additional-options "-cpp" }
+! { dg-additional-options "-fopt-info-optimized-omp" }
 
       IMPLICIT NONE
       INCLUDE "openacc_lib.h"
 
       INTEGER :: D
+      INTEGER :: I
       INTEGER, VOLATILE :: X
       LOGICAL :: Y
 
@@ -21,12 +23,24 @@
       CALL ACC_INIT (D)
 
 !$ACC DATA COPYOUT(X, Y)
-!$ACC KERNELS ! { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } }
-      X = 33
+!$ACC KERNELS
+      X = 33 ! { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" }
       Y = ACC_ON_DEVICE (ACC_DEVICE_HOST)
 !$ACC END KERNELS
 !$ACC END DATA
 
+      ! The following will trigger "avoid offloading".
+!$ACC KERNELS
+!$ACC LOOP AUTO ! { dg-warning "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" }
+! { dg-warning "OpenACC kernels construct will be executed sequentially; will by default avoid offloading to prevent data copy penalty" "" { target { openacc_nvidia_accel_selected && opt_levels_2_plus } } 34 }
+! { dg-warning "note: assigned OpenACC seq loop parallelism" "" { target *-*-* } 34 }
+      DO I = 1, X
+         IF (X .EQ. 0) THEN
+            X = 1
+         END IF
+      END DO
+!$ACC END KERNELS
+
       IF (X .NE. 33) CALL ABORT
 #if defined ACC_DEVICE_TYPE_nvidia
       IF (Y) CALL ABORT
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-3.f b/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-3.f
index 1c09f83b099..f101186ca19 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-3.f
+++ b/libgomp/testsuite/libgomp.oacc-fortran/avoid-offloading-3.f
@@ -2,6 +2,7 @@
 
 ! { dg-do run }
 ! { dg-additional-options "-cpp" }
+! { dg-additional-options "-fopt-info-optimized-omp" }
 !     Override the compiler's "avoid offloading" decision.
 ! { dg-additional-options "-foffload-force" }
 
@@ -9,16 +10,28 @@
       INCLUDE "openacc_lib.h"
 
       INTEGER :: D
+      INTEGER :: I
       INTEGER, VOLATILE :: X
       LOGICAL :: Y
 
 !$ACC DATA COPYOUT(X, Y)
 !$ACC KERNELS
-      X = 33
+      X = 33 ! { dg-warning "note: beginning .gang-single. region in OpenACC .kernels. construct" }
       Y = ACC_ON_DEVICE (ACC_DEVICE_HOST)
 !$ACC END KERNELS
 !$ACC END DATA
 
+      ! The following would trigger "avoid offloading".
+!$ACC KERNELS
+!$ACC LOOP AUTO ! { dg-warning "note: forwarded loop nest in OpenACC .kernels. construct to .parloops. for analysis" }
+! { dg-warning "note: assigned OpenACC seq loop parallelism" "" { target *-*-* } 26 }
+      DO I = 1, X
+         IF (X .EQ. 0) THEN
+            X = 1
+         END IF
+      END DO
+!$ACC END KERNELS
+
       IF (X .NE. 33) CALL ABORT
 #if defined ACC_DEVICE_TYPE_nvidia
       IF (Y) CALL ABORT
diff --git a/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90 b/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90
index 35e909f8278..e8b4f3abc05 100644
--- a/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90
+++ b/libgomp/testsuite/libgomp.oacc-fortran/initialize_kernels_loops.f90
@@ -1,5 +1,4 @@
 ! { dg-do run }
-! { dg-additional-options "-fopenacc-kernels=split" }
 ! { dg-additional-options "-fopt-info-optimized-omp" }
 
 subroutine kernel(lo, hi, a, b, c)
-- 
2.17.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2022-03-17  8:05 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-01  0:00 [og8] OpenACC 'kernels' construct changes: splitting of the construct into several regions Thomas Schwinge
2019-02-01 19:48 ` Thomas Schwinge
2019-07-17 21:03 ` [PATCH 00/10, OpenACC] Rework handling of OpenACC kernels regions Kwok Cheung Yeung
2019-07-17 21:04   ` [PATCH 01/10, OpenACC] Use "-fopenacc-kernels=parloops" to document "parloops" test cases Kwok Cheung Yeung
2019-07-17 21:05   ` [PATCH 02/10, OpenACC] Add OpenACC target kinds for decomposed kernels regions Kwok Cheung Yeung
2019-07-18  9:28     ` Jakub Jelinek
2019-08-05 22:31       ` Kwok Cheung Yeung
2019-07-17 21:06   ` [PATCH 03/10, OpenACC] Separate OpenACC kernels regions in data and parallel parts Kwok Cheung Yeung
2019-07-17 21:11   ` [PATCH 04/10, OpenACC] Turn OpenACC kernels regions into a sequence of, parallel regions Kwok Cheung Yeung
2019-07-18 10:09     ` Jakub Jelinek
2019-08-05 21:58       ` Kwok Cheung Yeung
2020-11-13 22:33         ` In 'gcc/omp-oacc-kernels-decompose.cc', use langhook instead of accessing language-specific decl information (was: [PATCH 04/10, OpenACC] Turn OpenACC kernels regions into a sequence of, parallel regions) Thomas Schwinge
2019-07-17 21:12   ` [PATCH 05/10, OpenACC] Handle conditional execution of loops in OpenACC, kernels regions Kwok Cheung Yeung
2019-07-17 21:13   ` [PATCH 06/10, OpenACC] Adjust parallelism of loops in gang-single parts of OpenACC " Kwok Cheung Yeung
2019-08-05 22:17     ` Kwok Cheung Yeung
2019-07-17 21:24   ` [PATCH 07/10, OpenACC] Launch kernels asynchronously in " Kwok Cheung Yeung
2019-07-17 21:30   ` [PATCH 08/10, OpenACC] New OpenACC kernels region decompose algorithm Kwok Cheung Yeung
2019-07-17 21:32   ` [PATCH 09/10, OpenACC] Avoid introducing 'create' mapping clauses for loop index variables in kernels regions Kwok Cheung Yeung
2019-07-17 21:37   ` [PATCH 10/10, OpenACC] Make new OpenACC kernels conversion the default; adjust and add tests Kwok Cheung Yeung
2019-07-18  9:24   ` [PATCH 00/10, OpenACC] Rework handling of OpenACC kernels regions Jakub Jelinek
2020-11-13 22:22 ` Decompose OpenACC 'kernels' constructs into parts, a sequence of compute constructs (was: [og8] OpenACC 'kernels' construct changes: splitting of the construct into several regions) Thomas Schwinge
2020-11-15  9:14   ` Tobias Burnus
2021-04-19  8:29     ` Decompose OpenACC 'kernels' constructs into parts, a sequence of compute constructs Thomas Schwinge
2021-04-19 12:38       ` Thomas Schwinge
2020-11-27 13:50   ` Thomas Schwinge
2022-01-13  9:44   ` Enhance OpenACC 'kernels' decomposition testing (was: Decompose OpenACC 'kernels' constructs into parts, a sequence of compute constructs) Thomas Schwinge
2022-01-19 22:29   ` Catch 'GIMPLE_DEBUG' misbehavior in OpenACC 'kernels' decomposition [PR100400, PR103836, PR104061] " Thomas Schwinge
2022-01-19 23:00     ` Jakub Jelinek
2022-01-20  8:26       ` Catch 'GIMPLE_DEBUG' misbehavior in OpenACC 'kernels' decomposition [PR100400, PR103836, PR104061] Thomas Schwinge
2022-01-20  9:58         ` Jakub Jelinek
2022-03-12 12:38   ` Add 'c-c++-common/goacc/kernels-decompose-pr104086-1.c' [PR104086] Thomas Schwinge
2022-03-12 12:42     ` OpenACC 'kernels' decomposition: Mark variables used in 'present' clauses as addressable [PR100280, PR104086] Thomas Schwinge
2022-03-17  8:04   ` Enhance further testcases to verify Openacc 'kernels' decomposition Thomas Schwinge

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).