From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by sourceware.org (Postfix) with ESMTPS id 101A5382EF03 for ; Mon, 27 May 2024 11:55:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 101A5382EF03 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=baylibre.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=baylibre.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 101A5382EF03 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::32b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1716810937; cv=none; b=A7FEbxH1PQ+kJOdj2j9xLRLloZ2kgVfg/13XGXKllpT+yw+CsXue8jP5WTYCO3iSz9j7j7kykoW6Zyp0XSXhSWLIQWdhbSC+NIvpuv8Nkg71LsVDerjGshjEgcXdRUWOGm4Jm/zylVVGV7vbhW9goXXBWG4Y/Rgxn2mFyg893ig= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1716810937; c=relaxed/simple; bh=oShRI+Mj94bTkbxkItzjUc1WgLBjH0zETdleAz3jtxI=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=wulTOrcvW1o6ZTgtlNOVMe3hz2VK+iFE/uYCreaUiDWhr93grribeOk/Bx1LqMTQjQ87Iso/otlPQEn1+yyVmJhTHU/QJ/0bzHl9xeG0Ef4XQrFnqoiZiTbczOXIFBndJs3Pn0TopH2R2W0hXRiNZUbDxDCKXVLsBbPhe1hDOkw= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-420180b58c3so84401695e9.2 for ; Mon, 27 May 2024 04:55:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1716810931; x=1717415731; darn=gcc.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QHtXc2NSa6J1nO/qNeDenUCLybMZI1/h9rWZO2m7dks=; b=pFhO6MqteiUHEkVvxipLdscAcdMrU4Zje6Yma1zkei/wBYmh1yzvoeTVArBmxuWvGj 22tqeWyE+ls6eCJp/cHlHrehdV2Mg5LPFIApEhKFp1RKU14QCnnKoqqzwucbDVgPVSr5 sRK8PYnv0rG3roW//SL19XIz6KWtIBlFCkk0gyRzy2gQ66kZkrA1XoDVymQYqwli83+K eAVyE+lFhXua/W4Mw72LRegZF9QQwOYFAtaeUlMS7JoQtnzLHj0zpZk/p+HgOnP4WCKK QtYEWuQcpfzbVpgdf6/MfqWElYNgC8vtR1f4P9LWWZjSwcnwFh8QAbmn1qSr2cWZOcqN 2KCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716810931; x=1717415731; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QHtXc2NSa6J1nO/qNeDenUCLybMZI1/h9rWZO2m7dks=; b=KmxdiEI4l+B8TSaKpd2mjwOqNrUzqprj5i1qu3SnIeeIjwl0YtcRYQo3t0x8YdQs3d F2mOIdBrkZ0XbfVbVXlmB+w5LdTHnUXjU9OYFYIw2WhHIq9Uv3tCAKxn3ljXQXSc6+9G 7bmdMbA/7RTsLiJIlpRZjSCYnypkRdxZZzoeSLhL4GBgqDNI+tORKpI/3E5y+mvy8aMl Py+iVtLJx4i0qwOW0VX2IGIlbvZx+xUGMj/rkiGb5MgeUOhZeU2lWsSYNoBP67w0uX5H B/+7S8re/yawdVsRggtxhUx2wjXU+ejjZ6bbru45DBevmRWqXUVvIpLJs521JkSXhron ojbg== X-Gm-Message-State: AOJu0Yz1QlgLwDlEMQCzrSKiLXy/6jW89loKuYXO1T2rkg3xo4+qHHWG rilWeYQ+5kDhOIJ/ajWfqveaWIuJCQWEocY2gMvxCv38G1+N1cr9deguhN44dynN1ojezrrE231 G X-Google-Smtp-Source: AGHT+IGA6e+SjC6GQeQ8qxCQILqEeUx4SPhv5mThkAs/G53DN+EZ+8tvhgXRMCPOPcQMP+ZdNrrJrQ== X-Received: by 2002:a05:600c:444e:b0:420:14fb:de1f with SMTP id 5b1f17b1804b1-42108a5965fmr59542945e9.14.1716810930865; Mon, 27 May 2024 04:55:30 -0700 (PDT) Received: from localhost.localdomain ([2a0d:3344:2340:ce10::2f3]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42108966b58sm107636955e9.3.2024.05.27.04.55.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 May 2024 04:55:30 -0700 (PDT) From: Paul-Antoine Arras To: gcc-patches@gcc.gnu.org Cc: Paul-Antoine Arras Subject: [PATCH 2/7] OpenMP: middle-end support for dispatch + adjust_args Date: Mon, 27 May 2024 13:54:34 +0200 Message-ID: <20240527115439.3967217-3-parras@baylibre.com> X-Mailer: git-send-email 2.45.1 In-Reply-To: <20240527115439.3967217-1-parras@baylibre.com> References: <20240527115439.3967217-1-parras@baylibre.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,GIT_PATCH_0,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This patch adds middle-end support for the `dispatch` construct and the `adjust_args` clause. The heavy lifting is done in `gimplify_omp_dispatch` and `gimplify_call_expr` respectively. For `adjust_args`, this mostly consists in emitting a call to `gomp_get_mapped_ptr` for the adequate device. For dispatch, the following steps are performed: * Handle the device clause, if any. This may affect `need_device_ptr` arguments. * Handle novariants and nocontext clauses, if any. Evaluate compile-time constants and select a variant, if possible. Otherwise, emit code to handle all possible cases at run time. * Create an explicit task, as if the `task` construct was used, that wraps the body of the `dispatch` statement. Move relevant clauses to the task. gcc/ChangeLog: * gimple-low.cc (lower_stmt): Handle GIMPLE_OMP_DISPATCH. * gimple-pretty-print.cc (dump_gimple_omp_dispatch): New function. (pp_gimple_stmt_1): Handle GIMPLE_OMP_DISPATCH. * gimple-walk.cc (walk_gimple_stmt): Likewise. * gimple.cc (gimple_build_omp_dispatch): New function. (gimple_copy): Handle GIMPLE_OMP_DISPATCH. * gimple.def (GIMPLE_OMP_DISPATCH): Define. * gimple.h (gimple_build_omp_dispatch): Declare. (gimple_has_substatements): Handle GIMPLE_OMP_DISPATCH. (gimple_omp_dispatch_clauses): New function. (gimple_omp_dispatch_clauses_ptr): Likewise. (gimple_omp_dispatch_set_clauses): Likewise. (gimple_return_set_retval): Handle GIMPLE_OMP_DISPATCH. * gimplify.cc (enum omp_region_type): Add ORT_DISPATCH. (gimplify_call_expr): Handle need_device_ptr arguments. (is_gimple_stmt): Handle OMP_DISPATCH. (gimplify_scan_omp_clauses): Handle OMP_CLAUSE_DEVICE in a dispatch construct. Handle OMP_CLAUSE_NOVARIANTS and OMP_CLAUSE_NOCONTEXT. (gimplify_adjust_omp_clauses): Handle OMP_CLAUSE_NOVARIANTS and OMP_CLAUSE_NOCONTEXT. (omp_construct_selector_matches): Handle OMP_DISPATCH with nocontext clause. (omp_has_novariants): New function. (omp_has_nocontext): Likewise. (gimplify_omp_dispatch): Likewise. (gimplify_expr): Handle OMP_DISPATCH. * gimplify.h (omp_has_novariants): Declare. (omp_has_nocontext): Declare. * omp-builtins.def (BUILT_IN_OMP_GET_MAPPED_PTR): Define. (BUILT_IN_OMP_GET_DEFAULT_DEVICE): Define. (BUILT_IN_OMP_SET_DEFAULT_DEVICE): Define. * omp-expand.cc (expand_omp_dispatch): New function. (expand_omp): Handle GIMPLE_OMP_DISPATCH. (omp_make_gimple_edges): Likewise. * omp-general.cc (omp_construct_traits_to_codes): Add OMP_DISPATCH. (struct omp_ts_info): Add dispatch. (omp_context_selector_matches): Handle OMP_TRAIT_SET_NEED_DEVICE_PTR. (omp_resolve_declare_variant): Handle novariants. Adjust DECL_ASSEMBLER_NAME. --- gcc/gimple-low.cc | 1 + gcc/gimple-pretty-print.cc | 33 +++ gcc/gimple-walk.cc | 1 + gcc/gimple.cc | 20 ++ gcc/gimple.def | 5 + gcc/gimple.h | 33 ++- gcc/gimplify.cc | 417 ++++++++++++++++++++++++++++++++++++- gcc/gimplify.h | 2 + gcc/omp-builtins.def | 6 + gcc/omp-expand.cc | 18 ++ gcc/omp-general.cc | 16 +- gcc/omp-low.cc | 35 ++++ gcc/tree-inline.cc | 7 + 13 files changed, 583 insertions(+), 11 deletions(-) diff --git a/gcc/gimple-low.cc b/gcc/gimple-low.cc index e0371988705..712a1ebf776 100644 --- a/gcc/gimple-low.cc +++ b/gcc/gimple-low.cc @@ -746,6 +746,7 @@ lower_stmt (gimple_stmt_iterator *gsi, struct lower_data *data) case GIMPLE_EH_MUST_NOT_THROW: case GIMPLE_OMP_FOR: case GIMPLE_OMP_SCOPE: + case GIMPLE_OMP_DISPATCH: case GIMPLE_OMP_SECTIONS: case GIMPLE_OMP_SECTIONS_SWITCH: case GIMPLE_OMP_SECTION: diff --git a/gcc/gimple-pretty-print.cc b/gcc/gimple-pretty-print.cc index a71e1e0efc7..d9a24ad2169 100644 --- a/gcc/gimple-pretty-print.cc +++ b/gcc/gimple-pretty-print.cc @@ -1726,6 +1726,35 @@ dump_gimple_omp_scope (pretty_printer *buffer, const gimple *gs, } } +/* Dump a GIMPLE_OMP_DISPATCH tuple on the pretty_printer BUFFER. */ + +static void +dump_gimple_omp_dispatch (pretty_printer *buffer, const gimple *gs, int spc, + dump_flags_t flags) +{ + if (flags & TDF_RAW) + { + dump_gimple_fmt (buffer, spc, flags, "%G <%+BODY <%S>%nCLAUSES <", gs, + gimple_omp_body (gs)); + dump_omp_clauses (buffer, gimple_omp_dispatch_clauses (gs), spc, flags); + dump_gimple_fmt (buffer, spc, flags, " >"); + } + else + { + pp_string (buffer, "#pragma omp dispatch"); + dump_omp_clauses (buffer, gimple_omp_dispatch_clauses (gs), spc, flags); + if (!gimple_seq_empty_p (gimple_omp_body (gs))) + { + newline_and_indent (buffer, spc + 2); + pp_left_brace (buffer); + pp_newline (buffer); + dump_gimple_seq (buffer, gimple_omp_body (gs), spc + 4, flags); + newline_and_indent (buffer, spc + 2); + pp_right_brace (buffer); + } + } +} + /* Dump a GIMPLE_OMP_TARGET tuple on the pretty_printer BUFFER. */ static void @@ -2805,6 +2834,10 @@ pp_gimple_stmt_1 (pretty_printer *buffer, const gimple *gs, int spc, dump_gimple_omp_scope (buffer, gs, spc, flags); break; + case GIMPLE_OMP_DISPATCH: + dump_gimple_omp_dispatch (buffer, gs, spc, flags); + break; + case GIMPLE_OMP_MASTER: case GIMPLE_OMP_SECTION: case GIMPLE_OMP_STRUCTURED_BLOCK: diff --git a/gcc/gimple-walk.cc b/gcc/gimple-walk.cc index 9f768ca20fd..1122713a98b 100644 --- a/gcc/gimple-walk.cc +++ b/gcc/gimple-walk.cc @@ -707,6 +707,7 @@ walk_gimple_stmt (gimple_stmt_iterator *gsi, walk_stmt_fn callback_stmt, case GIMPLE_OMP_PARALLEL: case GIMPLE_OMP_TASK: case GIMPLE_OMP_SCOPE: + case GIMPLE_OMP_DISPATCH: case GIMPLE_OMP_SECTIONS: case GIMPLE_OMP_SINGLE: case GIMPLE_OMP_TARGET: diff --git a/gcc/gimple.cc b/gcc/gimple.cc index a9f968cb038..3a26c74a105 100644 --- a/gcc/gimple.cc +++ b/gcc/gimple.cc @@ -1235,6 +1235,21 @@ gimple_build_omp_scope (gimple_seq body, tree clauses) return p; } +/* Build a GIMPLE_OMP_DISPATCH statement. + + BODY is the target function call to be dispatched. + CLAUSES are any of the OMP dispatch construct's clauses: ... */ + +gimple * +gimple_build_omp_dispatch (gimple_seq body, tree clauses) +{ + gimple *p = gimple_alloc (GIMPLE_OMP_DISPATCH, 0); + gimple_omp_dispatch_set_clauses (p, clauses); + if (body) + gimple_omp_set_body (p, body); + + return p; +} /* Build a GIMPLE_OMP_TARGET statement. @@ -2148,6 +2163,11 @@ gimple_copy (gimple *stmt) gimple_omp_scope_set_clauses (copy, t); goto copy_omp_body; + case GIMPLE_OMP_DISPATCH: + t = unshare_expr (gimple_omp_dispatch_clauses (stmt)); + gimple_omp_dispatch_set_clauses (copy, t); + goto copy_omp_body; + case GIMPLE_OMP_TARGET: { gomp_target *omp_target_stmt = as_a (stmt); diff --git a/gcc/gimple.def b/gcc/gimple.def index fbcd727f945..21c7405875d 100644 --- a/gcc/gimple.def +++ b/gcc/gimple.def @@ -350,6 +350,11 @@ DEFGSCODE(GIMPLE_OMP_SCAN, "gimple_omp_scan", GSS_OMP_SINGLE_LAYOUT) CLAUSES is an OMP_CLAUSE chain holding the associated clauses. */ DEFGSCODE(GIMPLE_OMP_SCOPE, "gimple_omp_scope", GSS_OMP_SINGLE_LAYOUT) +/* GIMPLE_OMP_DISPATCH represents #pragma omp dispatch + BODY is the target function call to be dispatched. + CLAUSES is an OMP_CLAUSE chain holding the associated clauses. */ +DEFGSCODE(GIMPLE_OMP_DISPATCH, "gimple_omp_dispatch", GSS_OMP_SINGLE_LAYOUT) + /* OMP_SECTION represents #pragma omp section. BODY is the sequence of statements in the section body. */ DEFGSCODE(GIMPLE_OMP_SECTION, "gimple_omp_section", GSS_OMP) diff --git a/gcc/gimple.h b/gcc/gimple.h index bd315ffc2dd..25590a22ffb 100644 --- a/gcc/gimple.h +++ b/gcc/gimple.h @@ -746,7 +746,7 @@ struct GTY((tag("GSS_OMP_CONTINUE"))) }; /* GIMPLE_OMP_SINGLE, GIMPLE_OMP_ORDERED, GIMPLE_OMP_TASKGROUP, - GIMPLE_OMP_SCAN, GIMPLE_OMP_MASKED, GIMPLE_OMP_SCOPE. */ + GIMPLE_OMP_SCAN, GIMPLE_OMP_MASKED, GIMPLE_OMP_SCOPE, GIMPLE_OMP_DISPATCH. */ struct GTY((tag("GSS_OMP_SINGLE_LAYOUT"))) gimple_statement_omp_single_layout : public gimple_statement_omp @@ -1595,6 +1595,7 @@ gomp_task *gimple_build_omp_task (gimple_seq, tree, tree, tree, tree, gimple *gimple_build_omp_section (gimple_seq); gimple *gimple_build_omp_structured_block (gimple_seq); gimple *gimple_build_omp_scope (gimple_seq, tree); +gimple *gimple_build_omp_dispatch (gimple_seq, tree); gimple *gimple_build_omp_master (gimple_seq); gimple *gimple_build_omp_masked (gimple_seq, tree); gimple *gimple_build_omp_taskgroup (gimple_seq, tree); @@ -1886,6 +1887,7 @@ gimple_has_substatements (gimple *g) case GIMPLE_OMP_PARALLEL: case GIMPLE_OMP_TASK: case GIMPLE_OMP_SCOPE: + case GIMPLE_OMP_DISPATCH: case GIMPLE_OMP_SECTIONS: case GIMPLE_OMP_SINGLE: case GIMPLE_OMP_TARGET: @@ -5437,6 +5439,34 @@ gimple_omp_scope_set_clauses (gimple *gs, tree clauses) = clauses; } +/* Return the clauses associated with OMP_DISPATCH statement GS. */ + +inline tree +gimple_omp_dispatch_clauses (const gimple *gs) +{ + GIMPLE_CHECK (gs, GIMPLE_OMP_DISPATCH); + return static_cast (gs)->clauses; +} + +/* Return a pointer to the clauses associated with OMP dispatch statement + GS. */ + +inline tree * +gimple_omp_dispatch_clauses_ptr (gimple *gs) +{ + GIMPLE_CHECK (gs, GIMPLE_OMP_DISPATCH); + return &static_cast (gs)->clauses; +} + +/* Set CLAUSES to be the clauses associated with OMP dispatch statement + GS. */ + +inline void +gimple_omp_dispatch_set_clauses (gimple *gs, tree clauses) +{ + GIMPLE_CHECK (gs, GIMPLE_OMP_DISPATCH); + static_cast (gs)->clauses = clauses; +} /* Return the kind of the OMP_FOR statemement G. */ @@ -6771,6 +6801,7 @@ gimple_return_set_retval (greturn *gs, tree retval) case GIMPLE_OMP_TARGET: \ case GIMPLE_OMP_TEAMS: \ case GIMPLE_OMP_SCOPE: \ + case GIMPLE_OMP_DISPATCH: \ case GIMPLE_OMP_SECTION: \ case GIMPLE_OMP_STRUCTURED_BLOCK: \ case GIMPLE_OMP_MASTER: \ diff --git a/gcc/gimplify.cc b/gcc/gimplify.cc index b0ed58ed0f9..1dd69dbb1de 100644 --- a/gcc/gimplify.cc +++ b/gcc/gimplify.cc @@ -161,7 +161,8 @@ enum omp_region_type { ORT_WORKSHARE = 0x00, ORT_TASKGROUP = 0x01, - ORT_SIMD = 0x04, + ORT_DISPATCH = 0x02, + ORT_SIMD = 0x04, ORT_PARALLEL = 0x08, ORT_COMBINED_PARALLEL = ORT_PARALLEL | 1, @@ -4051,6 +4052,7 @@ gimplify_call_expr (tree *expr_p, gimple_seq *pre_p, bool want_value) /* Gimplify the function arguments. */ if (nargs > 0) { + tree device_num = NULL_TREE; for (i = (PUSH_ARGS_REVERSED ? nargs - 1 : 0); PUSH_ARGS_REVERSED ? i >= 0 : i < nargs; PUSH_ARGS_REVERSED ? i-- : i++) @@ -4061,8 +4063,99 @@ gimplify_call_expr (tree *expr_p, gimple_seq *pre_p, bool want_value) be the plain PARM_DECL. */ if ((i != 1) || !builtin_va_start_p) { - t = gimplify_arg (&CALL_EXPR_ARG (*expr_p, i), pre_p, - EXPR_LOCATION (*expr_p), ! returns_twice); + tree *arg_p = &CALL_EXPR_ARG (*expr_p, i); + if (flag_openmp && EXPR_P (CALL_EXPR_FN (*expr_p)) + && lookup_attribute ("omp declare variant variant", + DECL_ATTRIBUTES (TREE_OPERAND ( + CALL_EXPR_FN (*expr_p), 0))) + != NULL_TREE) + { + tree param + = DECL_ARGUMENTS (TREE_OPERAND (CALL_EXPR_FN (*expr_p), 0)); + + if (param != NULL_TREE) + { + for (int param_idx = 0; param_idx < i; param_idx++) + param = TREE_CHAIN (param); + + bool is_device_ptr = false; + if (gimplify_omp_ctxp != NULL + && gimplify_omp_ctxp->code == OMP_DISPATCH) + { + for (tree c = gimplify_omp_ctxp->clauses; c; + c = TREE_CHAIN (c)) + { + if (OMP_CLAUSE_CODE (c) + == OMP_CLAUSE_IS_DEVICE_PTR) + { + tree decl1 = DECL_NAME (OMP_CLAUSE_DECL (c)); + tree decl2 + = tree_strip_nop_conversions (*arg_p); + if (TREE_CODE (decl2) == ADDR_EXPR) + decl2 = TREE_OPERAND (decl2, 0); + gcc_assert (TREE_CODE (decl2) == VAR_DECL + || TREE_CODE (decl2) + == PARM_DECL); + decl2 = DECL_NAME (decl2); + if (decl1 == decl2) + { + is_device_ptr = true; + break; + } + } + else if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_DEVICE) + device_num = OMP_CLAUSE_OPERAND (c, 0); + } + } + + if (!is_device_ptr + && lookup_attribute ("omp declare variant " + "adjust_args need_device_ptr", + DECL_ATTRIBUTES (param)) + != NULL_TREE) + { + if (device_num == NULL_TREE) + { + // device_num = omp_get_default_device(); + tree fn = builtin_decl_explicit ( + BUILT_IN_OMP_GET_DEFAULT_DEVICE); + gcall *call = gimple_build_call (fn, 0); + device_num = create_tmp_var ( + gimple_call_return_type (call)); + gimple_call_set_lhs (call, device_num); + gimplify_seq_add_stmt (pre_p, call); + } + + // mapped_arg = omp_get_mapped_ptr(arg, device_num); + tree fn = builtin_decl_explicit ( + BUILT_IN_OMP_GET_MAPPED_PTR); + *arg_p = (TREE_CODE (*arg_p) == NOP_EXPR) + ? TREE_OPERAND (*arg_p, 0) + : *arg_p; + gimplify_arg (arg_p, pre_p, loc); + gimplify_arg (&device_num, pre_p, loc); + call = gimple_build_call (fn, 2, *arg_p, device_num); + tree mapped_arg + = create_tmp_var (gimple_call_return_type (call)); + gimple_call_set_lhs (call, mapped_arg); + gimplify_seq_add_stmt (pre_p, call); + + *arg_p = mapped_arg; + + // Mark mapped argument as device pointer to ensure + // idempotency in gimplification + gcc_assert (gimplify_omp_ctxp->code == OMP_DISPATCH); + tree c = build_omp_clause (input_location, + OMP_CLAUSE_IS_DEVICE_PTR); + OMP_CLAUSE_DECL (c) = *arg_p; + OMP_CLAUSE_CHAIN (c) = gimplify_omp_ctxp->clauses; + gimplify_omp_ctxp->clauses = c; + } + } + } + + t = gimplify_arg (arg_p, pre_p, EXPR_LOCATION (*expr_p), + !returns_twice); if (t == GS_ERROR) ret = GS_ERROR; @@ -6307,6 +6400,7 @@ is_gimple_stmt (tree t) case OACC_LOOP: case OMP_SCAN: case OMP_SCOPE: + case OMP_DISPATCH: case OMP_SECTIONS: case OMP_SECTION: case OMP_STRUCTURED_BLOCK: @@ -13080,6 +13174,21 @@ gimplify_scan_omp_clauses (tree *list_p, gimple_seq *pre_p, break; } } + else if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_DEVICE + && code == OMP_DISPATCH) + { + bool saved_into_ssa = gimplify_ctxp->into_ssa; + gimplify_ctxp->into_ssa = false; + if (gimplify_expr (&OMP_CLAUSE_DEVICE_ID (c), pre_p, NULL, + is_gimple_val, fb_rvalue) + == GS_ERROR) + remove = true; + else if (DECL_P (OMP_CLAUSE_DEVICE_ID (c))) + omp_add_variable (ctx, OMP_CLAUSE_DEVICE_ID (c), + GOVD_SHARED | GOVD_SEEN); + gimplify_ctxp->into_ssa = saved_into_ssa; + break; + } /* Fall through. */ case OMP_CLAUSE_PRIORITY: @@ -13309,6 +13418,20 @@ gimplify_scan_omp_clauses (tree *list_p, gimple_seq *pre_p, } break; + case OMP_CLAUSE_NOVARIANTS: + { + OMP_CLAUSE_NOVARIANTS_EXPR (c); + tree t = gimple_boolify (OMP_CLAUSE_NOVARIANTS_EXPR (c)); + OMP_CLAUSE_NOVARIANTS_EXPR (c) = t; + } + break; + case OMP_CLAUSE_NOCONTEXT: + { + OMP_CLAUSE_NOCONTEXT_EXPR (c); + tree t = gimple_boolify (OMP_CLAUSE_NOCONTEXT_EXPR (c)); + OMP_CLAUSE_NOCONTEXT_EXPR (c) = t; + } + break; case OMP_CLAUSE_NOHOST: default: gcc_unreachable (); @@ -13763,7 +13886,9 @@ gimplify_adjust_omp_clauses (gimple_seq *pre_p, gimple_seq body, tree *list_p, { struct gimplify_omp_ctx *octx; for (octx = ctx; octx; octx = octx->outer_context) - if ((octx->region_type & (ORT_PARALLEL | ORT_TASK | ORT_TEAMS)) != 0) + if ((octx->region_type + & (ORT_DISPATCH | ORT_PARALLEL | ORT_TASK | ORT_TEAMS)) + != 0) break; if (octx) { @@ -14574,6 +14699,8 @@ gimplify_adjust_omp_clauses (gimple_seq *pre_p, gimple_seq body, tree *list_p, case OMP_CLAUSE_FINALIZE: case OMP_CLAUSE_INCLUSIVE: case OMP_CLAUSE_EXCLUSIVE: + case OMP_CLAUSE_NOVARIANTS: + case OMP_CLAUSE_NOCONTEXT: break; case OMP_CLAUSE_NOHOST: @@ -14663,9 +14790,9 @@ omp_construct_selector_matches (enum tree_code *constructs, int nconstructs, == ORT_TARGET && ctx->code == OMP_TARGET) || ((ctx->region_type & ORT_TEAMS) && ctx->code == OMP_TEAMS) || (ctx->region_type == ORT_WORKSHARE && ctx->code == OMP_FOR) - || (ctx->region_type == ORT_SIMD - && ctx->code == OMP_SIMD - && !omp_find_clause (ctx->clauses, OMP_CLAUSE_BIND))) + || (ctx->region_type == ORT_SIMD && ctx->code == OMP_SIMD + && !omp_find_clause (ctx->clauses, OMP_CLAUSE_BIND)) + || (ctx->code == OMP_DISPATCH && omp_has_nocontext () != 1)) { ++cnt; if (scores) @@ -14783,6 +14910,60 @@ omp_construct_selector_matches (enum tree_code *constructs, int nconstructs, return 0; } +/* Try to evaluate a novariants clause. Return 1 if true, 0 if false or absent, + * -1 if run-time evaluation is needed. */ + +int +omp_has_novariants (void) +{ + for (struct gimplify_omp_ctx *ctx = gimplify_omp_ctxp; ctx; + ctx = ctx->outer_context) + { + if (ctx->code == OMP_DISPATCH) + { + tree c = omp_find_clause (ctx->clauses, OMP_CLAUSE_NOVARIANTS); + if (c != NULL_TREE) + { + if (integer_nonzerop (OMP_CLAUSE_NOVARIANTS_EXPR (c))) + return 1; + else if (integer_zerop (OMP_CLAUSE_NOVARIANTS_EXPR (c))) + return 0; + else + return -1; + } + return 0; + } + } + return 0; +} + +/* Try to evaluate a nocontext clause. Return 1 if true, 0 if false or absent, + * -1 if run-time evaluation is needed. */ + +int +omp_has_nocontext (void) +{ + for (struct gimplify_omp_ctx *ctx = gimplify_omp_ctxp; ctx; + ctx = ctx->outer_context) + { + if (ctx->code == OMP_DISPATCH) + { + tree c = omp_find_clause (ctx->clauses, OMP_CLAUSE_NOCONTEXT); + if (c != NULL_TREE) + { + if (integer_nonzerop (OMP_CLAUSE_NOCONTEXT_EXPR (c))) + return 1; + else if (integer_zerop (OMP_CLAUSE_NOCONTEXT_EXPR (c))) + return 0; + else + return -1; + } + return 0; + } + } + return 0; +} + /* Gimplify OACC_CACHE. */ static void @@ -17614,6 +17795,221 @@ gimplify_omp_ordered (tree expr, gimple_seq body) return gimple_build_omp_ordered (body, OMP_ORDERED_CLAUSES (expr)); } +/* Gimplify an OMP_DISPATCH construct. */ + +static enum gimplify_status +gimplify_omp_dispatch (tree *expr_p, gimple_seq *pre_p) +{ + tree expr = *expr_p; + gimple_seq body = NULL; + + gimplify_scan_omp_clauses (&OMP_DISPATCH_CLAUSES (expr), pre_p, ORT_DISPATCH, + OMP_DISPATCH); + push_gimplify_context (); + + // If device clause, adjust ICV + tree device + = omp_find_clause (OMP_DISPATCH_CLAUSES (expr), OMP_CLAUSE_DEVICE); + if (device) + { + tree t = builtin_decl_explicit (BUILT_IN_OMP_SET_DEFAULT_DEVICE); + t = build_call_expr_loc (input_location, t, 1, + OMP_CLAUSE_DEVICE_ID (device)); + gimplify_and_add (t, &body); + if (DECL_P (OMP_CLAUSE_DEVICE_ID (device))) + omp_notice_variable (gimplify_omp_ctxp, OMP_CLAUSE_DEVICE_ID (device), + true); + } + + // If the novariants and nocontext clauses are not compile-time constants, + // we need to generate code for all possible cases: + // if (novariants) // implies nocontext + // base() + // else if (nocontext) + // variant1() + // else + // variant2() + tree dispatch_body = OMP_DISPATCH_BODY (expr); + if (TREE_CODE (dispatch_body) == BIND_EXPR) + dispatch_body = BIND_EXPR_BODY (dispatch_body); + if (TREE_CODE (dispatch_body) == STATEMENT_LIST) + { + // Fortran FE may insert some pre-call code, for instance when an + // array is passed as argument. Skip to the actual call. + dispatch_body = expr_last (dispatch_body); + } + gcc_assert (TREE_CODE (dispatch_body) == CALL_EXPR + || TREE_CODE (dispatch_body) == MODIFY_EXPR); + tree base_call_expr = dispatch_body; + tree dst; + if (TREE_CODE (base_call_expr) == MODIFY_EXPR) + { + dst = TREE_OPERAND (base_call_expr, 0); + base_call_expr = TREE_OPERAND (base_call_expr, 1); + while (TREE_CODE (base_call_expr) == FLOAT_EXPR + || TREE_CODE (base_call_expr) == CONVERT_EXPR + || TREE_CODE (base_call_expr) == COMPLEX_EXPR) + base_call_expr = TREE_OPERAND (base_call_expr, 0); + } + + tree base_fndecl = get_callee_fndecl (STRIP_NOPS (base_call_expr)); + if (base_fndecl != NULL_TREE) + { + if (DECL_VIRTUAL_P (base_fndecl)) + { + error_at ( + EXPR_LOCATION (base_call_expr), + "%qD is a virtual function but only a direct call is allowed " + "in a dispatch construct", + DECL_NAME (base_fndecl)); + } + + tree variant_fndecl = omp_resolve_declare_variant (base_fndecl); + if (base_fndecl != variant_fndecl + && (omp_has_novariants () == -1 || omp_has_nocontext () == -1)) + { + tree novariants_clause = NULL_TREE, nocontext_clause = NULL_TREE, + novariants_cond = NULL_TREE, nocontext_cond = NULL_TREE; + for (tree c = OMP_DISPATCH_CLAUSES (expr); c; c = TREE_CHAIN (c)) + { + if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NOVARIANTS) + { + gcc_assert (novariants_cond == NULL_TREE); + novariants_clause = c; + novariants_cond = OMP_CLAUSE_NOVARIANTS_EXPR (c); + } + else if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NOCONTEXT) + { + gcc_assert (nocontext_cond == NULL_TREE); + nocontext_clause = c; + nocontext_cond = OMP_CLAUSE_NOCONTEXT_EXPR (c); + } + } + gcc_assert (novariants_cond != NULL_TREE + || nocontext_cond != NULL_TREE); + + enum gimplify_status ret + = gimplify_expr (&novariants_cond, &body, NULL, is_gimple_val, + fb_rvalue); + if (ret == GS_ERROR || ret == GS_UNHANDLED) + return ret; + ret = gimplify_expr (&nocontext_cond, &body, NULL, is_gimple_val, + fb_rvalue); + if (ret == GS_ERROR || ret == GS_UNHANDLED) + return ret; + + tree base_label = create_artificial_label (UNKNOWN_LOCATION); + tree variant1_label = create_artificial_label (UNKNOWN_LOCATION); + tree cond_label = create_artificial_label (UNKNOWN_LOCATION); + tree variant2_label = create_artificial_label (UNKNOWN_LOCATION); + tree end_label = create_artificial_label (UNKNOWN_LOCATION); + + if (novariants_cond != NULL_TREE) + { + gcond *novariants_cond_stmt + = gimple_build_cond_from_tree (novariants_cond, base_label, + cond_label); + gimplify_seq_add_stmt (&body, novariants_cond_stmt); + + gimplify_seq_add_stmt (&body, gimple_build_label (base_label)); + tree base_call_expr2 = copy_node (base_call_expr); + if (TREE_CODE (dispatch_body) == MODIFY_EXPR) + { + base_call_expr2 = build2 (MODIFY_EXPR, TREE_TYPE (dst), dst, + base_call_expr2); + } + OMP_CLAUSE_NOVARIANTS_EXPR (novariants_clause) + = boolean_true_node; + gimplify_and_add (base_call_expr2, &body); + gimplify_seq_add_stmt (&body, gimple_build_goto (end_label)); + + OMP_CLAUSE_NOVARIANTS_EXPR (novariants_clause) + = boolean_false_node; + } + + gimplify_seq_add_stmt (&body, gimple_build_label (cond_label)); + if (nocontext_cond != NULL_TREE) + { + gcond *nocontext_cond_stmt + = gimple_build_cond_from_tree (nocontext_cond, variant1_label, + variant2_label); + gimplify_seq_add_stmt (&body, nocontext_cond_stmt); + + gimplify_seq_add_stmt (&body, + gimple_build_label (variant1_label)); + tree variant_call_expr = copy_node (base_call_expr); + if (TREE_CODE (dispatch_body) == MODIFY_EXPR) + { + variant_call_expr = build2 (MODIFY_EXPR, TREE_TYPE (dst), dst, + variant_call_expr); + } + OMP_CLAUSE_NOCONTEXT_EXPR (nocontext_clause) = boolean_true_node; + gimplify_and_add (variant_call_expr, &body); + gimplify_seq_add_stmt (&body, gimple_build_goto (end_label)); + OMP_CLAUSE_NOCONTEXT_EXPR (nocontext_clause) = boolean_false_node; + } + + gimplify_seq_add_stmt (&body, gimple_build_label (variant2_label)); + tree variant_call_expr = copy_node (base_call_expr); + if (TREE_CODE (dispatch_body) == MODIFY_EXPR) + { + variant_call_expr + = build2 (MODIFY_EXPR, TREE_TYPE (dst), dst, variant_call_expr); + } + gimplify_and_add (variant_call_expr, &body); + gimplify_seq_add_stmt (&body, gimple_build_goto (end_label)); + gimplify_seq_add_stmt (&body, gimple_build_label (end_label)); + } + else + gimplify_and_add (OMP_DISPATCH_BODY (expr), &body); + } + else + gimplify_and_add (OMP_DISPATCH_BODY (expr), &body); + + // Wrap dispatch body into a bind + gimple *bind = gimple_build_bind (NULL_TREE, body, NULL_TREE); + pop_gimplify_context (bind); + + gimplify_adjust_omp_clauses (pre_p, bind, &OMP_DISPATCH_CLAUSES (expr), + OMP_DISPATCH); + + // Move relevant clauses to the task construct + tree task_clauses = NULL_TREE; + tree *task_clauses_ptr = &task_clauses; + bool has_nowait = false; + for (tree c = OMP_DISPATCH_CLAUSES (expr); c; c = OMP_CLAUSE_CHAIN (c)) + { + if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_SHARED + || OMP_CLAUSE_CODE (c) == OMP_CLAUSE_DEPEND) + { + *task_clauses_ptr = c; + task_clauses_ptr = &OMP_CLAUSE_CHAIN (c); + } + else if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_IS_DEVICE_PTR) + { + *task_clauses_ptr + = build_omp_clause (input_location, OMP_CLAUSE_SHARED); + OMP_CLAUSE_DECL (*task_clauses_ptr) = OMP_CLAUSE_DECL (c); + task_clauses_ptr = &OMP_CLAUSE_CHAIN (*task_clauses_ptr); + } + else if (OMP_CLAUSE_CODE (c) == OMP_CLAUSE_NOWAIT) + has_nowait = true; + } + *task_clauses_ptr = build_omp_clause (input_location, OMP_CLAUSE_IF); + OMP_CLAUSE_IF_EXPR (*task_clauses_ptr) + = has_nowait ? boolean_true_node : boolean_false_node; + + // Wrap bind into a task + gimple *task + = gimple_build_omp_task (bind, task_clauses, NULL_TREE, NULL_TREE, + NULL_TREE, NULL_TREE, NULL_TREE); + + gimple *stmt = gimple_build_omp_dispatch (task, OMP_DISPATCH_CLAUSES (expr)); + gimplify_seq_add_stmt (pre_p, stmt); + *expr_p = NULL_TREE; + return GS_ALL_DONE; +} + /* Convert the GENERIC expression tree *EXPR_P to GIMPLE. If the expression produces a value to be used as an operand inside a GIMPLE statement, the value will be stored back in *EXPR_P. This value will @@ -18540,6 +18936,10 @@ gimplify_expr (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p, ret = gimplify_omp_atomic (expr_p, pre_p); break; + case OMP_DISPATCH: + ret = gimplify_omp_dispatch (expr_p, pre_p); + break; + case TRANSACTION_EXPR: ret = gimplify_transaction (expr_p, pre_p); break; @@ -18865,7 +19265,8 @@ gimplify_expr (tree *expr_p, gimple_seq *pre_p, gimple_seq *post_p, && code != OMP_SECTION && code != OMP_STRUCTURED_BLOCK && code != OMP_SINGLE - && code != OMP_SCOPE); + && code != OMP_SCOPE + && code != OMP_DISPATCH); } #endif diff --git a/gcc/gimplify.h b/gcc/gimplify.h index ac3cc8eb552..55aece2b65b 100644 --- a/gcc/gimplify.h +++ b/gcc/gimplify.h @@ -77,6 +77,8 @@ extern enum gimplify_status gimplify_expr (tree *, gimple_seq *, gimple_seq *, bool (*) (tree), fallback_t); int omp_construct_selector_matches (enum tree_code *, int, int *); +int omp_has_novariants (void); +int omp_has_nocontext (void); extern void gimplify_type_sizes (tree, gimple_seq *); extern void gimplify_one_sizepos (tree *, gimple_seq *); diff --git a/gcc/omp-builtins.def b/gcc/omp-builtins.def index 044d5d087b6..c83edabbcc3 100644 --- a/gcc/omp-builtins.def +++ b/gcc/omp-builtins.def @@ -76,6 +76,12 @@ DEF_GOMP_BUILTIN (BUILT_IN_OMP_GET_TEAM_NUM, "omp_get_team_num", BT_FN_INT, ATTR_CONST_NOTHROW_LEAF_LIST) DEF_GOMP_BUILTIN (BUILT_IN_OMP_GET_NUM_TEAMS, "omp_get_num_teams", BT_FN_INT, ATTR_CONST_NOTHROW_LEAF_LIST) +DEF_GOMP_BUILTIN (BUILT_IN_OMP_GET_MAPPED_PTR, "omp_get_mapped_ptr", + BT_FN_PTR_CONST_PTR_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_GOMP_BUILTIN (BUILT_IN_OMP_GET_DEFAULT_DEVICE, "omp_get_default_device", + BT_FN_INT, ATTR_NOTHROW_LEAF_LIST) +DEF_GOMP_BUILTIN (BUILT_IN_OMP_SET_DEFAULT_DEVICE, "omp_set_default_device", + BT_FN_INT, ATTR_NOTHROW_LEAF_LIST) DEF_GOMP_BUILTIN (BUILT_IN_GOMP_ATOMIC_START, "GOMP_atomic_start", BT_FN_VOID, ATTR_NOTHROW_LEAF_LIST) diff --git a/gcc/omp-expand.cc b/gcc/omp-expand.cc index 24287826444..6fa372a550b 100644 --- a/gcc/omp-expand.cc +++ b/gcc/omp-expand.cc @@ -8636,6 +8636,19 @@ expand_omp_single (struct omp_region *region) single_succ_edge (exit_bb)->flags = EDGE_FALLTHRU; } +/* Expand code for an OpenMP dispatch directive... */ + +static void +expand_omp_dispatch (struct omp_region *region) +{ + basic_block entry_bb = region->entry; + gimple_stmt_iterator si = gsi_last_nondebug_bb (entry_bb); + enum gimple_code code = gimple_code (gsi_stmt (si)); + gcc_assert (code == GIMPLE_OMP_DISPATCH); + gsi_remove (&si, true); + single_succ_edge (entry_bb)->flags = EDGE_FALLTHRU; +} + /* Generic expansion for OpenMP synchronization directives: master, ordered and critical. All we need to do here is remove the entry and exit markers for REGION. */ @@ -10654,6 +10667,10 @@ expand_omp (struct omp_region *region) expand_omp_single (region); break; + case GIMPLE_OMP_DISPATCH: + expand_omp_dispatch (region); + break; + case GIMPLE_OMP_ORDERED: { gomp_ordered *ord_stmt @@ -11001,6 +11018,7 @@ omp_make_gimple_edges (basic_block bb, struct omp_region **region, case GIMPLE_OMP_MASTER: case GIMPLE_OMP_MASKED: case GIMPLE_OMP_SCOPE: + case GIMPLE_OMP_DISPATCH: case GIMPLE_OMP_CRITICAL: case GIMPLE_OMP_SECTION: cur_region = new_omp_region (bb, code, cur_region); diff --git a/gcc/omp-general.cc b/gcc/omp-general.cc index 2c095200d5b..d585df400d1 100644 --- a/gcc/omp-general.cc +++ b/gcc/omp-general.cc @@ -1037,7 +1037,7 @@ omp_construct_traits_to_codes (tree ctx, int nconstructs, /* Order must match the OMP_TRAIT_CONSTRUCT_* enumerators in enum omp_ts_code. */ static enum tree_code code_map[] - = { OMP_TARGET, OMP_TEAMS, OMP_PARALLEL, OMP_FOR, OMP_SIMD }; + = { OMP_TARGET, OMP_TEAMS, OMP_PARALLEL, OMP_FOR, OMP_SIMD, OMP_DISPATCH }; for (tree ts = ctx; ts; ts = TREE_CHAIN (ts), i--) { @@ -1130,6 +1130,7 @@ const char *omp_tss_map[] = "target_device", "implementation", "user", + "need_device_ptr", NULL }; @@ -1236,10 +1237,14 @@ struct omp_ts_info omp_ts_map[] = OMP_TRAIT_PROPERTY_CLAUSE_LIST, false, NULL }, + { "dispatch", + (1 << OMP_TRAIT_SET_CONSTRUCT), + OMP_TRAIT_PROPERTY_NONE, false, + NULL + }, { NULL, 0, OMP_TRAIT_PROPERTY_NONE, false, NULL } /* OMP_TRAIT_LAST */ }; - /* Return a name from PROP, a property in selectors accepting name lists. */ @@ -1445,6 +1450,8 @@ omp_context_selector_matches (tree ctx) for (tree tss = ctx; tss; tss = TREE_CHAIN (tss)) { enum omp_tss_code set = OMP_TSS_CODE (tss); + if (set == OMP_TRAIT_SET_NEED_DEVICE_PTR) + continue; tree selectors = OMP_TSS_TRAIT_SELECTORS (tss); /* Immediately reject the match if there are any ignored @@ -2484,6 +2491,9 @@ omp_resolve_declare_variant (tree base) if (cfun && (cfun->curr_properties & PROP_gimple_any) != 0) return omp_resolve_late_declare_variant (base); + if (omp_has_novariants () == 1) + return base; + auto_vec variants; auto_vec defer; bool any_deferred = false; @@ -2630,6 +2640,8 @@ omp_resolve_declare_variant (tree base) (*slot)->variants = entry.variants; tree alt = build_decl (DECL_SOURCE_LOCATION (base), FUNCTION_DECL, DECL_NAME (base), TREE_TYPE (base)); + if (DECL_ASSEMBLER_NAME_SET_P (base)) + SET_DECL_ASSEMBLER_NAME (alt, DECL_ASSEMBLER_NAME (base)); DECL_ARTIFICIAL (alt) = 1; DECL_IGNORED_P (alt) = 1; TREE_STATIC (alt) = 1; diff --git a/gcc/omp-low.cc b/gcc/omp-low.cc index 4d003f42098..693d8ca7d8d 100644 --- a/gcc/omp-low.cc +++ b/gcc/omp-low.cc @@ -4185,6 +4185,11 @@ scan_omp_1_stmt (gimple_stmt_iterator *gsi, bool *handled_ops_p, scan_omp (gimple_omp_body_ptr (stmt), ctx); break; + case GIMPLE_OMP_DISPATCH: + ctx = new_omp_context (stmt, ctx); + scan_omp (gimple_omp_body_ptr (stmt), ctx); + break; + case GIMPLE_OMP_SECTIONS: scan_omp_sections (as_a (stmt), ctx); break; @@ -8926,6 +8931,31 @@ lower_omp_scope (gimple_stmt_iterator *gsi_p, omp_context *ctx) if (BLOCK_VARS (block)) TREE_USED (block) = 1; } + +/* Lower code for an OMP dispatch directive. */ + +static void +lower_omp_dispatch (gimple_stmt_iterator *gsi_p, omp_context *ctx) +{ + tree block; + gimple *stmt = gsi_stmt (*gsi_p); + gbind *bind; + + push_gimplify_context (); + + block = make_node (BLOCK); + bind = gimple_build_bind (NULL, NULL, block); + gsi_replace (gsi_p, bind, true); + + lower_omp (gimple_omp_body_ptr (stmt), ctx); + gimple_bind_set_body (bind, maybe_catch_exception (gimple_omp_body (stmt))); + + pop_gimplify_context (bind); + + gimple_bind_append_vars (bind, ctx->block_vars); + BLOCK_VARS (block) = ctx->block_vars; +} + /* Expand code for an OpenMP master or masked directive. */ static void @@ -14399,6 +14429,11 @@ lower_omp_1 (gimple_stmt_iterator *gsi_p, omp_context *ctx) gcc_assert (ctx); lower_omp_scope (gsi_p, ctx); break; + case GIMPLE_OMP_DISPATCH: + ctx = maybe_lookup_ctx (stmt); + gcc_assert (ctx); + lower_omp_dispatch (gsi_p, ctx); + break; case GIMPLE_OMP_SINGLE: ctx = maybe_lookup_ctx (stmt); gcc_assert (ctx); diff --git a/gcc/tree-inline.cc b/gcc/tree-inline.cc index f31a34ac410..2e06b706025 100644 --- a/gcc/tree-inline.cc +++ b/gcc/tree-inline.cc @@ -1679,6 +1679,12 @@ remap_gimple_stmt (gimple *stmt, copy_body_data *id) (s1, gimple_omp_scope_clauses (stmt)); break; + case GIMPLE_OMP_DISPATCH: + s1 = remap_gimple_seq (gimple_omp_body (stmt), id); + copy = gimple_build_omp_dispatch (s1, + gimple_omp_dispatch_clauses (stmt)); + break; + case GIMPLE_OMP_TASKGROUP: s1 = remap_gimple_seq (gimple_omp_body (stmt), id); copy = gimple_build_omp_taskgroup @@ -4609,6 +4615,7 @@ estimate_num_insns (gimple *stmt, eni_weights *weights) case GIMPLE_OMP_MASTER: case GIMPLE_OMP_MASKED: case GIMPLE_OMP_SCOPE: + case GIMPLE_OMP_DISPATCH: case GIMPLE_OMP_TASKGROUP: case GIMPLE_OMP_ORDERED: case GIMPLE_OMP_SCAN: -- 2.45.1