* [PATCH] Fixes for profile count/probability maintenance
@ 2023-09-19 18:16 Eugene Rozenfeld
2023-09-27 11:32 ` Richard Biener
0 siblings, 1 reply; 3+ messages in thread
From: Eugene Rozenfeld @ 2023-09-19 18:16 UTC (permalink / raw)
To: gcc-patches
Verifier checks have recently been strengthened to check that
all counts and probabilities are initialized. The checks fired
during autoprofiledbootstrap build and this patch fixes it.
gcc/ChangeLog:
* auto-profile.cc (afdo_calculate_branch_prob): Fix count comparisons
* ipa-utils.cc (ipa_merge_profiles): Guard against zero count when
computing probabilities
* tree-vect-loop-manip.cc (vect_do_peeling): Guard against zero count
when scaling loop profile
Tested on x86_64-pc-linux-gnu.
---
gcc/auto-profile.cc | 4 ++--
gcc/ipa-utils.cc | 16 +++++++++-------
gcc/tree-vect-loop-manip.cc | 2 +-
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/gcc/auto-profile.cc b/gcc/auto-profile.cc
index ff3b763945c..3e61f36c29b 100644
--- a/gcc/auto-profile.cc
+++ b/gcc/auto-profile.cc
@@ -1434,7 +1434,7 @@ afdo_calculate_branch_prob (bb_set *annotated_bb)
else
total_count += AFDO_EINFO (e)->get_count ();
}
- if (num_unknown_succ == 0 && total_count > profile_count::zero ())
+ if (num_unknown_succ == 0 && total_count > profile_count::zero ().afdo ())
{
FOR_EACH_EDGE (e, ei, bb->succs)
e->probability
@@ -1571,7 +1571,7 @@ afdo_annotate_cfg (const stmt_set &promoted_stmts)
DECL_SOURCE_LOCATION (current_function_decl));
afdo_source_profile->mark_annotated (cfun->function_start_locus);
afdo_source_profile->mark_annotated (cfun->function_end_locus);
- if (max_count > profile_count::zero ())
+ if (max_count > profile_count::zero ().afdo ())
{
/* Calculate, propagate count and probability information on CFG. */
afdo_calculate_branch_prob (&annotated_bb);
diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
index 956c6294fd7..3aaf7e595df 100644
--- a/gcc/ipa-utils.cc
+++ b/gcc/ipa-utils.cc
@@ -651,13 +651,15 @@ ipa_merge_profiles (struct cgraph_node *dst,
{
edge srce = EDGE_SUCC (srcbb, i);
edge dste = EDGE_SUCC (dstbb, i);
- dste->probability =
- dste->probability * dstbb->count.ipa ().probability_in
- (dstbb->count.ipa ()
- + srccount.ipa ())
- + srce->probability * srcbb->count.ipa ().probability_in
- (dstbb->count.ipa ()
- + srccount.ipa ());
+ profile_count total = dstbb->count.ipa () + srccount.ipa ();
+ if (total.nonzero_p ())
+ {
+ dste->probability =
+ dste->probability * dstbb->count.ipa ().probability_in
+ (total)
+ + srce->probability * srcbb->count.ipa ().probability_in
+ (total);
+ }
}
dstbb->count = dstbb->count.ipa () + srccount.ipa ();
}
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
index 09641901ff1..2608c286e5d 100644
--- a/gcc/tree-vect-loop-manip.cc
+++ b/gcc/tree-vect-loop-manip.cc
@@ -3335,7 +3335,7 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
free (bbs);
free (original_bbs);
}
- else
+ else if (old_count.nonzero_p ())
scale_loop_profile (epilog, guard_to->count.probability_in (old_count), -1);
/* Only need to handle basic block before epilog loop if it's not
--
2.25.1
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] Fixes for profile count/probability maintenance
2023-09-19 18:16 [PATCH] Fixes for profile count/probability maintenance Eugene Rozenfeld
@ 2023-09-27 11:32 ` Richard Biener
2023-09-27 11:40 ` Jan Hubicka
0 siblings, 1 reply; 3+ messages in thread
From: Richard Biener @ 2023-09-27 11:32 UTC (permalink / raw)
To: Eugene Rozenfeld, Jan Hubicka; +Cc: gcc-patches
On Tue, Sep 19, 2023 at 8:17 PM Eugene Rozenfeld
<Eugene.Rozenfeld@microsoft.com> wrote:
>
> Verifier checks have recently been strengthened to check that
> all counts and probabilities are initialized. The checks fired
> during autoprofiledbootstrap build and this patch fixes it.
OK if Honza doesn't have any comments this week.
Richard.
> gcc/ChangeLog:
>
> * auto-profile.cc (afdo_calculate_branch_prob): Fix count comparisons
> * ipa-utils.cc (ipa_merge_profiles): Guard against zero count when
> computing probabilities
> * tree-vect-loop-manip.cc (vect_do_peeling): Guard against zero count
> when scaling loop profile
>
> Tested on x86_64-pc-linux-gnu.
>
> ---
> gcc/auto-profile.cc | 4 ++--
> gcc/ipa-utils.cc | 16 +++++++++-------
> gcc/tree-vect-loop-manip.cc | 2 +-
> 3 files changed, 12 insertions(+), 10 deletions(-)
>
> diff --git a/gcc/auto-profile.cc b/gcc/auto-profile.cc
> index ff3b763945c..3e61f36c29b 100644
> --- a/gcc/auto-profile.cc
> +++ b/gcc/auto-profile.cc
> @@ -1434,7 +1434,7 @@ afdo_calculate_branch_prob (bb_set *annotated_bb)
> else
> total_count += AFDO_EINFO (e)->get_count ();
> }
> - if (num_unknown_succ == 0 && total_count > profile_count::zero ())
> + if (num_unknown_succ == 0 && total_count > profile_count::zero ().afdo ())
> {
> FOR_EACH_EDGE (e, ei, bb->succs)
> e->probability
> @@ -1571,7 +1571,7 @@ afdo_annotate_cfg (const stmt_set &promoted_stmts)
> DECL_SOURCE_LOCATION (current_function_decl));
> afdo_source_profile->mark_annotated (cfun->function_start_locus);
> afdo_source_profile->mark_annotated (cfun->function_end_locus);
> - if (max_count > profile_count::zero ())
> + if (max_count > profile_count::zero ().afdo ())
> {
> /* Calculate, propagate count and probability information on CFG. */
> afdo_calculate_branch_prob (&annotated_bb);
> diff --git a/gcc/ipa-utils.cc b/gcc/ipa-utils.cc
> index 956c6294fd7..3aaf7e595df 100644
> --- a/gcc/ipa-utils.cc
> +++ b/gcc/ipa-utils.cc
> @@ -651,13 +651,15 @@ ipa_merge_profiles (struct cgraph_node *dst,
> {
> edge srce = EDGE_SUCC (srcbb, i);
> edge dste = EDGE_SUCC (dstbb, i);
> - dste->probability =
> - dste->probability * dstbb->count.ipa ().probability_in
> - (dstbb->count.ipa ()
> - + srccount.ipa ())
> - + srce->probability * srcbb->count.ipa ().probability_in
> - (dstbb->count.ipa ()
> - + srccount.ipa ());
> + profile_count total = dstbb->count.ipa () + srccount.ipa ();
> + if (total.nonzero_p ())
> + {
> + dste->probability =
> + dste->probability * dstbb->count.ipa ().probability_in
> + (total)
> + + srce->probability * srcbb->count.ipa ().probability_in
> + (total);
> + }
> }
> dstbb->count = dstbb->count.ipa () + srccount.ipa ();
> }
> diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> index 09641901ff1..2608c286e5d 100644
> --- a/gcc/tree-vect-loop-manip.cc
> +++ b/gcc/tree-vect-loop-manip.cc
> @@ -3335,7 +3335,7 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
> free (bbs);
> free (original_bbs);
> }
> - else
> + else if (old_count.nonzero_p ())
> scale_loop_profile (epilog, guard_to->count.probability_in (old_count), -1);
>
> /* Only need to handle basic block before epilog loop if it's not
> --
> 2.25.1
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] Fixes for profile count/probability maintenance
2023-09-27 11:32 ` Richard Biener
@ 2023-09-27 11:40 ` Jan Hubicka
0 siblings, 0 replies; 3+ messages in thread
From: Jan Hubicka @ 2023-09-27 11:40 UTC (permalink / raw)
To: Richard Biener; +Cc: Eugene Rozenfeld, gcc-patches
> > gcc/ChangeLog:
> >
> > * auto-profile.cc (afdo_calculate_branch_prob): Fix count comparisons
> > * ipa-utils.cc (ipa_merge_profiles): Guard against zero count when
> > computing probabilities
> > * tree-vect-loop-manip.cc (vect_do_peeling): Guard against zero count
> > when scaling loop profile
> >
> > Tested on x86_64-pc-linux-gnu.
> >
> > ---
> > gcc/auto-profile.cc | 4 ++--
> > gcc/ipa-utils.cc | 16 +++++++++-------
> > gcc/tree-vect-loop-manip.cc | 2 +-
> > 3 files changed, 12 insertions(+), 10 deletions(-)
> >
> > diff --git a/gcc/auto-profile.cc b/gcc/auto-profile.cc
> > index ff3b763945c..3e61f36c29b 100644
> > --- a/gcc/auto-profile.cc
> > +++ b/gcc/auto-profile.cc
> > @@ -1434,7 +1434,7 @@ afdo_calculate_branch_prob (bb_set *annotated_bb)
> > else
> > total_count += AFDO_EINFO (e)->get_count ();
> > }
> > - if (num_unknown_succ == 0 && total_count > profile_count::zero ())
> > + if (num_unknown_succ == 0 && total_count > profile_count::zero ().afdo ())
I think you want nonzero_p() here (it is usual guard to use when
computing probability_in)
> > @@ -651,13 +651,15 @@ ipa_merge_profiles (struct cgraph_node *dst,
> > {
> > edge srce = EDGE_SUCC (srcbb, i);
> > edge dste = EDGE_SUCC (dstbb, i);
> > - dste->probability =
> > - dste->probability * dstbb->count.ipa ().probability_in
> > - (dstbb->count.ipa ()
> > - + srccount.ipa ())
> > - + srce->probability * srcbb->count.ipa ().probability_in
> > - (dstbb->count.ipa ()
> > - + srccount.ipa ());
> > + profile_count total = dstbb->count.ipa () + srccount.ipa ();
> > + if (total.nonzero_p ())
> > + {
> > + dste->probability =
> > + dste->probability * dstbb->count.ipa ().probability_in
> > + (total)
> > + + srce->probability * srcbb->count.ipa ().probability_in
> > + (total);
> > + }
> > }
> > dstbb->count = dstbb->count.ipa () + srccount.ipa ();
> > }
> > diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc
> > index 09641901ff1..2608c286e5d 100644
> > --- a/gcc/tree-vect-loop-manip.cc
> > +++ b/gcc/tree-vect-loop-manip.cc
> > @@ -3335,7 +3335,7 @@ vect_do_peeling (loop_vec_info loop_vinfo, tree niters, tree nitersm1,
> > free (bbs);
> > free (original_bbs);
> > }
> > - else
> > + else if (old_count.nonzero_p ())
> > scale_loop_profile (epilog, guard_to->count.probability_in (old_count), -1);
This is fine. Sorry for late reaction. I had some travelling in last
two weeks.
Honza
> >
> > /* Only need to handle basic block before epilog loop if it's not
> > --
> > 2.25.1
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-09-27 11:40 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-19 18:16 [PATCH] Fixes for profile count/probability maintenance Eugene Rozenfeld
2023-09-27 11:32 ` Richard Biener
2023-09-27 11:40 ` Jan Hubicka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).