[Bug gcov-profile/53547] New: Changing the source file between -fprofile-generate and -fprofile-use can lead to performance degradation

[Bug gcov-profile/53547] New: Changing the source file between -fprofile-generate and -fprofile-use can lead to performance degradation

[asharif at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53547

             Bug #: 53547
           Summary: Changing the source file between -fprofile-generate
                    and -fprofile-use can lead to performance degradation
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: gcov-profile
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: asharif@gcc.gnu.org


Created attachment 27536
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=27536
Test case.

Please take a look at the attached test case.

gcc -O2 -fprofile-generate test.c
rm -rf test.gcda && ./a.out
gcc -O2 -fprofile-use -DSTALE test.c

Note that with -DSTALE, a function is added which changes the function id
ordering within the module. With this re-ordering, the profile data no longer
matches up with the updated source file.

What is worse is that the performance actually *drops* below that of regular
-O2 (without -fprofile-use). This is because the summary information (sum_max,
etc.) is still valid while the edge count is considered to be 0 for all edges.
This leads to pessimistic inlining decisions.

Here is a patch that fixes solves the performance loss:
http://codereview.appspot.com/5989046/

[Bug gcov-profile/53547] Changing the source file between -fprofile-generate and -fprofile-use can lead to performance degradation

[asharif at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53547

--- Comment #1 from asharif at gcc dot gnu.org 2012-06-01 00:22:06 UTC ---
Created attachment 27537
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=27537
Proposed fix.

[Bug gcov-profile/53547] Changing the source file between -fprofile-generate and -fprofile-use can lead to performance degradation

[steven at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53547

Steven Bosscher <steven at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2012-07-09
                 CC|                            |steven at gcc dot gnu.org
     Ever Confirmed|0                           |1
           Severity|normal                      |enhancement

--- Comment #2 from Steven Bosscher <steven at gcc dot gnu.org> 2012-07-09 15:32:33 UTC ---
Doctor, it hurts when i do this...

It seems to me that one deserves a hard error if the source of the profile-use
compilation is different from the profile-generate source. But perhaps you have
your reasons, and can explain them?

[Bug gcov-profile/53547] Changing the source file between -fprofile-generate and -fprofile-use can lead to performance degradation

[asharif at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53547

--- Comment #3 from asharif at gcc dot gnu.org 2012-07-10 12:00:21 UTC ---
+cc: Jan Hubicka

The use case here represents what happened as we were deploying for Chrome (the
browser). Chrome is a moving source base and they did not want to integrate PGO
into their build system. Doing so would require their nightly builders to run
instrumented Chrome binaries on various devices (example, x86- and arm- based
devices), get the PGO data and rebuild the optimized binary.

Instead they wanted to generate PGO data offline and use it for slightly newer
code. That way they could decouple the generation of PGO data from their build
system. That is understandable because changing the build systems for large
projects is a tedious task.

Currently there are problems with offline PGO:

1. gcc can ICE in certain situations when the profile is out of date.
2. gcc thinks that the mismatched functions execute exactly 0 times and still
considers the "program summary" of PGO to be valid (that includes run_max,
sum_max, etc.). This misleads gcc into believing that the hot functions are
cold (if they have mismatches).

gcc should distinguish the case where the edge counts are truly 0, vs. the case
where there is a profile mismatch. My patch addresses that.

In general, I think PGO will get higher adoption if we make it resilient to
slight changes in source code. That way developers can generate good PGO data
independently of their nightly builds.

Right now the situation is such that gcc's profile files have function ids with
checksums instead of function names. If we add a single dummy function to the
compilation unit, all the function ids are different and the entire profile
file has checksum mismatches. My patch just addresses a part of the problem. If
gcc uses function names instead of ids, gcc will be able to tolerate more
source changes and I think PGO adoption will increase.