public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug gcov-profile/105238] New: Regression: using -fprofile-dir: gcno files not ccache cachable anymore / gcovr report broken
@ 2022-04-12 11:32 esgergn at hotmail dot com
  2022-04-12 12:25 ` [Bug gcov-profile/105238] " marxin at gcc dot gnu.org
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: esgergn at hotmail dot com @ 2022-04-12 11:32 UTC (permalink / raw)
  To: gcc-bugs

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105238

            Bug ID: 105238
           Summary: Regression: using -fprofile-dir: gcno files not ccache
                    cachable anymore / gcovr report broken
           Product: gcc
           Version: 9.4.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: gcov-profile
          Assignee: unassigned at gcc dot gnu.org
          Reporter: esgergn at hotmail dot com
                CC: marxin at gcc dot gnu.org
  Target Milestone: ---

Gist:

- gcc 9 / 10 (and later?) gcno files are not ccache cachable anymore because of
the absolute CWD path they contain
- when using -fprofile-dir with gcc 9 and 10 gcovr is not able to produce
correct reports where it can do so when not using -fprofile-dir.


Situation:

CMake project with multiple subprojects with the following structure
(condensed)

/home/esger/src/
    application/
        CMakeLists.txt
        source/
            staticlibA/
                CMakeLists.txt
                source/
                    A.cpp
            staticlibB/
                CMakeLists.txt
                source/
                    B.cpp
            main.cpp
        test/
            CMakeLists.txt
            source/
            test_main.cpp

We are building, with coverage, from different directories with often only
minor code changes between directories, therefore ccache is able to provide a
very significant speedup. When using ccache 4.5.1 (with a small bug fix, see
https://github.com/ccache/ccache/issues/1032) and gcc 7.5.0 this works
perfectly. When compiling for instance B.cpp we use flags:

-fprofile-dir=./build/source/staticlibB/CMakeFiles/staticlibB.dir/source
-fdebug-prefix-map=/home/esger/src/application=.

and

CCACHE_BASEDIR=/home/esger/src/application
(this rewrites the absolute paths CMake uses to relative paths starting at the
base dir. Ie.
/home/esger/src/application/build/source/staticlibB/CMakeFiles/staticlibB.dir/source/B.cpp
becomes ./build/source/staticlibB/CMakeFiles/staticlibB.dir/source/B.cpp)

with
/home/esger/src/application/build/source/staticlibB/CMakeFiles/staticlibB.dir/source
as cwd for the compilation of B.cpp.

This produces .o and .gcno files containing relative paths that can be cached
by ccache. When running ./build/bin/test_main from /home/esger/src/application
gcda files are created next to the gcno files. When compiling a second time
from a new directory /home/esger/src/application-2, the .o and .gcno are taken
from the cache and when running ./build/bin/test_main from
/home/esger/src/application-2 again gcda are created next to the .gcno files
below application-2.

Gcovr produces an identical coverage report for both dirs. So far so good ;)


When switching to gcc 9.3 however this does not work anymore and the coverage
reports are empty (even on builds were the cache was empty and no cached
.o/gcno's were used). I found
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97193 about incorrect mangling of
gcno files so tried gcc 9.4 as well which still does not work.

It seems the behavior of -fprofile-dir has changed according to the
documentation:

"-fprofile-dir=path

Set the directory to search for the profile data files in to path. This option
affects only the profile data generated by -fprofile-generate, -ftest-coverage,
-fprofile-arcs and used by -fprofile-use and -fbranch-probabilities and its
related options. Both absolute and relative paths can be used. By default, GCC
uses the current directory as path, thus the profile data file appears in the
same directory as the object file. In order to prevent the file name clashing,
if the object file name is not an absolute path, we mangle the absolute path of
the sourcename.gcda file and use it as the file name of a .gcda file." 


Note that the latter part where the absolute path of the source is mangled and
used as the file name is not helpful in our use case. We absolutely want a
relative path when specifying a relative path to -fprofile-dir and have no
absolute paths anywhere. I am certain in some cases there can be file name
clashing, but as far as I am aware that is not a problem at all in our use
case. (so it would be nice if this behavior at least can be turned off)

Added to that in 9.4 there seem to be more problems with -fprofile-dir? Given a
trivial main.cpp:

cwd = /home/esger/src/application/coverage_test/dir1/source

$ g++ --coverage main.cpp
$ ls
a.out  main.cpp  main.gcno
$ ./a.out
$ ls
a.out  main.cpp  main.gcda  main.gcno
$ gcovr --html -o coverage.html .
[coverage report OK]

$ g++ --coverage -fprofile-dir=. main.cpp
$ ls
a.out  main.cpp  main.gcno
$ ./a.out
$ ls
'#home#esger#src#application#coverage_test#dir1#source#main.gcda'   a.out  
main.cpp   main.gcno
$ gcovr --html -o coverage.html .
[coverage report broken, shows main.cpp but with 0% coverage]

$ g++ --coverage
-fprofile-dir=/home/esger/src/application/coverage_test/dir1/source main.cpp
$ ls
a.out  main.cpp  main.gcno
$ ./a.out
$ ls
'#home#esger#src#application#coverage_test#dir1#source#main.gcda'   a.out  
main.cpp   main.gcno
$ gcovr --html -o coverage.html .
[coverage report broken, shows main.cpp but with 0% coverage]

Either one of the last two should be the default value (when not specifying
-fprofile-dir) I'd imagine, but the behavior is different from the first case?


$ g++ --coverage -fprofile-dir=./somedir/ main.cpp
$ ls
a.out  main.cpp  main.gcno
$ ./a.out
$ ls
a.out  main.cpp  main.gcno  somedir
$ ls somedir
'#home#esger#src#application#coverage_test#dir1#source#main.gcda'
$ gcovr --html -o coverage.html .
[coverage report broken, shows main.cpp but with 0% coverage]

It seems that whenever -fprofile-dir is used the gcda name is always mangled
and gcovr (version 5.1) is not able to produce a correct report.


If I repeat the same four cases above with gcc 7.5.0 the first three all
produce gcda files named main.gcda and a correct coverage report. The last case
also produces a 0% coverage report (even though -v output seems to indicate it
is finding the source and gcno files, not sure what is happening there). Gcc
10.3 shows the same behavior as 9.4 for all four cases.


Back to our original use case, if I compare gcno files between gcc 7.5 and 9.4
it seems the 9.4 ones contain the cwd from compilation, which would bar them
from being cached (or when cached contain not-entirely-correct information).

gcc 7.5 (anonymised file from actual project):

$ gcov-dump somefile.cpp.gcno
somefile.cpp.gcno:note:magic `gcno':version `A75*'
somefile.cpp.gcno:stamp 3077156474
somefile.cpp.gcno: 01000000:  41:FUNCTION ident=57878848,
lineno_checksum=0xc0902347, cfg_checksum=0xc0bbb23e,
`_GLOBAL__sub_I__ZN6some9function12hereIN6with8somedEttE5nameE'
../../../../../source/common/source/somefile.cpp:22
somefile.cpp.gcno:  01410000:   4:BLOCKS 4 blocks
somefile.cpp.gcno:  01430000:   3:ARCS 1 arcs
somefile.cpp.gcno:  01430000:   5:ARCS 2 arcs
somefile.cpp.gcno:  01430000:   3:ARCS 1 arcs
somefile.cpp.gcno:  01450000:  21:LINES
...

Note the relative path to the source file.

gcc 9.4:

$ gcov-dump main.gcno
main.gcno:note:magic `gcno':version `A94*'
main.gcno:stamp 492126994
main.gcno:cwd: /home/esger/src/application/coverage_test/dir1/source
main.gcno:  01000000:  20:FUNCTION ident=1826217343,
lineno_checksum=0x36a9d8e5, cfg_checksum=0xc0bbb23e, `_GLOBAL__sub_I__Z5funcAi'
main.cpp:21:1-21:1, artificial
main.gcno:    01410000:   1:BLOCKS 4 blocks
main.gcno:    01430000:   3:ARCS 1 arcs
main.gcno:    01430000:   5:ARCS 2 arcs
main.gcno:    01430000:   3:ARCS 1 arcs
main.gcno:    01450000:   9:LINES
main.gcno:    01450000:   9:LINES
...

Contains undesired absolute path.


I don't think our use case (non-incremental builds from different directories
on a CI server with coverage for unittests) is at all esoteric or unusual but
as it stands with gcc 9.4 (or gcc 10.3) we either have to disable coverage or
disable ccache (and not use -fprofile-dir) and suffer much much longer build
times.

^ permalink raw reply	[flat|nested] 12+ messages in thread
end of thread, other threads:[~2024-05-07  7:40 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-12 11:32 [Bug gcov-profile/105238] New: Regression: using -fprofile-dir: gcno files not ccache cachable anymore / gcovr report broken esgergn at hotmail dot com
2022-04-12 12:25 ` [Bug gcov-profile/105238] " marxin at gcc dot gnu.org
2022-04-12 13:28 ` esgergn at hotmail dot com
2022-04-13  8:12 ` marxin at gcc dot gnu.org
2022-04-14  9:11 ` esgergn at hotmail dot com
2022-04-15 12:19 ` esgergn at hotmail dot com
2022-04-15 13:55 ` esgergn at hotmail dot com
2022-04-15 22:41 ` esgergn at hotmail dot com
2022-04-19  9:55 ` marxin at gcc dot gnu.org
2023-04-03  8:57 ` marxin at gcc dot gnu.org
2023-04-03  9:03 ` marxin at gcc dot gnu.org
2024-05-07  7:40 ` rguenth at gcc dot gnu.org

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).