[Bug gcov-profile/58602] New: .gcno files not truncated at gcov_close

[Bug gcov-profile/58602] New: .gcno files not truncated at gcov_close

[laurent.alfonsi at st dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

            Bug ID: 58602
           Summary: .gcno files not truncated at gcov_close
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: gcov-profile
          Assignee: unassigned at gcc dot gnu.org
          Reporter: laurent.alfonsi at st dot com

Created attachment 30950
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=30950&action=edit
script to reproduce

Gcc doesnot clean the .gcno file before writing in. Thus, it might contain
garbage at the end.

At one point, it was truncated, but this has been removed.
  http://gcc.gnu.org/ml/gcc-patches/2004-07/msg02541.html

Test case : 
 $ rm test.gcno 
 $ gcc -ftest-coverage -fprofile-arcs test.c -O3
 $ ls -la test.gcno 
 -rw-r--r-- 1 alfonsi gnbcomp 688 Oct  3 11:57 test.gcno

 $ gcc -ftest-coverage -fprofile-arcs test.c -O0
 $ ls -la test.gcno 
 -rw-r--r-- 1 alfonsi gnbcomp 908 Oct  3 11:57 test.gcno

 $ gcc -ftest-coverage -fprofile-arcs test.c -O3
 $ ls -la test.gcno 
 -rw-r--r-- 1 alfonsi gnbcomp 908 Oct  3 11:58 test.gcno

As you may see, the last 220 bytes are garbage from the preceding O0
compilation.

I enclosed a small script that expose the problem.

This has been trackeddown from a kernel issue on lcov:
  http://sourceforge.net/p/ltp/mailman/message/31141937/

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[gbanks at sgi dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

Greg Banks <gbanks at sgi dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |gbanks at sgi dot com

--- Comment #1 from Greg Banks <gbanks at sgi dot com> ---
This bug can affect users of ggcov (http://ggcov.sf.net), which reads
the .gcno files directly rather than running any gcc code.  Usually the
problem is harmless, as ggcov complains and stops parsing because it
tries to read random leftover bytes as a gcov tag.  However it's possible
that other subtle problems can creep in, like lines being double-counted.

The bug appears to have been introduced in this commit

http://gcc.gnu.org/git/?p=gcc.git;a=commit;h=8ab576411b46496e6baf77ba1aed1f38b6c818cd

which nine years ago (!) changed the actual code path taken from
an fopen("wb") to an open(O_RDWR|O_CREAT) plus an fdopen().  One
of the differences between those is that the old code had the side
effect of truncating an existing file, and the new code doesn't.

It looks like every version of gcc since 3.4 is affected.

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[gbanks at sgi dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

--- Comment #2 from Greg Banks <gbanks at sgi dot com> ---
Created attachment 31085
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31085&action=edit
source for test case

Here's a test case which doesn't rely on optimisation behaviour
to demonstrate the bug.

me@machine 2603> gcc --coverage -c foo.c

me@machine 2604> ls -l foo.gcno
-rw-rw-r-- 1 gnb gnb 420 Oct 23 22:18 foo.gcno
                     ^^^

me@machine 2605> gcc --coverage -DMAKE_ME_SMALLER -c foo.c

me@machine 2606> ls -l foo.gcno
-rw-rw-r-- 1 gnb gnb 420 Oct 23 22:19 foo.gcno
                     ^^^
                     # new file is incorrectly not smaller

me@machine 2607> rm foo.gcno
me@machine 2608> gcc --coverage -DMAKE_ME_SMALLER -c foo.c

me@machine 2609> ls -l foo.gcno
-rw-rw-r-- 1 gnb gnb 216 Oct 23 22:19 foo.gcno
                     ^^^
                     # this is the real size of the new file

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[laurent.alfonsi at st dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

--- Comment #3 from Laurent Aflonsi <laurent.alfonsi at st dot com> ---
Created attachment 31086
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31086&action=edit
source patch for trunk

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[laurent.alfonsi at st dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

--- Comment #4 from Laurent Aflonsi <laurent.alfonsi at st dot com> ---
When the .gcno graph file is opened for generating the coverage graph
information, the mode used is w+ as this code is shared with updating tools
such as libgcov. Thus, when GCC outputs .gcno files, it may leave garbage at
the end of the file if the file already exists when opening it.

This has been trackeddown from a kernel issue on lcov:
     http://sourceforge.net/p/ltp/mailman/message/31141937/

This patch fixes the function gcov_open() such that the .gcno file is opened
with truncation when gcc asks for creating a new file (mode<0). 

which I also proposed on the mailing list some time ago, but i got no
feedbacks.
     http://gcc.gnu.org/ml/gcc-patches/2013-10/msg00254.html

Here is a Changelog:
   2013-10-03  Laurent Alfonsi  <laurent.alfonsi@st.com>

    PR gcov-profile/58602
        * gcc/gcov-io.c (gcov_open): Open with truncation when mode<0


Regards,
Laurent

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[laurent.alfonsi at st dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

--- Comment #5 from Laurent Aflonsi <laurent.alfonsi at st dot com> ---
Ping ?
Is this ok for trunk ?

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[mpolacek at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

Marek Polacek <mpolacek at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mpolacek at gcc dot gnu.org

--- Comment #6 from Marek Polacek <mpolacek at gcc dot gnu.org> ---
Please ping the patch on the gcc-patches mailing list.

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[law at redhat dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

Jeffrey A. Law <law at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |law at redhat dot com

--- Comment #7 from Jeffrey A. Law <law at redhat dot com> ---
It seems like all we've done in the last 10 years is rotate through a set of
bugs in this code.

The whole point of the Richard's original removal of ftruncate was to make
things easier for folks on windows & using simulators (newlib).   That patch
introduced a race  which led to Jakub's change form Feb 2004 to close the race
condition, but which can leave the file untruncated.

So the question is does this introduce a race similar to what Jakub was trying
to fix back in Feb 2004?  What testing has been done to see if there's a race. 
Testing similar to Jakub's but scaled up for modern hardware would raise the
confidence level of this patch significantly.

http://marc.info/?l=gcc-patches&m=107747608611324&w=2

[ Oddly enough, the gcc.gnu.org archives seem to be missing for this thread and
much of Feb2004. ]

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[laurent.alfonsi at st dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

--- Comment #8 from Laurent Aflonsi <laurent.alfonsi at st dot com> ---
  Jeffrey,

As you asked for, I have performed much more testing on that fix.
I am far from having done as many tests as Jakub in 2004. In particular, I have
not been able to test on ppc/iaxx, because i don't have any available.

I have performed a profiledbootstrap in a loop all over the weekend on my
x86_64 with parallelized builds (-j 4, -j 5, ... -j 12). which represents 80
builds using various parallel schedules. All of them have finished
successfully. I have not observed any instabilities.
It has been also regtested gcc/g+/libstdc++, 7 times in a row without any
failure.

Thanks
Laurent

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[law at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

--- Comment #9 from Jeffrey A. Law <law at gcc dot gnu.org> ---
Author: law
Date: Mon Feb  3 21:55:36 2014
New Revision: 207441

URL: http://gcc.gnu.org/viewcvs?rev=207441&root=gcc&view=rev
Log:
    PR gcov-profile/58602
        * gcc/gcov-io.c (gcov_open): Open with truncation when mode<0

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/gcov-io.c

[Bug gcov-profile/58602] .gcno files not truncated at gcov_close

[law at redhat dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58602

Jeffrey A. Law <law at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |FIXED

--- Comment #10 from Jeffrey A. Law <law at redhat dot com> ---
Fixed on the trunk.