LCOV of current GCC

LCOV of current GCC

[Martin Liška]

Hello.

I've been working on some patches for GCOV and lcov was of my test scenarios.
I'm sending link to static HTML pages made by the tool which are recorded
for GCC (w/o bootstrap) build + running test-suite on x86_64-linux-gnu.
I'm planning to set up a periodic build of that that will eventually rsync
content to a public website:

I guess it can be interesting for instance to see which folding branches are
not used, or which files (functionality) is basically not much tested via
the testsuite.

https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8

P.S. I've noticed David fixed doxygen of the project, I can rsync also that
to public website.

Martin

Re: LCOV of current GCC

[Richard Biener]

On Fri, Apr 28, 2017 at 11:07 AM, Martin Liška <mliska@suse.cz> wrote:
> Hello.
>
> I've been working on some patches for GCOV and lcov was of my test scenarios.
> I'm sending link to static HTML pages made by the tool which are recorded
> for GCC (w/o bootstrap) build + running test-suite on x86_64-linux-gnu.
> I'm planning to set up a periodic build of that that will eventually rsync
> content to a public website:
>
> I guess it can be interesting for instance to see which folding branches are
> not used, or which files (functionality) is basically not much tested via
> the testsuite.
>
> https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8
>
> P.S. I've noticed David fixed doxygen of the project, I can rsync also that
> to public website.

Nice!  Results look better than anticipated ;)

Richard.

> Martin

Re: LCOV of current GCC

[Jeff Law]

On 04/28/2017 03:38 AM, Richard Biener wrote:
> On Fri, Apr 28, 2017 at 11:07 AM, Martin Liška <mliska@suse.cz> wrote:
>> Hello.
>>
>> I've been working on some patches for GCOV and lcov was of my test scenarios.
>> I'm sending link to static HTML pages made by the tool which are recorded
>> for GCC (w/o bootstrap) build + running test-suite on x86_64-linux-gnu.
>> I'm planning to set up a periodic build of that that will eventually rsync
>> content to a public website:
>>
>> I guess it can be interesting for instance to see which folding branches are
>> not used, or which files (functionality) is basically not much tested via
>> the testsuite.
>>
>> https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8
>>
>> P.S. I've noticed David fixed doxygen of the project, I can rsync also that
>> to public website.
> 
> Nice!  Results look better than anticipated ;)
I did some similar work a few years back.   Martin's results are 
comparable to mine.   Interestingly enough the 70-80% coverage we see 
for GCC is a "sweet spot" in that several of the ancillary tools I 
tested were in the same overall range.

jeff

Re: LCOV of current GCC

[David Malcolm]

On Fri, 2017-04-28 at 11:38 +0200, Richard Biener wrote:
> On Fri, Apr 28, 2017 at 11:07 AM, Martin Liška <mliska@suse.cz>
> wrote:
> > Hello.
> > 
> > I've been working on some patches for GCOV and lcov was of my test
> > scenarios.
> > I'm sending link to static HTML pages made by the tool which are
> > recorded
> > for GCC (w/o bootstrap) build + running test-suite on x86_64-linux
> > -gnu.
> > I'm planning to set up a periodic build of that that will
> > eventually rsync
> > content to a public website:
> > 
> > I guess it can be interesting for instance to see which folding
> > branches are
> > not used, or which files (functionality) is basically not much
> > tested via
> > the testsuite.
> > 
> > https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8
> > 
> > P.S. I've noticed David fixed doxygen of the project, I can rsync
> > also that
> > to public website.
> 
> Nice!  Results look better than anticipated ;)
> 
> Richard.
> 
> > Martin

Excellent; thanks.

For your periodic builds, please can you add "jit" to the enabled
languages (it will also need --enable-host-shared).

Would be nice to add libiberty and libcpp to this, but maybe that needs
extra work?

Dave

Re: LCOV of current GCC

[Joseph Myers]

On Fri, 28 Apr 2017, Jeff Law wrote:

> I did some similar work a few years back.   Martin's results are comparable to
> mine.   Interestingly enough the 70-80% coverage we see for GCC is a "sweet
> spot" in that several of the ancillary tools I tested were in the same overall
> range.

My observation from looking at some of the C front-end results is that 
many of the coverage omissions are from features only used / supported on 
certain architectures, e.g. address spaces and fixed point (though 
actually there are address spaces on x86, but maybe not tests covering 
much of the generic address space support).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: LCOV of current GCC

[Martin Liška]

On 04/28/2017 06:53 PM, David Malcolm wrote:
> On Fri, 2017-04-28 at 11:38 +0200, Richard Biener wrote:
>> On Fri, Apr 28, 2017 at 11:07 AM, Martin Liška <mliska@suse.cz>
>> wrote:
>>> Hello.
>>>
>>> I've been working on some patches for GCOV and lcov was of my test
>>> scenarios.
>>> I'm sending link to static HTML pages made by the tool which are
>>> recorded
>>> for GCC (w/o bootstrap) build + running test-suite on x86_64-linux
>>> -gnu.
>>> I'm planning to set up a periodic build of that that will
>>> eventually rsync
>>> content to a public website:
>>>
>>> I guess it can be interesting for instance to see which folding
>>> branches are
>>> not used, or which files (functionality) is basically not much
>>> tested via
>>> the testsuite.
>>>
>>> https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8
>>>
>>> P.S. I've noticed David fixed doxygen of the project, I can rsync
>>> also that
>>> to public website.
>>
>> Nice!  Results look better than anticipated ;)
>>
>> Richard.
>>
>>> Martin
> 
> Excellent; thanks.

Hello.

I've just done that, periodically built LCOV can be found here:

http://gcc.opensuse.org/gcc-lcov/

> 
> For your periodic builds, please can you add "jit" to the enabled
> languages (it will also need --enable-host-shared).

Done that and will be seen in next build. I do it every weekend.

> 
> Would be nice to add libiberty and libcpp to this, but maybe that needs
> extra work?

Yep, it's currently done for gcc subfolder. Can be done in the future.

Martin

> 
> Dave
> 

Re: LCOV of current GCC

[Mikhail Maltsev]

Hi all!
I noticed that you started to publish GCC test coverage data. About a
year ago I also experimented with GCC's coverage and would like to
share some possible improvement ideas for your scripts.

I postprocess the coverage data with this script:
https://gist.github.com/miyuki-chan/943efe7d8e0eb72fdd74997c8a10d6c5

It has the following features:
1. removes gengtype routines from coverage data
2. removes code from system headers (e.g., the host libstdc++)
3. it outputs a summary (which can be used for building graphs, like
you do for benchmarks)

I hope, you might find some of these ideas helpful.


On Tue, May 30, 2017 at 9:21 AM, Martin Liška <mliska@suse.cz> wrote:
> On 04/28/2017 06:53 PM, David Malcolm wrote:
>> On Fri, 2017-04-28 at 11:38 +0200, Richard Biener wrote:
>>> On Fri, Apr 28, 2017 at 11:07 AM, Martin Liška <mliska@suse.cz>
>>> wrote:
>>>> Hello.
>>>>
>>>> I've been working on some patches for GCOV and lcov was of my test
>>>> scenarios.
>>>> I'm sending link to static HTML pages made by the tool which are
>>>> recorded
>>>> for GCC (w/o bootstrap) build + running test-suite on x86_64-linux
>>>> -gnu.
>>>> I'm planning to set up a periodic build of that that will
>>>> eventually rsync
>>>> content to a public website:
>>>>
>>>> I guess it can be interesting for instance to see which folding
>>>> branches are
>>>> not used, or which files (functionality) is basically not much
>>>> tested via
>>>> the testsuite.
>>>>
>>>> https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8
>>>>
>>>> P.S. I've noticed David fixed doxygen of the project, I can rsync
>>>> also that
>>>> to public website.
>>>
>>> Nice!  Results look better than anticipated ;)
>>>
>>> Richard.
>>>
>>>> Martin
>>
>> Excellent; thanks.
>
> Hello.
>
> I've just done that, periodically built LCOV can be found here:
>
> http://gcc.opensuse.org/gcc-lcov/
>
>>
>> For your periodic builds, please can you add "jit" to the enabled
>> languages (it will also need --enable-host-shared).
>
> Done that and will be seen in next build. I do it every weekend.
>
>>
>> Would be nice to add libiberty and libcpp to this, but maybe that needs
>> extra work?
>
> Yep, it's currently done for gcc subfolder. Can be done in the future.
>
> Martin
>
>>
>> Dave
>>
>



-- 
Regards,
   Mikhail Maltsev

Re: LCOV of current GCC

[Martin Liška]

On 07/02/2017 07:19 PM, Mikhail Maltsev wrote:
> Hi all!
> I noticed that you started to publish GCC test coverage data. About a
> year ago I also experimented with GCC's coverage and would like to
> share some possible improvement ideas for your scripts.
> 
> I postprocess the coverage data with this script:
> https://gist.github.com/miyuki-chan/943efe7d8e0eb72fdd74997c8a10d6c5

Hello.

Thank you for the notes. As my build is driven by buildbot I decided to incorporate
your changes:

> 
> It has the following features:
> 1. removes gengtype routines from coverage data

Done.

> 2. removes code from system headers (e.g., the host libstdc++)

Likewise.

> 3. it outputs a summary (which can be used for building graphs, like
> you do for benchmarks)

Yep, written to my TODO list.

Please be aware that we have some connection issues with gcc.opensuse.org, but
updated LCOV will appear there soon.

Thanks,
Martin

> 
> I hope, you might find some of these ideas helpful.
> 
> 
> On Tue, May 30, 2017 at 9:21 AM, Martin Liška <mliska@suse.cz> wrote:
>> On 04/28/2017 06:53 PM, David Malcolm wrote:
>>> On Fri, 2017-04-28 at 11:38 +0200, Richard Biener wrote:
>>>> On Fri, Apr 28, 2017 at 11:07 AM, Martin Liška <mliska@suse.cz>
>>>> wrote:
>>>>> Hello.
>>>>>
>>>>> I've been working on some patches for GCOV and lcov was of my test
>>>>> scenarios.
>>>>> I'm sending link to static HTML pages made by the tool which are
>>>>> recorded
>>>>> for GCC (w/o bootstrap) build + running test-suite on x86_64-linux
>>>>> -gnu.
>>>>> I'm planning to set up a periodic build of that that will
>>>>> eventually rsync
>>>>> content to a public website:
>>>>>
>>>>> I guess it can be interesting for instance to see which folding
>>>>> branches are
>>>>> not used, or which files (functionality) is basically not much
>>>>> tested via
>>>>> the testsuite.
>>>>>
>>>>> https://drive.google.com/open?id=0B0pisUJ80pO1X0s3eEpuQ25GTG8
>>>>>
>>>>> P.S. I've noticed David fixed doxygen of the project, I can rsync
>>>>> also that
>>>>> to public website.
>>>>
>>>> Nice!  Results look better than anticipated ;)
>>>>
>>>> Richard.
>>>>
>>>>> Martin
>>>
>>> Excellent; thanks.
>>
>> Hello.
>>
>> I've just done that, periodically built LCOV can be found here:
>>
>> http://gcc.opensuse.org/gcc-lcov/
>>
>>>
>>> For your periodic builds, please can you add "jit" to the enabled
>>> languages (it will also need --enable-host-shared).
>>
>> Done that and will be seen in next build. I do it every weekend.
>>
>>>
>>> Would be nice to add libiberty and libcpp to this, but maybe that needs
>>> extra work?
>>
>> Yep, it's currently done for gcc subfolder. Can be done in the future.
>>
>> Martin
>>
>>>
>>> Dave
>>>
>>
> 
> 
>