Re: Does glibc has complete test coverage?

[Jeffrey Walton <noloader@gmail.com>]

On Tue, Mar 23, 2021 at 4:43 PM Mike Frysinger via Libc-help
<libc-help@sourceware.org> wrote:
>
> On 23 Mar 2021 11:39, Peng Yu via Libc-help wrote:
> > https://www.kernel.org/doc/man-pages/missing_pages.html
> >
> > "... quite a few kernel and glibc bugs have been uncovered while
> > writing test programs during the preparation of man pages. "
> >
> > I see the above text. It doesn't make too much sense, as it indicates
> > that glibc does not have complete test coverage.
> >
> > Why not taking an approach of always accompanying each line of source
> > code with appopriate test cases? If this approach is taken, then most
> > bugs should have been eliminated beforehand?
>
> ignoring the legacy aspect (code that's in the tree now but lacks tests),
> you have diminishing returns when it comes to writing unittests, and, as
> can be seen in a recent discussion, glibc is pretty tightly coupled to
> the runtime environment (i.e. the host kernel).  so getting an env that
> matches all the different code paths is challenging.
>
> plus it comes down a bit to this being an open source project for many
> of us, not a job, and you have to be respectful of balancing quality
> and developer time with any requests you make on other volunteers.
>
> along those lines, this is an open source project where "patches are
> welcome", so if you wanted to spend your time improving the frameworks
> and coverage of our tests, we'd welcome you.

Interns are usually a good choice for writing test cases. It gets them
familiar with the code, frees up a senior developer's time, and helps
avoid the developer's bias.

Test cases are monkey work that should be delegated. When delegation
does not occur it usually points back to shortcomings in project
management.

> also try googling for "100% test coverage" and reading the variety of
> opinions the wider world has on the topic.

Sorry, I could not resist.... But you know the funny thing is, when
you perform a post-mortem to determine why the bug made it into
production, it usually points to (1) a developer mistake and (2) lack
of test case.

If you break (1) or (2) you break the chain for the bug. So you either
have to hire developers who don't make mistakes or provide complete
test cases.

Jeff