From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by sourceware.org (Postfix) with ESMTPS id E556E3857829 for ; Tue, 23 Mar 2021 23:09:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org E556E3857829 Received: by mail-io1-xd32.google.com with SMTP id n198so19557187iod.0 for ; Tue, 23 Mar 2021 16:09:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xj+LolNBKeJ/Q0oUNzlyOqxtfBMiUU4knjK67C8jcP4=; b=dOk6UiJ21PgQtxCW3zOgCz1lbA4v1c9TFoeW/Fe4oFVCOGAUaCZuCIATwEpXcI166n CkexxjaTXeEciUbkLRKk2zooE8ndXm8uojfi0wRCNiTXkdV+O5Ax2rSgVKJtWMeFhAF6 uH+xmlodyRcR7s8HWzL2kkl0sKW1Jgzao1DrbP7SpWxuUnHUTIXXYqSesQ4EzhvXmI1O CavdET+6Hm7lzzmBBX9nKZNaQtiKhAHgq6j75TrkzBMMqtlgcWuiTIaaMRC/heck9Lf5 ZZ87+FOV3BS51YWA3azoYVdrAD5qNJrA7p1bu0jSPRzitveDNYSdyYuLYk/HokogsygZ 7Kzg== X-Gm-Message-State: AOAM531Ne7a7L6R3ehOi6/joymZg3ep0+N8Z2p599bC0LswchOVdWG7s 9muveLcvfbIxMsGqXnVSR/yQeY06rsVoJVlYAEI= X-Google-Smtp-Source: ABdhPJw42E+n6ySFTRHcXgs2CqlNIrQrKvEOP62VwZ61zGxxDZ7TffM8ncdG4d810wvoJje72Nto23TePxP81c+32ss= X-Received: by 2002:a5d:93ce:: with SMTP id j14mr360719ioo.172.1616540993461; Tue, 23 Mar 2021 16:09:53 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Peng Yu Date: Tue, 23 Mar 2021 18:09:42 -0500 Message-ID: Subject: Re: Does glibc has complete test coverage? To: noloader@gmail.com Cc: Mike Frysinger , libc-help Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-help@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-help mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2021 23:09:55 -0000 On Tue, Mar 23, 2021 at 4:02 PM Jeffrey Walton wrote: > > On Tue, Mar 23, 2021 at 4:43 PM Mike Frysinger via Libc-help > wrote: > > > > On 23 Mar 2021 11:39, Peng Yu via Libc-help wrote: > > > https://www.kernel.org/doc/man-pages/missing_pages.html > > > > > > "... quite a few kernel and glibc bugs have been uncovered while > > > writing test programs during the preparation of man pages. " > > > > > > I see the above text. It doesn't make too much sense, as it indicates > > > that glibc does not have complete test coverage. > > > > > > Why not taking an approach of always accompanying each line of source > > > code with appopriate test cases? If this approach is taken, then most > > > bugs should have been eliminated beforehand? > > > > ignoring the legacy aspect (code that's in the tree now but lacks tests), > > you have diminishing returns when it comes to writing unittests, and, as > > can be seen in a recent discussion, glibc is pretty tightly coupled to > > the runtime environment (i.e. the host kernel). so getting an env that > > matches all the different code paths is challenging. > > > > plus it comes down a bit to this being an open source project for many > > of us, not a job, and you have to be respectful of balancing quality > > and developer time with any requests you make on other volunteers. > > > > along those lines, this is an open source project where "patches are > > welcome", so if you wanted to spend your time improving the frameworks > > and coverage of our tests, we'd welcome you. > > Interns are usually a good choice for writing test cases. It gets them > familiar with the code, frees up a senior developer's time, and helps > avoid the developer's bias. > > Test cases are monkey work that should be delegated. When delegation > does not occur it usually points back to shortcomings in project > management. > > > also try googling for "100% test coverage" and reading the variety of > > opinions the wider world has on the topic. Hi Mike, https://blog.ndepend.com/aim-100-percent-test-coverage/ I don't think the so-called "The Diminishing Returns Argument" is valid. When you have too many branches to test which rarely occurs in practice, this just indicates the code is badly structured. To have complete coverage, if the testing code is much more complex than the real code, this means that the real code should be restructured to make testing code simpler. > Sorry, I could not resist.... But you know the funny thing is, when > you perform a post-mortem to determine why the bug made it into > production, it usually points to (1) a developer mistake and (2) lack > of test case. > > If you break (1) or (2) you break the chain for the bug. So you either > have to hire developers who don't make mistakes or provide complete > test cases. Hi Jeff, (1) is guaranteed to happen in practice as all humans will make mistakes. Therefore, overcoming (2) is the only way to eliminate the bug. -- Regards, Peng