public inbox for glibc-bugs@sourceware.org help / color / mirror / Atom feed
From: "joseph at codesourcery dot com" <sourceware-bugzilla@sourceware.org> To: glibc-bugs@sources.redhat.com Subject: [Bug libc/14233] Many of tst-*.c lack any description of the assertions they test Date: Thu, 14 Jun 2012 18:45:00 -0000 [thread overview] Message-ID: <bug-14233-131-6LUZrfJw2M@http.sourceware.org/bugzilla/> (raw) In-Reply-To: <bug-14233-131@http.sourceware.org/bugzilla/> http://sourceware.org/bugzilla/show_bug.cgi?id=14233 --- Comment #1 from joseph at codesourcery dot com <joseph at codesourcery dot com> 2012-06-14 18:44:45 UTC --- On Thu, 14 Jun 2012, bugdal at aerifal dot cx wrote: > Browsing some of the tst-*.c files in glibc, I noticed that many lack any > comments about what requirement they're testing for. I'm not sure if this is a > "bug" strictly speaking, but it seems problematic in that: As with bug 13959, I think Bugzilla is not the right place for this. Anything involving "many" without a clear objective way to assess whether the source tree has been fixed for an issue is probably better suited to policy discussions on libc-alpha and tracking open projects on wiki pages. If you create your own project page then you can track for each test whether it meets your own standards and other people can independently track, if they wish, whether each test meets their standards. > 1. It's difficult to audit/debug them and determine if they're actually testing > for what they were intended to test. This is especially problematic for tests > that involve (either as the subject of the test or as a side-difficulty of > testing) race conditions, where it may not be at all obvious where the races > are. It's quite likely that the function of a test changes over time as the test itself changes. If you wish to propose a policy against changing existing tests to test new things - a policy that new tests should be added in preference to changing an existing test - then libc-alpha is the right place to do so, with the wiki being used to document any resulting consensus. > 2. If a change introduces a bug, it may be hard to determine the cause without > reverse engineering the test to see what it was testing. Patches explaining obscure aspects of a test are always welcome, but there's no need to repeat in English what the test already says in C or what is plain from the documented semantics of the functions, and it's inherently subjective what is an appropriate level of commenting for a test. If you want better explanations in new tests, you should do *both* of the following: raise the issue on libc-alpha of what appropriate levels of explanation are (with examples of specific well-explained and badly-explained tests), with a view to documenting consensus on the wiki; review testcases in submitted patches and ask for improvements if you think they are inadequately explained. Since it's subjective what's enough explanation, only you can review new tests for whether they meet your own standards of explanation, although to some extent a standard may be documented and illustrated by examples. > 3. It would be nice to know if tests are asserting behavior required by the > standards, behavior required as part of the documented or unofficial > specification of glibc, undocumented behavior that known applications expect, > and/or quality of implementation issues. Again, this is open-ended and in general a test is simply helping to ensure that unexpected changes to glibc's behavior are detected. -- Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.
next prev parent reply other threads:[~2012-06-14 18:45 UTC|newest] Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top 2012-06-14 0:23 [Bug libc/14233] New: " bugdal at aerifal dot cx 2012-06-14 18:45 ` joseph at codesourcery dot com [this message] 2012-06-24 14:53 ` [Bug libc/14233] " jsm28 at gcc dot gnu.org 2014-06-19 14:30 ` fweimer at redhat dot com
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=bug-14233-131-6LUZrfJw2M@http.sourceware.org/bugzilla/ \ --to=sourceware-bugzilla@sourceware.org \ --cc=glibc-bugs@sources.redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).