From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) by sourceware.org (Postfix) with ESMTPS id 0FD57387085F for ; Wed, 10 Mar 2021 14:56:09 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 0FD57387085F Received: by mail-qt1-x82a.google.com with SMTP id j3so13141756qtj.12 for ; Wed, 10 Mar 2021 06:56:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:cc:references:from:subject:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=oFVE0S/H6AMKxnvDI/WbGdcO3BLJgU4r17A6jkYZ4KE=; b=KhUF5Pe4029liXi29hzxHObT5CL7e4FEszqZ0SAq47l4ssT6EZ4iPS7l2XB+iRCN0U Ad0F06+MANn7Rf2CICJ+bLv3Uwkase+fFBP+W/m4rAXlQbxWLzrZDFCpgZjlyuafmCZI SJeMmh6GYrxXBH+g1t33k1x9vnx2HopI/yMoJ855CeQvOxdZHn7sEeg0w4nE22yP8CUU k5bSwAwaEFHhY0M1FbwiY2ot5hdzNqdMwr6XpV0LS5bwtCC4Un6VS+O2Q+y5wuuxjdXB 7fhysSGBgFyzLJvVZgBx6lvVQd1uf2m8Ia/7c9J1FZm2o9szpsaykjc67mcb28tkV+E0 PSbA== X-Gm-Message-State: AOAM533A7MkrPvTBFr0HzG9jePThXwAF28cfj9fShY2byy4vHfvPj+kC WI96oaX19VRjl4H8uw71sugOzrVB09S08Z17 X-Google-Smtp-Source: ABdhPJx2ilblKGyfMpvXhx4Z8ErEeZVbL0oWGNwUZBVWfaCQsfNDU06Hpuc6IPSmC7ESxOXQ/ZE4Zw== X-Received: by 2002:ac8:4783:: with SMTP id k3mr3054458qtq.231.1615388168337; Wed, 10 Mar 2021 06:56:08 -0800 (PST) Received: from [192.168.1.4] ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id v7sm1140098qkv.86.2021.03.10.06.56.07 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 10 Mar 2021 06:56:08 -0800 (PST) To: Arjun Shankar Cc: libc-alpha@sourceware.org References: <09df7704-4a63-75d4-f7f8-aca3fed1bd83@linaro.org> <1622066761.33421738.1615385607852.JavaMail.zimbra@redhat.com> From: Adhemerval Zanella Subject: Re: [PATCH v2] Mount /tmp as tmpfs in test-container and run utime tests in it Message-ID: <92c90492-b5ec-f1d0-4515-0c798de7eba2@linaro.org> Date: Wed, 10 Mar 2021 11:56:05 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <1622066761.33421738.1615385607852.JavaMail.zimbra@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2021 14:56:10 -0000 On 10/03/2021 11:13, Arjun Shankar wrote: > Hi Adhemerval, > >> Sorry, I still think we should not gloss over buggy filesystems. >> I really want this tests to be tested on multiple different filesystem >> and check if the interface is really working as intended. There is no >> point in providing y2038 interfaces that might be broken when used >> on different FS that is actually used in the wild. > > Testing various filesystems is certainly of value when done > systematically. The issues I see right now are: > > In its present state, this test can fail even though there is no known > glibc bug in the functionality it tests. This makes the test flaky > when it comes to verifying glibc functionality itself. > > In the future, this test could also fail because of a glibc regression. > At that point someone used to running the test and seeing it fail > often in the past could disregard the failure, or the filesystem bug > could mask a glibc regression, thus causing us to miss catching the > glibc regression. So this test is also flaky when it comes to catching > a potential regression. > > A test that doesn't always pass when the glibc interface under test is > actually working well and doesn't always clearly identify a regression > because it is masked by a well known bug in some other component is a > test that has potential for improvement. > > The way I understand it, this is a unit test of a glibc interface. The > goal of unit testing is to isolate each part of the program and show > that the individual parts are correct. In that spirit, I feel that > running this test in isolation from the filesystem has value and is an > improvement over its present state. We had in the past multiple occasion where buggy kernel interface triggered invalid results on valid tests: * On systems with Linux kernel 3.11 through 3.17, missing a backport of commit 69a91c237ab0ebe4e9fdeaf6d0090c85275594ec (present in 3.18, backports may be in some older stable series), io/tst-open-tmpfile fails. [1] * posix/tst-posix_fadvise64 failure results from a kernel bug with 32-bit powerpc kernels [2] [3]. * Various issues reported on WSL with 2.28 [2]. * Linux between 4.13 and 4.15 return EOVERFLOW for LFS OFD locks usage in compat mode (non-LFS ABI running on a LFS default kernel, such as i386 on a x86_64 kernel or s390-32 on a s390-64 kernel) [4] [5]. What we have done in such cases, and I think it was the correct way, is to *proper* document on each scenario the failure happen, the expected results, and if applicable a way to avoid it. If we start to add tests exception for each failure, it will complicate the code base and restrict the test itself (for instance, moving the y2038 to a container would actually check them on system/architecture/kernel without container support). But the whole idea here is *not* check only the glibc interface, otherwise we would make the test act a syscall wrapper where the syscall is not relly exercised (which is indeed possible with some ptrace tricks). > > The XFS bug is well known. I feel that the value to this community > from consistently verifying glibc functionality is higher than the > value of reminding ourselves of a well known bug in another component. > > I hope this makes a convincing argument, but irrespective of whether > you approve of this patch or the direction it takes: thank you for the > time you have taken to respond to these patches. Sorry, but still think we should keep the tests as-is and keep failing them on the buggy interface. As I said initially, they are doing *exactly* what they are suppose to do: expose a bug in the functionality that glibc is meant to provide to the users. This is very similar to a similar Red Hat issue [6], but back then it was marked as CLOSED/NOTABUG since "we don't have any work to do in glibc for this bug". I think we should thread this issue similar to this one (although back then was a missing backports, instead of upstream kernel bug; but I think the principle applies). > > Cheers, > Arjun > [1] https://sourceware.org/glibc/wiki/Release/2.32 [2] https://sourceware.org/glibc/wiki/Release/2.29 [3] https://lkml.org/lkml/2017/1/4/680 [4] https://sourceware.org/glibc/wiki/Release/2.28 [5] https://sourceware.org/ml/libc-alpha/2018-07/msg00243.html [6] https://bugzilla.redhat.com/show_bug.cgi?id=1647483