Re: [PATCH v2] Mount /tmp as tmpfs in test-container and run utime tests in it

[Carlos O'Donell <carlos@redhat.com>]

On 3/11/21 12:16 PM, Adhemerval Zanella wrote:
> 
> 
> On 11/03/2021 13:38, Carlos O'Donell wrote:
>> On 3/10/21 1:53 PM, Adhemerval Zanella via Libc-alpha wrote:
>>> On 10/03/2021 15:31, DJ Delorie wrote:
>>>> Adhemerval Zanella via Libc-alpha <libc-alpha@sourceware.org> writes:
>>>>> Sorry, I still think we should not gloss over buggy filesystems. 
>>>>
>>>> The right solution to this is to mount one of every filesystem and test
>>>> them all, not leave it up to the randomness of /tmp and have to try to
>>>> deduce if it's a known bug or a new bug.
>>>>
>>>> Oh wait, that's kernel testing.
>>>>
>>>
>>> This is similar to libidn2 requirement [1], where it has started to dump
>>> FAILs if the system did not provide an updated library version. Carlos
>>> back then [2] added that it should continue to dump a 'FAIL' to indicate 
>>> 'you have bugs in your system libidn'.
>>
>> We are smearing the definition of "testing" across two boundaries:
>>
>> * integrated system testing (test glibc with the host and host libraries)
>>
>> * unit testing (smallest logical piece to test depends on our definitions)
>>
>> If we want a glibc "integrated system test" that verifies the current glibc
>> against the host /tmp, then we can have that.
>>
>> I think that Arjun is arguing he would like to see the y2038 tests behave
>> more like unit tests and operate independently of the host.
>>
>> I think that DJ has noted we could have 2 tests, one which covers the
>> whole system testing (or integration test) and a unit test.
>>
>> In the case of libidn2, where glibc was calling out via dlopen to the
>> host libidn2, there was a good case to use the whole system testing
>> approach and FAIL the test if libidn2 was buggy. In order to make a unit
>> test out of that we'd need to bundle a known-good libidn2 or stub it out.
>>
>> In the case of this kernel bug we can relatively easily clone a unit test
>> that uses tmpfs, and also run a whole system test, and provide comments
>> explaining why we have two tests.
>>
>>> So maybe it would be better to do something similar as Joseph did to
>>> proper handle it [5] and mark the tests unsupported if the '/tmp'
>>> is XFS or any buggy one.
>>  
>> What if we did this?
>>
>> (1) Add unit tests that use a container and tmpfs to test functionality.
>>
>> (2) Refactor and use the same test but with the host /tmp and:
>>     - Look for XFS version that is buggy and mark it XFAIL.
>>     - Otherwise FAIL because it's a new kernel bug.
>>
> 
> My main point is such tests are not *unit tests* itself because different
> than other y2038 interface, like gmtime or like, they *do* require kernel
> support to work as intended. And my second point is, since glibc would be
> first gear that stress this out (since many other projects depends on
> out 64 bit time support), make it explicit that the system has a buggy
> interface is what the tests is meant to do.
 No matter how we define "unit test" we still want maximum code testing coverage.

The best way to get that today is with this:

(a) Leave the tests as they are, but add XFS version checking for XFAIL.
    - Fails to test the code paths we built because of XFS.

(b) Add new tests that, instead of stubbing out functionality (like a unit
    test might do) just use an always known-good tmpfs /tmp.
    - Succeeds in increasing code coverage.

-- 
Cheers,
Carlos.