[Y2038] Question about porting y2038-tests to glibc

[Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2136 bytes --]

Dear Community,

I'm going to port a set of Y2038 tests [1] into the glibc test suite.

The operation of porting the code to handle eligible syscalls (like
clock_settime, etc.) seems to be straightforward.

However, one issue is a bit problematic - namely the adjusting of time
when tests run. It would be best to run tests with "current" (i.e. time
not after 03:14:07 UTC 19 January 2038) and after it (when 32 bit
time_t and friends overflow).

Now, I do use a QEMU ARM BSP built with Yocto [2], so there is no issue
to run date +s and set time manually. Also, the time can be adjusted by
the test itself. As I do run it all on the QEMU the Host system is not
affected in any way.

Hence the question - it would be best to tie Y2038 test with all
eligible 32 bit systems (arm, ppc32, mips, riscv32, etc) and run them
when glibc-build-many.py is executed (as part of its check stage).

Possible ideas (the IMHO best approach first):

1. Use of "faketime" [3] to execute test program. This however imposes
the requirement to have it installed (by glibc-build-many as we did with
binutils and compilers). Good news is that there is already a python
wrapper for it [4], so maybe the time can be set for "check" stage.

2. Run tests with QEMU, but this also requires it to be installed and
it may be not easily deployable.

3. Adjust host time in the tests code (via clock_settime/gettime
syscalls). This is IMHO a no-op as we may end up with changed host time
when test/built breaks by any reason.

Or maybe somebody has better idea how to solve this issue?
Thanks in advance for input.

Links:
[1] - https://github.com/lmajewski/y2038-tests/commits/master
[2] - https://github.com/lmajewski/meta-y2038/blob/master/README
[3] - http://manpages.ubuntu.com/manpages/trusty/man1/faketime.1.html
[4] - https://pypi.org/project/faketime/


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Florian Weimer]

* Lukasz Majewski:

> Or maybe somebody has better idea how to solve this issue?

Have time namespaces landed in the upstream kernel?

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

Hi Florian,

> * Lukasz Majewski:
> 
> > Or maybe somebody has better idea how to solve this issue?  
> 
> Have time namespaces landed in the upstream kernel?

Yes, they are supported since 5.6 [1].

However, according to [2] only CLOCK_MONOTONIC and CLOCK_BOOTIME are
supported.

For CLOCK_REALTIME ("NOTES" section):

"Note that time namespaces do not virtualize the CLOCK_REALTIME clock.
Virtualization of this clock was avoided for reasons of complexity and
overhead within the kernel."


As far as I can see - glibc uses syscalls referring to
CLOCK_REALTIME not adjusted by time namespaces. This would cause errors.


Links:
[1] -
https://www.phoronix.com/scan.php?page=news_item&px=Linux-Time-Namespace-Coming

[2] - https://man7.org/linux/man-pages/man7/time_namespaces.7.html

> 
> Thanks,
> Florian




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Joseph Myers]

On Wed, 28 Oct 2020, Lukasz Majewski wrote:

> Hence the question - it would be best to tie Y2038 test with all
> eligible 32 bit systems (arm, ppc32, mips, riscv32, etc) and run them
> when glibc-build-many.py is executed (as part of its check stage).

build-many-glibcs.py, by design, only does compilation tests, not 
execution tests.  It would be possible to add an option to build QEMU and 
run some execution tests under it, for those ABIs supported by QEMU, but 
it would also be a lot of work.

> 1. Use of "faketime" [3] to execute test program. This however imposes
> the requirement to have it installed (by glibc-build-many as we did with
> binutils and compilers). Good news is that there is already a python
> wrapper for it [4], so maybe the time can be set for "check" stage.

We'd need to build such an LD_PRELOAD wrapper ourselves rather than 
depending on an external package for it, and I suspect it wouldn't work 
well for glibc testing (because LD_PRELOAD can't intercept internal calls 
from a libc function to a timing function within the same library, and 
because you'd effectively end up testing the wrapper functions rather than 
the glibc functions, which misses the point of testing).

A wrapper might be more useful for testing if it intercepted syscalls 
rather than intercepting library functions at the LD_PRELOAD level, but 
that would be a lot harder to implement.

> Or maybe somebody has better idea how to solve this issue?

I think any tests requiring changing the system clock are unsuitable for 
the glibc testsuite (until the Linux kernel has time namespace support 
that includes CLOCK_REALTIME - the sort of thing people might want after 
2038 to keep running existing i386 binaries in a namespace with the clock 
set to before 2038).

However, most or all existing glibc tests for time-related interfaces 
should be run both with and without _TIME_BITS=64 (add a new variant of 
each that defines _TIME_BITS and _FILE_OFFSET_BITS to 64 before #include 
of the main test implementation).  And if some time-related interfaces are 
not currently adequately covered in the glibc testsuite, tests should be 
added for them, that are run both with and without _TIME_BITS=64.  (Tests 
of interfaces doing things requiring privileges, such as setting the 
system clock, may only verify that a call to such a function can be 
compiled and linked, not that it does anything useful at runtime.)

Some interfaces can usefully be tested with times outside the range of 
32-bit time_t, without needing to set the system clock.  For example, 
timezone-related interfaces and difftime.  In such cases, it may be useful 
to add tests to make sure such times are covered in the testsuite.  (In 
the timezone case, we have an existing tst-y2039.c, which is UNSUPPORTED 
with 32-bit time_t.  A test that does

#define _TIME_BITS 64
#define _FILE_OFFSET_BITS 64
#include "tst-y2039.c"

should be added, so that that particular bug fix gets tested in the case 
where 64-bit time is not the default but is available with _TIME_BITS=64. 
But there's probably room for more tests using explicit 64-bit times like 
that.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [Y2038] Question about porting y2038-tests to glibc

[Carlos O'Donell]

On 10/28/20 7:29 AM, Lukasz Majewski wrote:
> Hi Florian,
> 
>> * Lukasz Majewski:
>>
>>> Or maybe somebody has better idea how to solve this issue?  
>>
>> Have time namespaces landed in the upstream kernel?
> 
> Yes, they are supported since 5.6 [1].
> 
> However, according to [2] only CLOCK_MONOTONIC and CLOCK_BOOTIME are
> supported.
> 
> For CLOCK_REALTIME ("NOTES" section):
> 
> "Note that time namespaces do not virtualize the CLOCK_REALTIME clock.
> Virtualization of this clock was avoided for reasons of complexity and
> overhead within the kernel."
> 
> 
> As far as I can see - glibc uses syscalls referring to
> CLOCK_REALTIME not adjusted by time namespaces. This would cause errors.

(1) Namespaces.

I think that time namespaces with CLOCK_REALTIME support are going to be
the only ways in which we properly support these tests.

(2) faketime.

I'm not keen to add another bootstrap dependency, particularly one which
will be depended upon for regression testing. While we do rely on python
for building, we don't want to depend upon anything but core modules.

The implementation of faketime is as expected, an LD_PRELOAD that does
what you want with the time values. This could be achieved locally with
less features and tied directly into the support/* infrastructure or
preloaded.

(3) Partial progress.

In the meantime, as Joseph suggests, some things can be tested without
needing to change the system time.

-- 
Cheers,
Carlos.

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 5739 bytes --]

Hi Joseph,

> On Wed, 28 Oct 2020, Lukasz Majewski wrote:
> 
> > Hence the question - it would be best to tie Y2038 test with all
> > eligible 32 bit systems (arm, ppc32, mips, riscv32, etc) and run
> > them when glibc-build-many.py is executed (as part of its check
> > stage).  
> 
> build-many-glibcs.py, by design, only does compilation tests, not 
> execution tests.  It would be possible to add an option to build QEMU
> and run some execution tests under it, for those ABIs supported by
> QEMU, but it would also be a lot of work.

Ok, so QEMU run of tests seems to be an option - especially the "user
mode" https://yurovsky.github.io/2016/12/14/qemu-user-mode.html

seems appealing as we do have cross compiled glibc available anyway. It
looks like we can get the $? of cross compiled program execution as
well.

However, according to:
https://www.qemu.org/docs/master/user/main.html QEMU does the syscalls
translation between target and host - this may be not the same as we
have with normally running kernel.

(But on the other hand - ARM 64 bit time supporting syscalls were added
in 5.1 kernel - now we do have 5.9, so those syscalls shall be
supported already in QEMU).

After running build-many-glibcs.py we do have cross-compiled glibc
(which includes tests).

Then we would just need to run qemu-{arm|mips|ppc} with y2038 tests and
read the status.

> 
> > 1. Use of "faketime" [3] to execute test program. This however
> > imposes the requirement to have it installed (by glibc-build-many
> > as we did with binutils and compilers). Good news is that there is
> > already a python wrapper for it [4], so maybe the time can be set
> > for "check" stage.  
> 
> We'd need to build such an LD_PRELOAD wrapper ourselves rather than 
> depending on an external package for it, and I suspect it wouldn't
> work well for glibc testing (because LD_PRELOAD can't intercept
> internal calls from a libc function to a timing function within the
> same library, and because you'd effectively end up testing the
> wrapper functions rather than the glibc functions, which misses the
> point of testing).

Yes, the faketime uses LD_PRELOAD to redirect eligible syscalls to
libfaketime.

> 
> A wrapper might be more useful for testing if it intercepted syscalls 
> rather than intercepting library functions at the LD_PRELOAD level,
> but that would be a lot harder to implement.

Then IMHO it would be more feasible to just use qemu in the user mode
as outlined above.

> 
> > Or maybe somebody has better idea how to solve this issue?  
> 
> I think any tests requiring changing the system clock are unsuitable
> for the glibc testsuite

I think that they are necessary, as most errors are visible after some
changes in external/internal structures. With setting/reading time after
Y2038 we can check this. Also we can check error paths when used
kernel mismatch glibc with time_t size.

> (until the Linux kernel has time namespace
> support that includes CLOCK_REALTIME - the sort of thing people might
> want after 2038 to keep running existing i386 binaries in a namespace
> with the clock set to before 2038).

Unfortunately, it seems like the CLOCK_REALTIME will not be supported
soon.

> 
> However, most or all existing glibc tests for time-related interfaces 
> should be run both with and without _TIME_BITS=64 (add a new variant
> of each that defines _TIME_BITS and _FILE_OFFSET_BITS to 64 before
> #include of the main test implementation).

Agree.

>  And if some time-related
> interfaces are not currently adequately covered in the glibc
> testsuite, tests should be added for them, that are run both with and
> without _TIME_BITS=64.  (Tests of interfaces doing things requiring
> privileges, such as setting the system clock, may only verify that a
> call to such a function can be compiled and linked, not that it does
> anything useful at runtime.)

Could you elaborate on this idea? Is there any test in glibc which only
checks if function can be compiled and linked? Wouldn't this be too
little to prevent regression?

Most code, which provides 64 bit time support, uses syscalls supporting
64 bit time, which were added to Linux 5.1. It looks natural for me
that we test this by reading / writing time by those syscalls. 

Overflow errors are apparent when we read time after Y2038 32 bit
time_t overflow.

> 
> Some interfaces can usefully be tested with times outside the range
> of 32-bit time_t, without needing to set the system clock.  For
> example, timezone-related interfaces and difftime. 

Yes - I do agree. Unfortunately, those represent just a small subset of
the test covered code (as documented here:
https://github.com/lmajewski/y2038-tests/commits/master).

> In such cases, it
> may be useful to add tests to make sure such times are covered in the
> testsuite.  

Ok.

> (In the timezone case, we have an existing tst-y2039.c,
> which is UNSUPPORTED with 32-bit time_t.  A test that does
> 
> #define _TIME_BITS 64
> #define _FILE_OFFSET_BITS 64
> #include "tst-y2039.c"
> 
> should be added, so that that particular bug fix gets tested in the
> case where 64-bit time is not the default but is available with
> _TIME_BITS=64. But there's probably room for more tests using
> explicit 64-bit times like that.)
> 

Thanks for sharing the above info. I will use it as a template where
possible.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2467 bytes --]

Hi Carlos,

> On 10/28/20 7:29 AM, Lukasz Majewski wrote:
> > Hi Florian,
> >   
> >> * Lukasz Majewski:
> >>  
> >>> Or maybe somebody has better idea how to solve this issue?    
> >>
> >> Have time namespaces landed in the upstream kernel?  
> > 
> > Yes, they are supported since 5.6 [1].
> > 
> > However, according to [2] only CLOCK_MONOTONIC and CLOCK_BOOTIME are
> > supported.
> > 
> > For CLOCK_REALTIME ("NOTES" section):
> > 
> > "Note that time namespaces do not virtualize the CLOCK_REALTIME
> > clock. Virtualization of this clock was avoided for reasons of
> > complexity and overhead within the kernel."
> > 
> > 
> > As far as I can see - glibc uses syscalls referring to
> > CLOCK_REALTIME not adjusted by time namespaces. This would cause
> > errors.  
> 
> (1) Namespaces.
> 
> I think that time namespaces with CLOCK_REALTIME support are going to
> be the only ways in which we properly support these tests.

Unfortunately it is not yet provided.

As I've suggested in the other mail (reply to Joseph) - maybe QEMU user
space emulator: https://www.qemu.org/docs/master/user/main.html

could be used?

However, it would require adding QEMU to test setup. Advantage of such
solution is that one package will support many targets for testing.

> 
> (2) faketime.
> 
> I'm not keen to add another bootstrap dependency, particularly one
> which will be depended upon for regression testing. While we do rely
> on python for building, we don't want to depend upon anything but
> core modules.

I see.

> 
> The implementation of faketime is as expected, an LD_PRELOAD that does
> what you want with the time values. This could be achieved locally
> with less features and tied directly into the support/*
> infrastructure or preloaded.

IMHO, such "interception" would require "catching" syscalls. I do have
a feeling that QEMU or time namespaces with CLOCK_REALTIME would do it
better and more reliable.

> 
> (3) Partial progress.
> 
> In the meantime, as Joseph suggests, some things can be tested without
> needing to change the system time.
> 

But those (unfortunately) are only a small subset of required tests.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Florian Weimer]

* Lukasz Majewski:

> Hi Florian,
>
>> * Lukasz Majewski:
>> 
>> > Or maybe somebody has better idea how to solve this issue?  
>> 
>> Have time namespaces landed in the upstream kernel?
>
> Yes, they are supported since 5.6 [1].
>
> However, according to [2] only CLOCK_MONOTONIC and CLOCK_BOOTIME are
> supported.
>
> For CLOCK_REALTIME ("NOTES" section):
>
> "Note that time namespaces do not virtualize the CLOCK_REALTIME clock.
> Virtualization of this clock was avoided for reasons of complexity and
> overhead within the kernel."

Ohh.  So this is strictly for preserving monotonic clocks during
checkpoint & restore?  That's unfortunate.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: [Y2038] Question about porting y2038-tests to glibc

[Joseph Myers]

On Thu, 29 Oct 2020, Lukasz Majewski wrote:

> > I think any tests requiring changing the system clock are unsuitable
> > for the glibc testsuite
> 
> I think that they are necessary, as most errors are visible after some

Not all tests that may be useful for testing a feature, or necessary for 
making sure it works before it's added to glibc, are suitable for 
inclusion in the glibc testsuite.  The glibc testsuite needs to avoid 
perturbing global system state.  (There are sometimes other reasons a test 
is, although useful, unsuitable for inclusion.  For example, anything 
related to using glibc headers with compilers that aren't supported for 
building glibc itself isn't suitable for the normal glibc testsuite.)

> > without _TIME_BITS=64.  (Tests of interfaces doing things requiring
> > privileges, such as setting the system clock, may only verify that a
> > call to such a function can be compiled and linked, not that it does
> > anything useful at runtime.)
> 
> Could you elaborate on this idea? Is there any test in glibc which only
> checks if function can be compiled and linked? Wouldn't this be too
> little to prevent regression?

I don't have a specific example of such a test, but there have been 
several instances in the past where a function was accidentally missing 
from some architectures (or, although compiled into libc, had no symbol 
version set and so could not actually be linked against), which provides 
clear evidence that if we can't test anything more about a function 
(because it requires privileges) it's still useful to test that the 
function is actually present and a call can be compiled and linked.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3684 bytes --]

Hi Joseph,

> On Thu, 29 Oct 2020, Lukasz Majewski wrote:
> 
> > > I think any tests requiring changing the system clock are
> > > unsuitable for the glibc testsuite  
> > 
> > I think that they are necessary, as most errors are visible after
> > some  
> 
> Not all tests that may be useful for testing a feature, or necessary
> for making sure it works before it's added to glibc, are suitable for 
> inclusion in the glibc testsuite.  The glibc testsuite needs to avoid 
> perturbing global system state.  (There are sometimes other reasons a
> test is, although useful, unsuitable for inclusion.  For example,
> anything related to using glibc headers with compilers that aren't
> supported for building glibc itself isn't suitable for the normal
> glibc testsuite.)

I do get your point. Thanks for shedding some more light on it.

My goal:

1. Best case - add self contained glibc tests for Y2038 compliance to
build-many-glibcs.py, so those would be invoked each time we run this
script. However, it would require adding/building qemu to this build
system. Of course it would be best to have time namespaces supporting
CLOCK_REALTIME, but we don't know when (or if) it will be added.

Pros:
- Regression tests are run each time build-many-glibc is executed.

Cons:
- Linux time namespaces doesn't support CLOCK_REALTIME
- qemu would need to be added to CI glibc setup

2. If 1. is not possible, then QEMU with user mode (to run cross
compiled binaries) would be executed with make check

Pros:
- Just use QEMU (with emulated syscalls - implicite assumption that it
  will correctly emulate 64 bit syscalls - e.g. clock_gettime64 on 32
  bit ARM/PPC) without the need to have rootfs

Cons:
- Not run with build-manu-glibc - run externally with make check &&
  make xcheck

3. Add tests to glibc source base - and use QEMU emulated complete
system (glibc under test, Linux 5.1+, rootfs). Then use
glibc/src/scripts/cross-test-ssh.sh (as described here:
https://github.com/lmajewski/meta-y2038/blob/master/README)

This is the setup which I do use now.

Pros:
- No modification to glibc current CI setup (i.e. build-many-glibcs.py)

Cons:
- Performance and robustness (SSHfs exported directory)
- Need to maintain QEMU based meta layer to build complete system to
  just test a few time related syscalls


> 
> > > without _TIME_BITS=64.  (Tests of interfaces doing things
> > > requiring privileges, such as setting the system clock, may only
> > > verify that a call to such a function can be compiled and linked,
> > > not that it does anything useful at runtime.)  
> > 
> > Could you elaborate on this idea? Is there any test in glibc which
> > only checks if function can be compiled and linked? Wouldn't this
> > be too little to prevent regression?  
> 
> I don't have a specific example of such a test, but there have been 
> several instances in the past where a function was accidentally
> missing from some architectures (or, although compiled into libc, had
> no symbol version set and so could not actually be linked against),
> which provides clear evidence that if we can't test anything more
> about a function (because it requires privileges) it's still useful
> to test that the function is actually present and a call can be
> compiled and linked.
> 

Ok, this seems like an option. Thanks for sharing this idea.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Carlos O'Donell]

On 10/30/20 5:44 AM, Lukasz Majewski wrote:
> 1. Best case - add self contained glibc tests for Y2038 compliance to
> build-many-glibcs.py, so those would be invoked each time we run this
> script. However, it would require adding/building qemu to this build
> system. Of course it would be best to have time namespaces supporting
> CLOCK_REALTIME, but we don't know when (or if) it will be added.
> 
> Pros:
> - Regression tests are run each time build-many-glibc is executed.
> 
> Cons:
> - Linux time namespaces doesn't support CLOCK_REALTIME
> - qemu would need to be added to CI glibc setup

 
> 3. Add tests to glibc source base - and use QEMU emulated complete
> system (glibc under test, Linux 5.1+, rootfs). Then use
> glibc/src/scripts/cross-test-ssh.sh (as described here:
> https://github.com/lmajewski/meta-y2038/blob/master/README)
> 
> This is the setup which I do use now.
> 
> Pros:
> - No modification to glibc current CI setup (i.e. build-many-glibcs.py)
> 
> Cons:
> - Performance and robustness (SSHfs exported directory)
> - Need to maintain QEMU based meta layer to build complete system to
>   just test a few time related syscalls

My technical opinion is as follows:

* We should be open to expanding what build-many-glibcs.py does,
  and providing options to select which operations are carried
  out, including new QEMU tests.

* Using QEMU user is OK in build-many-glibcs.py, but there may be
  false positives due to QEMU emulation of syscalls because of the
  host kernel.

* Using QEMU system in build-many-glibcs.py is even better, and
  would be a gold standard IMO.

-- 
Cheers,
Carlos.

Re: [Y2038] Question about porting y2038-tests to glibc

[Joseph Myers]

On Fri, 30 Oct 2020, Carlos O'Donell via Libc-alpha wrote:

> My technical opinion is as follows:
> 
> * We should be open to expanding what build-many-glibcs.py does,
>   and providing options to select which operations are carried
>   out, including new QEMU tests.
> 
> * Using QEMU user is OK in build-many-glibcs.py, but there may be
>   false positives due to QEMU emulation of syscalls because of the
>   host kernel.
> 
> * Using QEMU system in build-many-glibcs.py is even better, and
>   would be a gold standard IMO.

Options for those would be reasonable (they'd need to do the QEMU tests 
only for the subset of architectures and ABIs supported with QEMU), but 
getting to a clean baseline for even one of the supported glibc ABIs 
(thus, if using QEMU user emulation, annotating all the tests that might 
fail in such a configuration, reliably or randomly, because they use 
functionality such as threads and signals that's problematic for userspace 
QEMU - as well as the general matter of execution tests that may not 
reliably pass for all configurations) would probably be a lot of work.  
And a clean baseline is much better than needing to track known test 
failures individually.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [Y2038] Question about porting y2038-tests to glibc

[Carlos O'Donell]

On 10/30/20 1:08 PM, Joseph Myers wrote:
> On Fri, 30 Oct 2020, Carlos O'Donell via Libc-alpha wrote:
> 
>> My technical opinion is as follows:
>>
>> * We should be open to expanding what build-many-glibcs.py does,
>>   and providing options to select which operations are carried
>>   out, including new QEMU tests.
>>
>> * Using QEMU user is OK in build-many-glibcs.py, but there may be
>>   false positives due to QEMU emulation of syscalls because of the
>>   host kernel.
>>
>> * Using QEMU system in build-many-glibcs.py is even better, and
>>   would be a gold standard IMO.
> 
> Options for those would be reasonable (they'd need to do the QEMU tests 
> only for the subset of architectures and ABIs supported with QEMU), but 
> getting to a clean baseline for even one of the supported glibc ABIs 
> (thus, if using QEMU user emulation, annotating all the tests that might 
> fail in such a configuration, reliably or randomly, because they use 
> functionality such as threads and signals that's problematic for userspace 
> QEMU - as well as the general matter of execution tests that may not 
> reliably pass for all configurations) would probably be a lot of work.  
> And a clean baseline is much better than needing to track known test 
> failures individually.
 
I agree.

I would accept QEMU usage in build-many-glibcs.py, and I also agree
that such checks must have a clean baseline.

Lukasz, Does this give you enough direction to pursue a solution?

-- 
Cheers,
Carlos.

Re: [Y2038] Question about porting y2038-tests to glibc

[Joseph Myers]

On Fri, 30 Oct 2020, Carlos O'Donell via Libc-alpha wrote:

> I would accept QEMU usage in build-many-glibcs.py, and I also agree
> that such checks must have a clean baseline.

Note that anything involving QEMU user emulation for these time tests 
probably needs QEMU work to add an option to use a different time for the 
emulated program, unless such an option already exists.  (For that matter, 
QEMU system emulation would also need such an option.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [Y2038] Question about porting y2038-tests to glibc

[Carlos O'Donell]

On 10/30/20 1:29 PM, Joseph Myers wrote:
> On Fri, 30 Oct 2020, Carlos O'Donell via Libc-alpha wrote:
> 
>> I would accept QEMU usage in build-many-glibcs.py, and I also agree
>> that such checks must have a clean baseline.
> 
> Note that anything involving QEMU user emulation for these time tests 
> probably needs QEMU work to add an option to use a different time for the 
> emulated program, unless such an option already exists.  (For that matter, 
> QEMU system emulation would also need such an option.)
 
The option for '-rtc [base=utc|localtime|datetime][,clock=host|rt|vm][,driftfix=none|slew]'
exists today. It could be tested to verify it works in user or system modes
and as such represents a way forward. So '-rtc base=20380120,clock=rt' should
put you past the y2038 transition, and '-rtc base=20380119,clock=rt' should
put you before the y2038.

-- 
Cheers,
Carlos.

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2088 bytes --]

Hi Carlos, Joseph,

> On 10/30/20 1:08 PM, Joseph Myers wrote:
> > On Fri, 30 Oct 2020, Carlos O'Donell via Libc-alpha wrote:
> >   
> >> My technical opinion is as follows:
> >>
> >> * We should be open to expanding what build-many-glibcs.py does,
> >>   and providing options to select which operations are carried
> >>   out, including new QEMU tests.
> >>
> >> * Using QEMU user is OK in build-many-glibcs.py, but there may be
> >>   false positives due to QEMU emulation of syscalls because of the
> >>   host kernel.
> >>
> >> * Using QEMU system in build-many-glibcs.py is even better, and
> >>   would be a gold standard IMO.  
> > 
> > Options for those would be reasonable (they'd need to do the QEMU
> > tests only for the subset of architectures and ABIs supported with
> > QEMU), but getting to a clean baseline for even one of the
> > supported glibc ABIs (thus, if using QEMU user emulation,
> > annotating all the tests that might fail in such a configuration,
> > reliably or randomly, because they use functionality such as
> > threads and signals that's problematic for userspace QEMU - as well
> > as the general matter of execution tests that may not reliably pass
> > for all configurations) would probably be a lot of work. And a
> > clean baseline is much better than needing to track known test
> > failures individually.  
>  
> I agree.
> 
> I would accept QEMU usage in build-many-glibcs.py, and I also agree
> that such checks must have a clean baseline.
> 
> Lukasz, Does this give you enough direction to pursue a solution?
> 

Yes. I think that we now have a good foundation to investigate the
possible solution.

Now, I do need to check if recent qemu-arm supports emulation of 64 bit
time related syscalls (e.g. clock_settime64) in the "user mode".


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3049 bytes --]

Hi, 

> Hi Carlos, Joseph,
> 
> > On 10/30/20 1:08 PM, Joseph Myers wrote:  
> > > On Fri, 30 Oct 2020, Carlos O'Donell via Libc-alpha wrote:
> > >     
> > >> My technical opinion is as follows:
> > >>
> > >> * We should be open to expanding what build-many-glibcs.py does,
> > >>   and providing options to select which operations are carried
> > >>   out, including new QEMU tests.
> > >>
> > >> * Using QEMU user is OK in build-many-glibcs.py, but there may be
> > >>   false positives due to QEMU emulation of syscalls because of
> > >> the host kernel.
> > >>
> > >> * Using QEMU system in build-many-glibcs.py is even better, and
> > >>   would be a gold standard IMO.    
> > > 
> > > Options for those would be reasonable (they'd need to do the QEMU
> > > tests only for the subset of architectures and ABIs supported with
> > > QEMU), but getting to a clean baseline for even one of the
> > > supported glibc ABIs (thus, if using QEMU user emulation,
> > > annotating all the tests that might fail in such a configuration,
> > > reliably or randomly, because they use functionality such as
> > > threads and signals that's problematic for userspace QEMU - as
> > > well as the general matter of execution tests that may not
> > > reliably pass for all configurations) would probably be a lot of
> > > work. And a clean baseline is much better than needing to track
> > > known test failures individually.    
> >  
> > I agree.
> > 
> > I would accept QEMU usage in build-many-glibcs.py, and I also agree
> > that such checks must have a clean baseline.
> > 
> > Lukasz, Does this give you enough direction to pursue a solution?
> >   
> 
> Yes. I think that we now have a good foundation to investigate the
> possible solution.
> 
> Now, I do need to check if recent qemu-arm supports emulation of 64
> bit time related syscalls (e.g. clock_settime64) in the "user mode".

It looks like the "user mode" will not help us much. It adjust received
from target syscalls and pass them to host.

This means that clock_settime64 will adjust host time.

It now looks to me that the most feasible solution would be to use
./cross-test-ssh.sh script with QEMU emulated ARM system [1].

I do use there sshfs to mount glibc build directory. In terms of
performance it could be replaced with NFS or I could use ssh to execute
test program (built with e.g. -static) and get result.

Links:
[1] - https://github.com/lmajewski/meta-y2038/

> 
> 
> Best regards,
> 
> Lukasz Majewski
> 
> --
> 
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email:
> lukma@denx.de




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Carlos O'Donell]

On 11/4/20 6:37 AM, Lukasz Majewski wrote:
> It looks like the "user mode" will not help us much. It adjust received
> from target syscalls and pass them to host.
> 
> This means that clock_settime64 will adjust host time.
> 
> It now looks to me that the most feasible solution would be to use
> ./cross-test-ssh.sh script with QEMU emulated ARM system [1].
> 
> I do use there sshfs to mount glibc build directory. In terms of
> performance it could be replaced with NFS or I could use ssh to execute
> test program (built with e.g. -static) and get result.

This is disappointing.

It is the least appealing option because of the amount of maintenance
we need to do to keep this option working.

How do we plan to maintain this solution?

* How is the kernel built?
  - From kernel sources and defconfig?
* How is the kernel updated?
  - qemu is told to run a specific built kernel
* How is the VM image maintained?
* Where does the VM image come from?

-- 
Cheers,
Carlos.

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 1836 bytes --]

Hi Carlos,

> On 11/4/20 6:37 AM, Lukasz Majewski wrote:
> > It looks like the "user mode" will not help us much. It adjust
> > received from target syscalls and pass them to host.
> > 
> > This means that clock_settime64 will adjust host time.
> > 
> > It now looks to me that the most feasible solution would be to use
> > ./cross-test-ssh.sh script with QEMU emulated ARM system [1].
> > 
> > I do use there sshfs to mount glibc build directory. In terms of
> > performance it could be replaced with NFS or I could use ssh to
> > execute test program (built with e.g. -static) and get result.  
> 
> This is disappointing.
> 
> It is the least appealing option because of the amount of maintenance
> we need to do to keep this option working.

Another very problematic option would be to build and test the glibc
solely in ARM VM (e.g. 4x Cortex-A9) .

The problem is that performance is very poor. The feasible solution
which I've found is described here:
https://github.com/lmajewski/meta-y2038/blob/master/README#L79

> 
> How do we plan to maintain this solution?
> 
> * How is the kernel built?
>   - From kernel sources and defconfig?
> * How is the kernel updated?
>   - qemu is told to run a specific built kernel
> * How is the VM image maintained?
> * Where does the VM image come from?
> 

It is all in one place - 
https://github.com/lmajewski/meta-y2038/blob/master/README

I do maintain such Yocto/OE meta layer for ARM 32 bit SoC system.
However, it cannot be easily integrated into build-many-glibcs.py
script.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Y2038] Question about porting y2038-tests to glibc

[Joseph Myers]

On Wed, 4 Nov 2020, Carlos O'Donell via Libc-alpha wrote:

> > It now looks to me that the most feasible solution would be to use
> > ./cross-test-ssh.sh script with QEMU emulated ARM system [1].
> > 
> > I do use there sshfs to mount glibc build directory. In terms of
> > performance it could be replaced with NFS or I could use ssh to execute
> > test program (built with e.g. -static) and get result.
> 
> This is disappointing.
> 
> It is the least appealing option because of the amount of maintenance
> we need to do to keep this option working.
> 
> How do we plan to maintain this solution?

I suspect the complexity of any working solution indicates this is not 
something that should be required to be integrated in glibc for inclusion 
of the Y2038 support - although it's useful for such tests to be run on as 
many architectures as possible to verify the Y2038 support before 
inclusion.

Maybe there should be a separate set somewhere of "tests for glibc 
functionality that modify global system state in a way that means they 
need to be run on a VM and different tests have to be run in series not in 
parallel" (and that includes whatever infrastructure is needed to set up 
such VMs automatically).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [Y2038] Question about porting y2038-tests to glibc

[Carlos O'Donell]

On 11/4/20 2:17 PM, Joseph Myers wrote:
> On Wed, 4 Nov 2020, Carlos O'Donell via Libc-alpha wrote:
> 
>>> It now looks to me that the most feasible solution would be to use
>>> ./cross-test-ssh.sh script with QEMU emulated ARM system [1].
>>>
>>> I do use there sshfs to mount glibc build directory. In terms of
>>> performance it could be replaced with NFS or I could use ssh to execute
>>> test program (built with e.g. -static) and get result.
>>
>> This is disappointing.
>>
>> It is the least appealing option because of the amount of maintenance
>> we need to do to keep this option working.
>>
>> How do we plan to maintain this solution?
> 
> I suspect the complexity of any working solution indicates this is not 
> something that should be required to be integrated in glibc for inclusion 
> of the Y2038 support - although it's useful for such tests to be run on as 
> many architectures as possible to verify the Y2038 support before 
> inclusion.
> 
> Maybe there should be a separate set somewhere of "tests for glibc 
> functionality that modify global system state in a way that means they 
> need to be run on a VM and different tests have to be run in series not in 
> parallel" (and that includes whatever infrastructure is needed to set up 
> such VMs automatically).
 
I agree.

If qemu-user can't be used then the overally complexity is going to be too
high warrant maintaining it via build-many-glibcs.py.

We are just going to have to do y2038 testing manually until we have a
future where CI/CD could be used to enable these test. Even then the average
CI/CD system would buckle under the load. We would need other kinds of test
infrastructure we've never developed because the value wasn't there.

-- 
Cheers,
Carlos.

Re: [Y2038] Question about porting y2038-tests to glibc

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

Hi Joseph, Carlos,

> On Wed, 4 Nov 2020, Carlos O'Donell via Libc-alpha wrote:
> 
> > > It now looks to me that the most feasible solution would be to use
> > > ./cross-test-ssh.sh script with QEMU emulated ARM system [1].
> > > 
> > > I do use there sshfs to mount glibc build directory. In terms of
> > > performance it could be replaced with NFS or I could use ssh to
> > > execute test program (built with e.g. -static) and get result.  
> > 
> > This is disappointing.
> > 
> > It is the least appealing option because of the amount of
> > maintenance we need to do to keep this option working.
> > 
> > How do we plan to maintain this solution?  
> 
> I suspect the complexity of any working solution indicates this is
> not something that should be required to be integrated in glibc for
> inclusion of the Y2038 support 

Ok.

> - although it's useful for such tests
> to be run on as many architectures as possible to verify the Y2038
> support before inclusion.

Yes. Correct. 

> 
> Maybe there should be a separate set somewhere of "tests for glibc 
> functionality that modify global system state in a way that means
> they need to be run on a VM and different tests have to be run in
> series not in parallel" (and that includes whatever infrastructure is
> needed to set up such VMs automatically).
> 

I do have such solution - https://github.com/lmajewski/meta-y2038/




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]