settimeofday breakage

settimeofday breakage

[J William Piggott]

  > NEWS Version 2.31
  > Deprecated and removed features, and other changes affecting compatibility:
  >
  >   However, to reduce the odds of this offset being set by accident,
  >   settimeofday can no longer be used to set the time and the offset
  >   simultaneously.  If both of its two arguments are non-null, the call
  >   will fail (setting errno to EINVAL).

I do not know who wrote this and my comment is not personal, but I
question the stated justification for this breakage.

The behavior of this function when called with two args has been in
place for nearly 30 years; I have never see a report of the kernel time
zone being set by 'accident'.

Then there is this recent comment:

  > Date: Thu, 6 Feb 2020 21:41:06 +0000
  > From: Joseph Myers <joseph@codesourcery.com>
  >
  > On Wed, 5 Feb 2020, Adhemerval Zanella wrote:
  >
  > > Once POSIX does remove the symbol, we can remove its definition from
  > > default visibility
  >
  > I don't see that being appropriate for a very long time.
  >

That was regarding gettimeofday, but should apply equally to
settimeofday.

I don't see how these two positions are reconciled:

  * get/settimeofday will be supported for the foreseeable future

  * settimeofday will be intentionally broken to prevent an 'accident'
    when using the 28 year old expected behavior.

Below is a table that I wrote for hwclock(8) to aid source readers in
understanding the block of code that follows it:

  *
  * +-------------------------------------------------------------------+
  * |                       settimeofday(tv, tz)                        |
  * |-------------------------------------------------------------------|
  * |     Arguments     |  System Time  | PCIL |           | warp_clock |
  * |   tv    |   tz    | set  | warped | set  | firsttime |   locked   |
  * |---------|---------|---------------|------|-----------|------------|
  * | pointer | NULL    |  yes |   no   |  no  |     1     |    no      |
  * | pointer | pointer |  yes |   no   |  no  |     0     |    yes     |
  * | NULL    | ptr2utc |  no  |   no   |  no  |     0     |    yes     |
  * | NULL    | pointer |  no  |   yes  |  yes |     0     |    yes     |
  * +-------------------------------------------------------------------+
  * ptr2utc: tz.tz_minuteswest is zero (UTC).
  * PCIL: persistent_clock_is_local, sets the "11 minute mode" timescale.
  * firsttime: locks the warp_clock function (initialized to 1 at boot).
  *

Note the unique behavior when called with two arguments which cannot be
easily reproduced.

What I used to accomplish in hwclock(8) with one call, now takes *three*!

Re: settimeofday breakage

[Adhemerval Zanella]

On 11/02/2020 12:33, J William Piggott wrote:
>  > NEWS Version 2.31
>  > Deprecated and removed features, and other changes affecting compatibility:
>  >
>  >   However, to reduce the odds of this offset being set by accident,
>  >   settimeofday can no longer be used to set the time and the offset
>  >   simultaneously.  If both of its two arguments are non-null, the call
>  >   will fail (setting errno to EINVAL).
> 
> I do not know who wrote this and my comment is not personal, but I
> question the stated justification for this breakage.
> 
> The behavior of this function when called with two args has been in
> place for nearly 30 years; I have never see a report of the kernel time
> zone being set by 'accident'.
> 
> Then there is this recent comment:
> 
>  > Date: Thu, 6 Feb 2020 21:41:06 +0000
>  > From: Joseph Myers <joseph@codesourcery.com>
>  >
>  > On Wed, 5 Feb 2020, Adhemerval Zanella wrote:
>  >
>  > > Once POSIX does remove the symbol, we can remove its definition from
>  > > default visibility
>  >
>  > I don't see that being appropriate for a very long time.
>  >
> 
> That was regarding gettimeofday, but should apply equally to
> settimeofday.
> 
> I don't see how these two positions are reconciled:
> 
>  * get/settimeofday will be supported for the foreseeable future
> 
>  * settimeofday will be intentionally broken to prevent an 'accident'
>    when using the 28 year old expected behavior.
> 
> Below is a table that I wrote for hwclock(8) to aid source readers in
> understanding the block of code that follows it:
> 
>  *
>  * +-------------------------------------------------------------------+
>  * |                       settimeofday(tv, tz)                        |
>  * |-------------------------------------------------------------------|
>  * |     Arguments     |  System Time  | PCIL |           | warp_clock |
>  * |   tv    |   tz    | set  | warped | set  | firsttime |   locked   |
>  * |---------|---------|---------------|------|-----------|------------|
>  * | pointer | NULL    |  yes |   no   |  no  |     1     |    no      |
>  * | pointer | pointer |  yes |   no   |  no  |     0     |    yes     |
>  * | NULL    | ptr2utc |  no  |   no   |  no  |     0     |    yes     |
>  * | NULL    | pointer |  no  |   yes  |  yes |     0     |    yes     |
>  * +-------------------------------------------------------------------+
>  * ptr2utc: tz.tz_minuteswest is zero (UTC).
>  * PCIL: persistent_clock_is_local, sets the "11 minute mode" timescale.
>  * firsttime: locks the warp_clock function (initialized to 1 at boot).
>  *
> 
> Note the unique behavior when called with two arguments which cannot be
> easily reproduced.
> 
> What I used to accomplish in hwclock(8) with one call, now takes *three*!

The start of the discussion, if I recall correctly, was on how handle
new time64 Linux ABIs that do not have this kernel support [1.1][1.2].

This lead to a more in-depth discussion if the offset handling should
be handled in kernel and/or if should be deprecated as well [2].  Linus
has agreed that:

"We should not do TZ on a kernel level at all. At least not a global
one. It makes no sense.

If the original TZ had been defined to have some sane model (perhaps
per session? Something like that), it would be worth doing. As it is,
a global TZ is just plain wrong. Per process would be sane (but
largely useless, I suspect)."

And Theodore Y. T.S'o added:

"As I recall (and I may or may not have been original for the original
sys_tz; it was many years ago, and my memories of 1992 are a bit
fuzzy) the only reason why we added it was because x86 systems that
were dual-booting with Windows had a RTC which ticked localtime, and
originally, the system time was fetched from the RTC in early boot,
and then when the timezone was set, the time would be warped so it
would be correct."

And answering a questioning from Leonard Poettering, Alexandre Belloni
stated:

"> What's the benefit of letting userspace do this? It sounds a lot more
> fragile to leave this syncing to userspace if the kernel can do this
> trivially on its own.
>

It does it trivially and badly:

 -  hctosys will always think the RTC is in UTC so if the RTC is in
    local time, you will anyway have up to 12 hours difference until
userspace fixes that.

 - the way systohc and hctosys are working will lead up to a 2 second
   drift until ntp runs which is an issue for NTP stratum servers. My
tests show that there is a way for userspace to reduce that to tens of
nanoseconds but this means having a one or two seconds delay when setting
and reading the time. I want that to be opt-in.

 - the RTC to be used for hctosys and systohc is hardcoded in Kconfig
   and distro usually let the default rtc0 but many platforms have a non
functional RTC that ends up being rtc0. I would prefer that to be a
userspace configuration change instead of a kernel configuration change"

So, it seems that current API is far from perfect and won't be supported
on newer architectures.  I am not sure which is current kernel stand on
such matter, and systemd has also an open defect to track it [3].

From glibc standpoint, we agreed that this offset setting interface is
clunky is very specific, so  handle multiple architectures and have a
concise semantic we decided to deprecate its usage.


[1.1] https://sourceware.org/ml/libc-alpha/2019-06/msg00609.html
[1.2] https://patchwork.ozlabs.org/patch/1121610/
[2] https://sourceware.org/ml/libc-alpha/2019-08/msg00277.html
[3] https://github.com/systemd/systemd/issues/13305

Re: settimeofday breakage

[J William Piggott]

[-- Attachment #1: Type: text/plain, Size: 7001 bytes --]



On Tue, 11 Feb 2020, Adhemerval Zanella wrote:

>
>
> On 11/02/2020 12:33, J William Piggott wrote:
>>  > NEWS Version 2.31
>>  > Deprecated and removed features, and other changes affecting compatibility:
>>  >
>>  >   However, to reduce the odds of this offset being set by accident,
>>  >   settimeofday can no longer be used to set the time and the offset
>>  >   simultaneously.  If both of its two arguments are non-null, the call
>>  >   will fail (setting errno to EINVAL).
>>
>> I do not know who wrote this and my comment is not personal, but I
>> question the stated justification for this breakage.
>>
>> The behavior of this function when called with two args has been in
>> place for nearly 30 years; I have never see a report of the kernel time
>> zone being set by 'accident'.
>>
>> Then there is this recent comment:
>>
>>  > Date: Thu, 6 Feb 2020 21:41:06 +0000
>>  > From: Joseph Myers <joseph@codesourcery.com>
>>  >
>>  > On Wed, 5 Feb 2020, Adhemerval Zanella wrote:
>>  >
>>  > > Once POSIX does remove the symbol, we can remove its definition from
>>  > > default visibility
>>  >
>>  > I don't see that being appropriate for a very long time.
>>  >
>>
>> That was regarding gettimeofday, but should apply equally to
>> settimeofday.
>>
>> I don't see how these two positions are reconciled:
>>
>>  * get/settimeofday will be supported for the foreseeable future
>>
>>  * settimeofday will be intentionally broken to prevent an 'accident'
>>    when using the 28 year old expected behavior.
>>
>> Below is a table that I wrote for hwclock(8) to aid source readers in
>> understanding the block of code that follows it:
>>
>>  *
>>  * +-------------------------------------------------------------------+
>>  * |                       settimeofday(tv, tz)                        |
>>  * |-------------------------------------------------------------------|
>>  * |     Arguments     |  System Time  | PCIL |           | warp_clock |
>>  * |   tv    |   tz    | set  | warped | set  | firsttime |   locked   |
>>  * |---------|---------|---------------|------|-----------|------------|
>>  * | pointer | NULL    |  yes |   no   |  no  |     1     |    no      |
>>  * | pointer | pointer |  yes |   no   |  no  |     0     |    yes     |
>>  * | NULL    | ptr2utc |  no  |   no   |  no  |     0     |    yes     |
>>  * | NULL    | pointer |  no  |   yes  |  yes |     0     |    yes     |
>>  * +-------------------------------------------------------------------+
>>  * ptr2utc: tz.tz_minuteswest is zero (UTC).
>>  * PCIL: persistent_clock_is_local, sets the "11 minute mode" timescale.
>>  * firsttime: locks the warp_clock function (initialized to 1 at boot).
>>  *
>>
>> Note the unique behavior when called with two arguments which cannot be
>> easily reproduced.
>>
>> What I used to accomplish in hwclock(8) with one call, now takes *three*!
>
> The start of the discussion, if I recall correctly, was on how handle
> new time64 Linux ABIs that do not have this kernel support [1.1][1.2].
>
> This lead to a more in-depth discussion if the offset handling should
> be handled in kernel and/or if should be deprecated as well [2].  Linus
> has agreed that:
>
> "We should not do TZ on a kernel level at all. At least not a global
> one. It makes no sense.
>
> If the original TZ had been defined to have some sane model (perhaps
> per session? Something like that), it would be worth doing. As it is,
> a global TZ is just plain wrong. Per process would be sane (but
> largely useless, I suspect)."
>
> And Theodore Y. T.S'o added:
>
> "As I recall (and I may or may not have been original for the original
> sys_tz; it was many years ago, and my memories of 1992 are a bit
> fuzzy) the only reason why we added it was because x86 systems that
> were dual-booting with Windows had a RTC which ticked localtime, and
> originally, the system time was fetched from the RTC in early boot,
> and then when the timezone was set, the time would be warped so it
> would be correct."
>
> And answering a questioning from Leonard Poettering, Alexandre Belloni
> stated:
>
> "> What's the benefit of letting userspace do this? It sounds a lot more
>> fragile to leave this syncing to userspace if the kernel can do this
>> trivially on its own.
>>
>
> It does it trivially and badly:
>
> -  hctosys will always think the RTC is in UTC so if the RTC is in
>    local time, you will anyway have up to 12 hours difference until
> userspace fixes that.
>
> - the way systohc and hctosys are working will lead up to a 2 second
>   drift until ntp runs which is an issue for NTP stratum servers. My
> tests show that there is a way for userspace to reduce that to tens of
> nanoseconds but this means having a one or two seconds delay when setting
> and reading the time. I want that to be opt-in.
>
> - the RTC to be used for hctosys and systohc is hardcoded in Kconfig
>   and distro usually let the default rtc0 but many platforms have a non
> functional RTC that ends up being rtc0. I would prefer that to be a
> userspace configuration change instead of a kernel configuration change"
>
> So, it seems that current API is far from perfect and won't be supported
> on newer architectures.  I am not sure which is current kernel stand on
> such matter, and systemd has also an open defect to track it [3].
>
> From glibc standpoint, we agreed that this offset setting interface is
> clunky is very specific, so  handle multiple architectures and have a
> concise semantic we decided to deprecate its usage.
>
>
> [1.1] https://sourceware.org/ml/libc-alpha/2019-06/msg00609.html
> [1.2] https://patchwork.ozlabs.org/patch/1121610/
> [2] https://sourceware.org/ml/libc-alpha/2019-08/msg00277.html
> [3] https://github.com/systemd/systemd/issues/13305
>

Adhemerval,

Thank you for taking the time to make this nice summary for me, it is
much appreciated. I haven't read your references yet, but I will.

One thing I do not see mentioned in your summary is the kernel ntp '11
minute mode', which will clobber a localtime RTC unless the kernel is
informed about it from userspace. If I recall correctly, as long ntp is
in sync '11 minute mode' cannot be turned off.

From a hwclock(8) maintenance prospective the question to be answered
is: is the Linux kernel going to stop supporting a localtime RTC?

Your summary makes it sound so. If yes, we simply strip that
functionality out of hwclock(8) and when the complaints come in we just
tell them Linux abandoned localtime RTC and send them to Linus.

Just kidding Linus, well mostly ;)  And of course, it's up to Karel what
happens in util-linux, not me.

Re: settimeofday breakage

[Joseph Myers]

On Tue, 11 Feb 2020, J William Piggott wrote:

>  > > Once POSIX does remove the symbol, we can remove its definition from
>  > > default visibility
>  >
>  > I don't see that being appropriate for a very long time.
>  >
> 
> That was regarding gettimeofday, but should apply equally to
> settimeofday.

gettimeofday is used by very many applications that want the current time 
with more precision than seconds (but for which microseconds are good 
enough).

settimeofday is used by very few applications (getting the current time is 
an unprivileged operation, much more common than changing it).

The cases are quite different.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: settimeofday breakage

[Karel Zak]

On Tue, Feb 11, 2020 at 03:43:48PM -0500, J William Piggott wrote:
> On Tue, 11 Feb 2020, Adhemerval Zanella wrote:
> > This lead to a more in-depth discussion if the offset handling should
> > be handled in kernel and/or if should be deprecated as well [2].  Linus
> > has agreed that:
> > 
> > "We should not do TZ on a kernel level at all. At least not a global
> > one. It makes no sense.

Well, I guess we all agree that global TZ is nothing nice and in an
ideal world all computers keep primary time resource(s) in in UTC, ...
but MS-DOS era has introduced locatime RTC and this legacy is still with
us and we probably need (?) to somehow deal with dual boots.

> > So, it seems that current API is far from perfect and won't be supported
> > on newer architectures.  I am not sure which is current kernel stand on
> > such matter, and systemd has also an open defect to track it [3].
> > 
> > From glibc standpoint, we agreed that this offset setting interface is
> > clunky is very specific, so  handle multiple architectures and have a
> > concise semantic we decided to deprecate its usage.

I don't think we need any extra support from glibc. 

From my point of view it does not make sense to maintain in libc extra API 
for only systemd and hwclock to set TZ. It would be enough to have some RTC 
specific ioctl(s) to deal with this very specific task.

I think it was already suggested by Lennart at:
 https://github.com/systemd/systemd/issues/13305

Frankly, this is so specific that I don't care about API elegance...

> From a hwclock(8) maintenance prospective the question to be answered
> is: is the Linux kernel going to stop supporting a localtime RTC?

Right. This is the point.

We (userspace) will follow kernel decision, what else ;-) So, all is
in LKML hands.

> Your summary makes it sound so. If yes, we simply strip that
> functionality out of hwclock(8) and when the complaints come in we just
> tell them Linux abandoned localtime RTC and send them to Linus.

Maybe we're just too soft and careful and year 2020 is the right time to say 
No to localtime RTC :-)

(Lennart added to CC)

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com