[Bug libc/11620] Bad design of timezone conversions

[Bug libc/11620] Bad design of timezone conversions

[maxim.yegorushkin at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=11620

Maxim Yegorushkin <maxim.yegorushkin at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |maxim.yegorushkin at gmail
                   |                            |dot com
         Resolution|WONTFIX                     |

--- Comment #2 from Maxim Yegorushkin <maxim.yegorushkin at gmail dot com> 2012-11-08 11:21:32 UTC ---
I would like to re-open this issue now, much has changed since the last
comment.

glibc has a timezone database parser. It only makes perfect sense to reuse the
functionality and provide extra functions for converting times from any
timezone to UTC and back in a useful and thread-safe manner. Many modern
application require this functionality.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[maxim.yegorushkin at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=11620

--- Comment #3 from Maxim Yegorushkin <maxim.yegorushkin at gmail dot com> 2012-11-08 23:57:21 UTC ---
I just wanted to expand on what I said earlier because I really believe this
would be a major improvement in timezone handling.

Ulrich mentioned that "There is absolutely no reason whatsoever that any such
set
of interfaces have to be in the C library.". It seems to me that for this
argument to be valid there should not be examples to the contrary. 

There is a standard C library function gmtime(). There is no reverse function
for it in the C standard. People often ask for the reverse of this function and
a quick google search for "gmtime reverse" retrieves around 1.3e6 hits, as of
today. Yet, glibc does provide a reverse of it, timegm(), a more than just a
useful function judging from the number of the web search hits. 

This example appears to contradict Ulrich's statement that was used as a
justification for closing this request. It feels like the reasoning for closing
this ticket was less than perfect and I felt compelled to re-open it. 

I looked at the sources of glibc today and it looks like __tzfile_read() loads
an entire Olson database into memory. This database allows conversion from any
timezone to UTC and back. Glibc provides functions for converting back and
forth between UTC and the current local timezone only. To complete the picture,
there is a "solution" mentioned in NOTES of man timegm, that involves changing
TZ environment variable to temporarily switch the local timezone to another one
to force mktime() convert struct tm expressed in that other timezone into a
time_t. This solution doesn't feel quite satisfactory and it is not thread-safe
(unless one explicitly holds a mutex while changing TZ environment variable
everywhere in the code, which may be harder to achieve in the presence of
3rd-party libraries without the source code).

To summarize, the Olson timezone database has always been in glibc's memory,
but applications haven't been able to fully utilize that, or at least, without
that verbose and unreliable code, which also doesn't seem to be a common
knowledge.

I would propose adding a couple of functions to fill this gap:

    time_t time_tz_to_utc(<timezone> tz, struct tm* from);
    struct tm* time_utc_to_tz(<timezone> tz, time_t);

The first function would convert a broken-down time expressed in timezone tz
into time_t (which is the number of seconds since UTC epoch).

The second function would do the opposite.

<timezone> should probably be a pointer to a structure, rather than a timezone
string, e.g. "Europe/Paris", to avoid string look-ups on each call. E.g.:

    typedef struct __Timezone* timezone_t;
    timezone_t time_find_timezone(char const* olson_tz_name);
    void time_release_timezone(timezone_t);

I remember reading other proposals to evolve the C standard library time
functions, but can't find it right now. It may provide more well-thought
interfaces. But, I think, for the majority of us this would be a major step
forward.

-- Maxim

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[maxim.yegorushkin at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=11620

--- Comment #4 from Maxim Yegorushkin <maxim.yegorushkin at gmail dot com> 2012-11-09 00:23:00 UTC ---
...

Because of the lack of this functionality in glibc there is a proliferation of
different tzdata copies in Linux. Python's pytz comes with a copy, ICU with
another one, to name a few.

Worst case scenario is that an application using one copy of tzdata talks to
another application using another copy of tzdata using non-local timezone
timestamps. tzdata mismatch can lead to unexpected behaviour, especially for
recent dates if one library has upgraded to the latest tzdata when the other
hasn't. (This problem, though, is inherent in web applications served by
different hosts, which can't be reasonably expected to have tzdata in-sync,
hence to mitigate the problem they must only talk UTC).

Different hosts aside, keeping just one copy of tzdata on a system feels to be
also a sensible idea.

Okay, I will try to keep trolling down now...

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[schwab@linux-m68k.org]

http://sourceware.org/bugzilla/show_bug.cgi?id=11620

--- Comment #5 from Andreas Schwab <schwab@linux-m68k.org> 2012-11-09 00:46:36 UTC ---
> Yet, glibc does provide a reverse of it, timegm(),

Only because it was prior art as part of the Olsen implementation.

> I looked at the sources of glibc today and it looks like __tzfile_read() loads
> an entire Olson database into memory.

No, it doesn't.  It loads a single time zone file.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[maxim.yegorushkin at gmail dot com]

http://sourceware.org/bugzilla/show_bug.cgi?id=11620

--- Comment #6 from Maxim Yegorushkin <maxim.yegorushkin at gmail dot com> 2012-11-09 10:34:24 UTC ---
(In reply to comment #5)

> > I looked at the sources of glibc today and it looks like __tzfile_read() loads
> > an entire Olson database into memory.
> 
> No, it doesn't.  It loads a single time zone file.

I stand corrected. 

Doesn't change the fact that it is capable of loading any timezone file on
demand though.

-- 
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[david at lang dot hm]

http://sourceware.org/bugzilla/show_bug.cgi?id=11620

David Lang <david at lang dot hm> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |david at lang dot hm

--- Comment #7 from David Lang <david at lang dot hm> ---
I'll start by saying that I have not yet looked at the code

But it seems to me that logically this code is going to have to be something
along the lines of 

gmtime(){
  get TZ variable
  load zone
  do conversion
}

Instead of all this being in one function, it should be pretty trivial to split
it to be

gmtime(){
  get TZ variable (existing code)
  *zone load_zone(*char tz)
  convert_to_gmt(*zone zone, time)
}
load_zone(){
  (existing code)
}
convert_to_gmt(){
  (existing code)
}

in other words this seems like it should be a code restructuring into helper
functions, and then exposing the helper functions to the application rather
than being significant new code to be written.

This isn't as nice an interface as convert_tz(*char from_tz, *char to_tz,
time), but it should be much easier to implement, and require far less
maintinance since this should be existing code

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[fweimer at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=11620

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
              Flags|                            |security-

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug libc/11620] Bad design of timezone conversions

[maxim.yegorushkin at gmail dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=11620

--- Comment #8 from Maxim Yegorushkin <maxim.yegorushkin at gmail dot com> ---
Upstream tz project exposes the functions required for easy and thread-save
timezone conversions, see
https://github.com/eggert/tz/blob/master/private.h#L409

    /*
    ** Define functions that are ABI compatible with NetBSD but have
    ** better prototypes.  NetBSD 6.1.4 defines a pointer type timezone_t
    ** and labors under the misconception that 'const timezone_t' is a
    ** pointer to a constant.  This use of 'const' is ineffective, so it
    ** is not done here.  What we call 'struct state' NetBSD calls
    ** 'struct __state', but this is a private name so it doesn't matter.
    */
    #if NETBSD_INSPIRED

    typedef struct state *timezone_t;
    struct tm *localtime_rz(timezone_t restrict, time_t const *restrict, struct
tm *restrict);
    time_t mktime_z(timezone_t restrict, struct tm *restrict);
    timezone_t tzalloc(char const *);
    void tzfree(timezone_t);

    /* ... */
    #endif

Would it be possible to expose these NETBSD_INSPIRED 4 functions in glibc?

-- Maxim

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug time/11620] Bad design of timezone conversions

[jsm28 at gcc dot gnu.org]

https://sourceware.org/bugzilla/show_bug.cgi?id=11620

Joseph Myers <jsm28 at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|libc                        |time

-- 
You are receiving this mail because:
You are on the CC list for the bug.