Re: [PATCH] libstdc++, configure: Fix GLIBCXX_ZONEINFO_DIR configuration macro.

[Jonathan Wakely <jwakely@redhat.com>]

On 23/12/22 17:06 +0000, Iain Sandoe wrote:
>This is a patch for comment on the approach - tested on x86_64-darwi21
> thoughts?
> Iain
>
> --- 8< ---
>
>Testing on Darwin revealed that the GLIBCXX_ZONEINFO_DIR was not doing quite
>the right thing (we ended up with ${withval} in the config.h file).
>
>This patch proposes revising the behaviour of the configure flag thus:
>
>--with-libstdcxx-zoneinfo-dir=
> unspecified : Set _GLIBCXX_ZONEINFO_DIR to a default suitable for $host
>         yes : Set _GLIBCXX_ZONEINFO_DIR to a default suitable for $host
>         no  : Do not set _GLIBCXX_ZONEINFO_DIR
> /some/path  : set _GLIBCXX_ZONEINFO_DIR = "/some/path"

Here's what I've pushed to trunk now, after discussion with Iain.

This bundles a copy of the tzdata.zi file with the libstdc++ sources.
This means we can provide fully functional std::chrono time zone
support on all targets (except freestanding). The only thing that
doesn't work now is the std::chrono::current_zone() function on AIX,
which cannot determine the current time zone, because AIX uses POSIX
time zone names in the $TZ env var, rather than an IANA name like
"Europe/London".

The bundles tzdata.zi file was include in this commit, but isn't
included in the patch below, because it's 105kB of boring data. If you
want to see it, it's in the repo now. The version committed is the
latest one from upstream, included in the tarball at
https://data.iana.org/time-zones/releases/tzdb-2022g.tar.lz
I will sync the copy in the libstdc++ sources with upstream as needed.
That should be done before a release from relevant branches (gcc-13
and later).

Packagers for Linux distros that ship the tzdata.zi and leapseconds
files might want to configure their gcc packages with
--with-libstdcxx-zoneinfo=/usr/share/zoneinfo (or similar path). This
applies to (at least) Debian, RHEL, Fedora, and their derivatives.
That will avoid bundling the 105kB file in the libstdc++.so.6 and
libstdc++.a libs on systems where the OS already provides the file
(and presumably already keeps it up to date when upstream release a
new version of the data).

Tested x86_64-linux, power-aix, sparc-solaris2.11. Iain has also
tested on darwin.

Pushed to trunk.

-- >8 --

commit 559993b85744ae09d33eedb1cb062392ac482f94
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Sat Jan 14 13:33:58 2023

     libstdc++: Embed a static copy of tzdata.zi
     
     This adds a copy of the tzdata.zi file to the library, and allows
     configuring to use it instead of a copy read from disk at runtime.
     The content of the file is in the public domain, but will need to be
     updated to the latest upstream file before making GCC releases.
     
     libstdc++-v3/ChangeLog:
     
             * acinclude.m4 (GLIBCXX_ZONEINFO_DIR): Replace the
             --with-libstdcxx-zoneinfo-dir configure option with
             --with-libstdcxx-zoneinfo with yes/no/static choices as well as
             a directory.
             * config.h.in: Regenerate.
             * configure: Regenerate.
             * doc/xml/manual/configure.xml: Document configure option.
             * doc/html/manual/configure.html: Regenerate.
             * src/c++20/Makefile.am: Generate tzdata.zi.h header.
             * src/c++20/Makefile.in: Regenerate.
             * src/c++20/tzdb.cc (__gnu_cxx::zoneinfo_dir_override): Return a
             null pointer if no directory is configured.
             (zoneinfo_dir): Replace with ...
             (zoneinfo_file): New function.
             (tzdata_stream): New istream class.
             (remote_version, reload_tzdb): Use tzdata_stream.
             * testsuite/lib/libstdc++.exp (check_effective_target_tzdb):
             Check new _GLIBCXX_STATIC_TZDATA macro and ignore presence of
             tzdata.zi file in default location.
             * src/c++20/tzdata.zi: New file.

diff --git a/libstdc++-v3/acinclude.m4 b/libstdc++-v3/acinclude.m4
index 51c3c510364..3840a9d761a 100644
--- a/libstdc++-v3/acinclude.m4
+++ b/libstdc++-v3/acinclude.m4
@@ -5157,28 +5157,87 @@ AC_DEFUN([GLIBCXX_EMERGENCY_EH_ALLOC], [
  dnl
  dnl Allow the location of tzdata files to be configured.
  dnl
-dnl --with-libstdcxx-zoneinfo-dir=PATH will set the directory to PATH.
+dnl --with-libstdcxx-zoneinfo=ARG where ARG can be:
+dnl   DIR - use DIR/tzdata.zi and DIR/leapseconds files.
+dnl   static - use static copy of tzdata.zi embedded in the library.
+dnl   DIR,static - use DIR, but use embedded static copy as fallback.
+dnl   yes - equivalent to DIR,static with a system-specific value for DIR.
+dnl   no - disable most tzdb functionality.
  dnl
  dnl Defines:
-dnl  _GLIBCXX_ZONEINFO_DIR if std::chrono::tzdb should use a non-default
+dnl  _GLIBCXX_ZONEINFO_DIR if std::chrono::tzdb should use the specified
  dnl    directory for the tzdata.zi and leapseconds files.
+dnl  _GLIBCXX_STATIC_TZDATA if std::chrono::tzdb should use an embedded
+dnl    static copy of the tzdata.zi file.
  dnl
  AC_DEFUN([GLIBCXX_ZONEINFO_DIR], [
-  AC_ARG_WITH([libstdcxx-zoneinfo-dir],
-    AC_HELP_STRING([--with-libstdcxx-zoneinfo-dir],
-		   [the directory to search for tzdata files]),
-    [zoneinfo_dir="${withval}"
-     AC_DEFINE(_GLIBCXX_ZONEINFO_DIR, "${withval}",
-       [Define if a non-default location should be used for tzdata files.])
-    ],
-    [
-    case "$host" in
-      # *-*-aix*) zoneinfo_dir="/usr/share/lib/zoneinfo" ;;
-      *) zoneinfo_dir="/usr/share/zoneinfo" ;;
-    esac
-    ])
+  AC_ARG_WITH([libstdcxx-zoneinfo],
+    AC_HELP_STRING([--with-libstdcxx-zoneinfo],
+		   [the location to use for tzdata]),
+    [],[with_libstdcxx_zoneinfo=yes])
  
+  if test "x${with_libstdcxx_zoneinfo}" = xyes; then
+    # Pick a default when no specific path is set.
+    case "$host" in
+      gnu* | linux* | kfreebsd*-gnu | knetbsd*-gnu)
+	# Not all distros ship tzdata.zi in this dir.
+	zoneinfo_dir="/usr/share/zoneinfo"
+	;;
+      *-*-aix*)
+	# Binary tzfile files are in /usr/share/lib/zoneinfo
+	# but tzdata.zi is not present there.
+	zoneinfo_dir=none
+	;;
+      *-*-darwin2*)
+	# Binary tzfile files are in /usr/share/lib/zoneinfo.default
+	# but tzdata.zi is not present there.
+	zoneinfo_dir=none
+	;;
+      *)
+	# Binary tzfile files are commonly found in /usr/share/zoneinfo
+	# but tzdata.zi is not present there.
+	zoneinfo_dir=none
+	;;
+    esac
+    # Also embed a copy of the tzdata.zi file as a static string.
+    embed_zoneinfo=yes
+  elif test "x${with_libstdcxx_zoneinfo}" = xno; then
+    # Disable tzdb support completely.
+    zoneinfo_dir=none
+    embed_zoneinfo=no
+  else
+    case "${with_libstdcxx_zoneinfo}" in
+      static)
+	# Do not attempt to read from disk, always use embedded data.
+	zoneinfo_dir=none
+	embed_zoneinfo=yes
+	;;
+      static,* | *,static)
+	# Try to read from disk, use embedded data as fallback.
+	zoneinfo_dir="${with_libstdcxx_zoneinfo#static,}"
+	zoneinfo_dir="${with_libstdcxx_zoneinfo%,static}"
+	embed_zoneinfo=yes
+	;;
+      *)
+	zoneinfo_dir="${with_libstdcxx_zoneinfo}"
+	embed_zoneinfo=no
+	;;
+    esac
+  fi
    AC_MSG_NOTICE([zoneinfo data directory: ${zoneinfo_dir}])
+  if test "x${zoneinfo_dir}" != xnone; then
+    AC_DEFINE_UNQUOTED(_GLIBCXX_ZONEINFO_DIR, "${zoneinfo_dir}",
+      [Define if a directory should be searched for tzdata files.])
+    if $GLIBCXX_IS_NATIVE -a ! test -f "$zoneinfo_dir/tzdata.zi"; then
+      AC_MSG_WARN("$zoneinfo_dir does not contain tzdata.zi file")
+    fi
+  fi
+  GLIBCXX_CONDITIONAL(USE_STATIC_TZDATA, test "${embed_zoneinfo}" = yes)
+  if test "x${embed_zoneinfo}" = xyes; then
+    AC_MSG_NOTICE([static tzdata.zi file will be compiled into the library])
+    AC_DEFINE_UNQUOTED(_GLIBCXX_STATIC_TZDATA, 1,
+      [Define if static tzdata should be compiled into the library.])
+  fi
  ])
  
  # Macros from the top-level gcc directory.
diff --git a/libstdc++-v3/doc/xml/manual/configure.xml b/libstdc++-v3/doc/xml/manual/configure.xml
index c6c5981968d..8b3b8cab8c7 100644
--- a/libstdc++-v3/doc/xml/manual/configure.xml
+++ b/libstdc++-v3/doc/xml/manual/configure.xml
@@ -469,6 +469,33 @@
      </para>
   </listitem></varlistentry>
  
+ <varlistentry><term><code>--with-libstdcxx-zoneinfo=OPTION</code></term>
+ <listitem>
+    <para>Choose how <classname>std::chrono::tzdb</classname> will obtain
+      the time zone info. The library requires a copy of the
+      <filename>tzdata.zi</filename> and <filename>leapseconds</filename>
+      files from the <link xmlns:xlink="http://www.w3.org/1999/xlink"
+	xlink:href="https://www.iana.org/time-zones">IANA Time Zone
+      Database</link>. The choice OPTION=static will embed a copy of the files
+      into the library, and use that static data when time zone information
+      is required. The choice OPTION=dir will use the files
+      <filename>dir/tzdata.zi</filename> and
+      <filename>dir/leapseconds</filename> (which must exist when a program
+      tries to access time zone information). The choice OPTION=dir,static
+      will try to use files in <filename>dir</filename> but if they are
+      not available the embedded static data will be used instead.
+      The default choice is OPTION=yes. This is equivalent to OPTION=dir,static
+      with a system-specific default directory (if a suitable default for
+      the target is known).
+      The choice OPTION=no will disable all code for loading time zone info
+      from file or from the embedded static data, which means that only the
+      "UTC" and "GMT" time zones are defined. Using OPTION=no results in a
+      smaller library, so is suitable for systems that will never need to
+      query the time zone database.
+      This does not change the library ABI.
+    </para>
+ </listitem></varlistentry>
+
  </variablelist>
  
  </section>
diff --git a/libstdc++-v3/src/c++20/Makefile.am b/libstdc++-v3/src/c++20/Makefile.am
index a95b8c24d21..5ae4145bb4a 100644
--- a/libstdc++-v3/src/c++20/Makefile.am
+++ b/libstdc++-v3/src/c++20/Makefile.am
@@ -40,6 +40,19 @@ sources = tzdb.cc
  
  vpath % $(top_srcdir)/src/c++20
  
+if USE_STATIC_TZDATA
+tzdata.zi.h: $(top_srcdir)/src/c++20/tzdata.zi
+	echo 'static const char tzdata_chars[] = R"__libstdcxx__(' > $@.tmp
+	cat $^ >> $@.tmp
+	echo ')__libstdcxx__";' >> $@.tmp
+	mv $@.tmp $@
+
+tzdb.lo: tzdb.cc tzdata.zi.h
+	$(LTCXXCOMPILE) -I. -c $<
+tzdb.o: tzdb.cc tzdata.zi.h
+	$(CXXCOMPILE) -I. -c $<
+endif
+
  libc__20convenience_la_SOURCES = $(sources)  $(inst_sources)
  
  # AM_CXXFLAGS needs to be in each subdirectory so that it can be
diff --git a/libstdc++-v3/src/c++20/tzdb.cc b/libstdc++-v3/src/c++20/tzdb.cc
index e335ea61c4d..94fb0436091 100644
--- a/libstdc++-v3/src/c++20/tzdb.cc
+++ b/libstdc++-v3/src/c++20/tzdb.cc
@@ -60,10 +60,6 @@
  # endif
  #endif
  
-#ifndef _GLIBCXX_ZONEINFO_DIR
-# define _GLIBCXX_ZONEINFO_DIR "/usr/share/zoneinfo"
-#endif
-
  namespace __gnu_cxx
  {
  #ifdef _AIX
@@ -75,7 +71,13 @@ namespace __gnu_cxx
  #if defined(__APPLE__) || defined(__hpux__)
    // Need a weak definition for Mach-O.
    [[gnu::weak]] const char* zoneinfo_dir_override()
-  { return _GLIBCXX_ZONEINFO_DIR; }
+  {
+#ifdef _GLIBCXX_ZONEINFO_DIR
+    return _GLIBCXX_ZONEINFO_DIR;
+#else
+    return nullptr;
+#endif
+  }
  #endif
  #endif
  }
@@ -1008,22 +1010,83 @@ namespace std::chrono
      return info;
    }
  
- namespace
- {
+  namespace
+  {
+    // If a zoneinfo directory is defined (either when the library was built,
+    // or via the zoneinfo_dir_override function) then append filename to it.
+    // The filename should have a leading '/' as one is not added explicitly.
      string
-    zoneinfo_dir()
+    zoneinfo_file(string_view filename)
      {
+      string path;
  #pragma GCC diagnostic push
  #pragma GCC diagnostic ignored "-Waddress"
-      static const string dir = __gnu_cxx::zoneinfo_dir_override
-				  ? __gnu_cxx::zoneinfo_dir_override()
-				  : _GLIBCXX_ZONEINFO_DIR;
+      if (__gnu_cxx::zoneinfo_dir_override)
+      {
+	if (auto override_dir = __gnu_cxx::zoneinfo_dir_override())
+	  path = override_dir;
  #pragma GCC diagnostic pop
-      return dir;
+      }
+#ifdef _GLIBCXX_ZONEINFO_DIR
+      else
+	path = _GLIBCXX_ZONEINFO_DIR;
+#endif
+      if (!path.empty())
+	path.append(filename);
+      return path;
      }
  
+    // N.B. Leading slash as required by zoneinfo_file function.
      const string tzdata_file = "/tzdata.zi";
      const string leaps_file = "/leapseconds";
+
+#ifdef _GLIBCXX_STATIC_TZDATA
+// Static copy of tzdata.zi embedded in the library as tzdata_chars[]
+#include "tzdata.zi.h"
+#endif
+
+    // An istream type that can read from a file or from a string.
+    struct tzdata_stream : istream
+    {
+      // std::spanbuf not available until C++23
+      struct ispanbuf : streambuf
+      {
+	ispanbuf() : streambuf()
+	{
+#ifdef _GLIBCXX_STATIC_TZDATA
+	  char* p = const_cast<char*>(tzdata_chars);
+	  this->setg(p, p, p + std::size(tzdata_chars) - 1);
+#endif
+	}
+
+	// N.B. seekoff and seekpos not overridden, not currently needed.
+      };
+
+      union {
+	filebuf fb;
+	ispanbuf sb;
+      };
+
+      tzdata_stream() : istream(nullptr)
+      {
+	if (string path = zoneinfo_file("/tzdata.zi"); !path.empty())
+	{
+	  filebuf fbuf;
+	  if (fbuf.open(path, std::ios::in))
+	    {
+	      std::construct_at(&fb, std::move(fbuf));
+	      this->init(&fb);
+	      return;
+	    }
+	}
+	std::construct_at(&sb);
+	this->init(&sb);
+      }
+
+      ~tzdata_stream() { std::destroy_at(this->rdbuf()); } // use virtual dtor
+
+      bool using_static_data() const { return this->rdbuf() == &sb; }
+    };
    }
  
    // Return leap_second values, and a bool indicating whether the values are
@@ -1031,7 +1094,7 @@ namespace std::chrono
    pair<vector<leap_second>, bool>
    tzdb_list::_Node::_S_read_leap_seconds()
    {
-    const string filename = zoneinfo_dir() + leaps_file;
+    const string filename = zoneinfo_file(leaps_file);
  
      // This list is valid until at least 2023-06-28 00:00:00 UTC.
      auto expires = sys_days{2023y/6/28};
@@ -1127,31 +1190,25 @@ namespace std::chrono
    namespace
    {
      // Read the version number from a tzdata.zi file.
-    // Note that some systems do not have this file available by default
-    // but we can configure the library to point to an alternate installation.
      string
-    remote_version(istream* zif)
+    remote_version(istream& zif)
      {
-      ifstream f;
-      if (!zif)
-	{
-	  f.open(zoneinfo_dir() + tzdata_file);
-	  zif = &f;
-	}
        char hash;
        string label;
        string version;
-      if (*zif >> hash >> label >> version)
+      if (zif >> hash >> label >> version)
  	if (hash == '#' && label == "version")
  	  return version;
+#if 0 // Ignore these files, because we're not using them anyway.
  #if defined __NetBSD__
-      if (string ver; ifstream(zoneinfo_dir() + "/TZDATA_VERSION") >> ver)
+      if (string ver; ifstream(zoneinfo_file("/TZDATA_VERSION")) >> ver)
  	return ver;
  #elif defined __APPLE__
        // The standard install on macOS has no tzdata.zi, but we can find the
        // version from +VERSION.
-      if (string ver; ifstream(zoneinfo_dir() + "/+VERSION") >> ver)
+      if (string ver; ifstream(zoneinfo_file("/+VERSION")) >> ver)
  	return ver;
+#endif
  #endif
        __throw_runtime_error("tzdb: no version found in tzdata.zi");
      }
@@ -1160,7 +1217,8 @@ namespace std::chrono
    // Definition of std::chrono::remote_version()
    string remote_version()
    {
-    return remote_version(nullptr);
+    tzdata_stream zif;
+    return remote_version(zif);
    }
  
    // Used by chrono::reload_tzdb() to add a new node to the front of the list.
@@ -1293,8 +1351,8 @@ namespace std::chrono
    {
      using Node = tzdb_list::_Node;
  
-    ifstream zif(zoneinfo_dir() + tzdata_file);
-    const string version = remote_version(&zif);
+    tzdata_stream zif;
+    const string version = remote_version(zif);
  
  #if USE_ATOMIC_SHARED_PTR
      auto head = Node::_S_head_owner.load(memory_order::acquire);
@@ -1311,7 +1369,7 @@ namespace std::chrono
  #endif
  
      auto [leaps, leaps_ok] = Node::_S_read_leap_seconds();
-    if (!leaps_ok)
+    if (!leaps_ok && !zif.using_static_data())
        __throw_runtime_error("tzdb: cannot parse leapseconds file");
  
      auto node = std::make_shared<Node>();
diff --git a/libstdc++-v3/testsuite/lib/libstdc++.exp b/libstdc++-v3/testsuite/lib/libstdc++.exp
index 0a484e66379..843497c0a80 100644
--- a/libstdc++-v3/testsuite/lib/libstdc++.exp
+++ b/libstdc++-v3/testsuite/lib/libstdc++.exp
@@ -1415,11 +1415,8 @@ proc check_effective_target_tzdb { } {
  	return 0
      }
      return [check_v3_target_prop_cached et_tzdb {
-	set cond "defined _GLIBCXX_ZONEINFO_DIR"
-	if {[v3_check_preprocessor_condition tzdb $cond]} {
-	    return 1
-	}
-	return [file exists /usr/share/zoneinfo/tzdata.zi]
+	set cond "defined _GLIBCXX_ZONEINFO_DIR || _GLIBCXX_STATIC_TZDATA"
+	return [v3_check_preprocessor_condition tzdb $cond]
      }]
  }