require et random_device for cons token test

require et random_device for cons token test

[Alexandre Oliva]

On target systems that don't support any random_device, not even the
default one, other random_device constructor tests are disabled by
dg-require-effective-target random_device.  The token.cc also
exercises the default constructor, in a way that doesn't expect an
exception to be raised, but it's not guarded by the same requirement.

Other potentially-raising ctors in token.cc expect exceptions and
handle them, but the ("default")-constructed one does not, so the
program terminates and the test fails without exercising the other
constructor variants.

This patch arranges to disable the test altogether when the
random_device feature is not available.  A reasonable alternative
would be to install a std::runtime_error handler around the test01
body, so that we exercise at least the exception raising, but then
test03 would have to be relaxed, since without even "default", it
likely wouldn't meet the tested requirement there.

Regstrapped on x86_64-linux-gnu and cross-tested for x86_64-vx7r2 along
with other patches, mostly for the testsuite.  Ok to install?


for  libstdc++-v3/ChangeLog

	* testsuite/26_numerics/random/random_device/cons/token.cc:
	Require effective target feature random_device.
---
 .../26_numerics/random/random_device/cons/token.cc |    1 +
 1 file changed, 1 insertion(+)

diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
index defb8d58c586a..105ae0ba87743 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
@@ -1,4 +1,5 @@
 // { dg-do run { target c++11 } }
+// { dg-require-effective-target random_device }
 // { dg-require-cstdint "" }
 //
 // 2008-11-24  Edward M. Smith-Rowland <3dw4rd@verizon.net>


-- 
Alexandre Oliva, happy hacker  https://FSFLA.org/blogs/lxo/
   Free Software Activist         GNU Toolchain Engineer
        Vim, Vi, Voltei pro Emacs -- GNUlius Caesar

Re: require et random_device for cons token test

[Jonathan Wakely]

On 24/03/21 03:53 -0300, Alexandre Oliva wrote:
>
>On target systems that don't support any random_device, not even the
>default one,

It should be impossible to have no random_device. As a fallback a
pseudo random number generator should be used.

>other random_device constructor tests are disabled by
>dg-require-effective-target random_device.  The token.cc also
>exercises the default constructor, in a way that doesn't expect an
>exception to be raised, but it's not guarded by the same requirement.
>
>Other potentially-raising ctors in token.cc expect exceptions and
>handle them, but the ("default")-constructed one does not, so the
>program terminates and the test fails without exercising the other
>constructor variants.

Why does the "default" token cause an exception?

>This patch arranges to disable the test altogether when the
>random_device feature is not available.  A reasonable alternative
>would be to install a std::runtime_error handler around the test01
>body, so that we exercise at least the exception raising, but then
>test03 would have to be relaxed, since without even "default", it
>likely wouldn't meet the tested requirement there.
>
>Regstrapped on x86_64-linux-gnu and cross-tested for x86_64-vx7r2 along
>with other patches, mostly for the testsuite.  Ok to install?

No, that would disable the test for Windows, where it works OK.

The 'random_device' effective-target is poorly named, it checks
whether the _GLIBCXX_USE_RANDOM_TR1 macro is defined, which is even
more poorly named. That macro is defined when /dev/random and
/dev/urandom are available. But we support std::random_device without
those files (e.g. on Windows).

If the default constructor throws then that suggests your target is
misconfigured. Why isn't the mt19937 PRNG being used?


>for  libstdc++-v3/ChangeLog
>
>	* testsuite/26_numerics/random/random_device/cons/token.cc:
>	Require effective target feature random_device.
>---
> .../26_numerics/random/random_device/cons/token.cc |    1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
>index defb8d58c586a..105ae0ba87743 100644
>--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
>+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
>@@ -1,4 +1,5 @@
> // { dg-do run { target c++11 } }
>+// { dg-require-effective-target random_device }
> // { dg-require-cstdint "" }
> //
> // 2008-11-24  Edward M. Smith-Rowland <3dw4rd@verizon.net>
>
>
>-- 
>Alexandre Oliva, happy hacker  https://FSFLA.org/blogs/lxo/
>   Free Software Activist         GNU Toolchain Engineer
>        Vim, Vi, Voltei pro Emacs -- GNUlius Caesar
>

Re: require et random_device for cons token test

[Alexandre Oliva]

On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:

> It should be impossible to have no random_device. As a fallback a
> pseudo random number generator should be used.

> If the default constructor throws then that suggests your target is
> misconfigured. Why isn't the mt19937 PRNG being used?

This is an x86_64-vx7r2 target, it has USE_RDRAND and USE_RDSEED both
enabled as the only RAND backends.  AFAICT both of them fail their cpuid
tests, so _M_init falls through to the default, and throws.

I suppose we need to cover the case in which all of the compile-time
presumed-available random backends turn out to not be available at
run-time, and introduce an MT19937 dynamic fallback in the following
default: block, no?

      default:
      { }
    }
    std::__throw_runtime_error(
	__N("random_device::random_device(const std::string&):"
	    " device not available"));


> The 'random_device' effective-target is poorly named,

I see, thanks, I didn't think of checking its definition.


-- 
Alexandre Oliva, happy hacker  https://FSFLA.org/blogs/lxo/
   Free Software Activist         GNU Toolchain Engineer
        Vim, Vi, Voltei pro Emacs -- GNUlius Caesar

Re: require et random_device for cons token test

[Jonathan Wakely]

On 24/03/21 07:33 -0300, Alexandre Oliva wrote:
>On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
>
>> It should be impossible to have no random_device. As a fallback a
>> pseudo random number generator should be used.
>
>> If the default constructor throws then that suggests your target is
>> misconfigured. Why isn't the mt19937 PRNG being used?
>
>This is an x86_64-vx7r2 target, it has USE_RDRAND and USE_RDSEED both
>enabled as the only RAND backends.  AFAICT both of them fail their cpuid
>tests, so _M_init falls through to the default, and throws.
>
>I suppose we need to cover the case in which all of the compile-time
>presumed-available random backends turn out to not be available at
>run-time, and introduce an MT19937 dynamic fallback in the following
>default: block, no?

Hmm, that would be tricky because it's currently a static decision
made during preprocessing. We have no disciminator to tell us at
runtime that the _M_mt member of the union is active:

     union
     {
       struct
       {
	void*      _M_file;
	result_type (*_M_func)(void*);
	int _M_fd;
       };
       mt19937    _M_mt;
     };

Does vxworks provide any platform-specific source of randomness, like
Linux getrandom(2) or BSD arc4random(3) or Windows rand_s? If yes, we
should add support for that (in the next stage 1).

But even if it does, we should still have a dynamic fallback at
runtime. I have a patch coming to do that ...

Re: require et random_device for cons token test

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 2148 bytes --]

On 24/03/21 11:27 +0000, Jonathan Wakely wrote:
>On 24/03/21 07:33 -0300, Alexandre Oliva wrote:
>>On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
>>
>>>It should be impossible to have no random_device. As a fallback a
>>>pseudo random number generator should be used.
>>
>>>If the default constructor throws then that suggests your target is
>>>misconfigured. Why isn't the mt19937 PRNG being used?
>>
>>This is an x86_64-vx7r2 target, it has USE_RDRAND and USE_RDSEED both
>>enabled as the only RAND backends.  AFAICT both of them fail their cpuid
>>tests, so _M_init falls through to the default, and throws.
>>
>>I suppose we need to cover the case in which all of the compile-time
>>presumed-available random backends turn out to not be available at
>>run-time, and introduce an MT19937 dynamic fallback in the following
>>default: block, no?
>
>Hmm, that would be tricky because it's currently a static decision
>made during preprocessing. We have no disciminator to tell us at
>runtime that the _M_mt member of the union is active:
>
>    union
>    {
>      struct
>      {
>	void*      _M_file;
>	result_type (*_M_func)(void*);
>	int _M_fd;
>      };
>      mt19937    _M_mt;
>    };
>
>Does vxworks provide any platform-specific source of randomness, like
>Linux getrandom(2) or BSD arc4random(3) or Windows rand_s? If yes, we
>should add support for that (in the next stage 1).
>
>But even if it does, we should still have a dynamic fallback at
>runtime. I have a patch coming to do that ...

Attached (the patch is generated with -b to ignore changes in
whitespace, for ease of reading the diff).

This works for me on x86_64-linux and powerpc64le-linux, and also on
x86_64-linux when I kluge the config macros so that the new code path
gets used. Does this work for VxWorks?

The problem this fixes is a regression. Before GCC 10 VxWorks would
have always used the mt19937 code. Now it tries to use rdrand/rdseed
instructions and fails if the hardware doesn't have them. This patch
should make it work again (using a different PRNG, and a crappy seed
but that's still better than mt19937 with the same seed every time).



[-- Attachment #2: patch.txt --]
[-- Type: text/x-patch, Size: 13122 bytes --]

commit e18187c00b0a65f3b5ab0dc340e78516d5a4f44f
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Wed Mar 24 13:53:22 2021

    libstdc++: Add PRNG fallback to std::random_device
    
    This makes std::random_device usable on VxWorks when running on older
    x86 hardware. Since the r10-728 fix for PR libstdc++/85494 the library
    will use the new code unconditionally on x86, but the cpuid checks for
    RDSEED and RDRAND can fail at runtime, depending on the hardware where
    the code is executing. If the OS does not provide /dev/random then this
    means the std::random_device constructor always fails. In previous
    releases if /dev/random is unavailable then std::mt19937 was used
    unconditionally.
    
    This patch adds a fallback for the case where the runtime cpuid checks
    for x86 hardware instructions fail, and no /dev/random is available.
    When this happens a std::linear_congruential_engine object will be
    used, with a (not very good) seed. This at least
    
    libstdc++-v3/ChangeLog:
    
            * src/c++11/random.cc (USE_LCG): Define when a pseudo-random
            fallback is needed.
            [USE_LCG] (lcg_type, __lcg): New typedef and callback.
            (random_device::_M_init): Add 'prng' and 'all' enumerators.
            Replace switch with fallthrough with a series of 'if' statements.
            [USE_LCG]: Construct an lcg_type engine and use __lcg when cpuid
            checks fail.
            (random_device::_M_init_pretr1) [USE_MT19937]: Accept "prng"
            token.
            (random_device::_M_getval): Check for callback unconditionally
            and always pass _M_file pointer.
            * testsuite/26_numerics/random/random_device/85494.cc: Remove
            effective-target check. Use new random_device_available helper.
            * testsuite/26_numerics/random/random_device/94087.cc: Likewise.
            * testsuite/26_numerics/random/random_device/cons/default-cow.cc:
            Remove effective-target check.
            * testsuite/26_numerics/random/random_device/cons/default.cc:
            Likewise.
            * testsuite/26_numerics/random/random_device/cons/token.cc: Use
            new random_device_available helper. Test "prng" token.
            * testsuite/util/testsuite_random.h (random_device_available):
            New helper function.

diff --git a/libstdc++-v3/src/c++11/random.cc b/libstdc++-v3/src/c++11/random.cc
index 1092299e56d..6b9b40c684a 100644
--- a/libstdc++-v3/src/c++11/random.cc
+++ b/libstdc++-v3/src/c++11/random.cc
@@ -42,6 +42,7 @@
 #include <cerrno>
 #include <cstdio>
 #include <cctype> // For std::isdigit.
+#include <ctime>  // For std::time.
 
 #if defined _GLIBCXX_HAVE_UNISTD_H && defined _GLIBCXX_HAVE_FCNTL_H
 # include <unistd.h>
@@ -66,9 +67,14 @@
 # include <stdlib.h>
 #endif
 
-#if defined USE_RDRAND || defined USE_RDSEED \
-  || defined _GLIBCXX_USE_CRT_RAND_S || defined _GLIBCXX_USE_DEV_RANDOM
+#if defined _GLIBCXX_USE_CRT_RAND_S || defined _GLIBCXX_USE_DEV_RANDOM
+// The OS provides a source of randomness we can use.
 # pragma GCC poison _M_mt
+#elif defined USE_RDRAND || defined USE_RDSEED
+// Hardware instructions might be available, but use cpuid checks at runtime.
+# pragma GCC poison _M_mt
+// If the runtime cpuid checks fail we'll use a linear congruential engine.
+# define USE_LCG 1
 #else
 // Use the mt19937 member of the union, as in previous GCC releases.
 # define USE_MT19937 1
@@ -136,6 +142,19 @@ namespace std _GLIBCXX_VISIBILITY(default)
       return val;
     }
 #endif
+
+#if USE_LCG
+    // Same as std::minstd_rand0 but using unsigned not uint_fast32_t.
+    using lcg_type
+      = std::linear_congruential_engine<unsigned, 16807UL, 0UL, 2147483647UL>;
+
+    unsigned int
+    __lcg(void* ptr)
+    {
+      auto& lcg = *static_cast<lcg_type*>(ptr);
+      return lcg();
+    }
+#endif
   }
 
   void
@@ -152,25 +171,16 @@ namespace std _GLIBCXX_VISIBILITY(default)
     _M_fd = -1;
 
     const char* fname [[gnu::unused]] = nullptr;
-    bool default_token [[gnu::unused]] = false;
 
-    enum { rand_s, rdseed, rdrand, device_file } which;
+    enum {
+	rand_s = 1, rdseed = 2, rdrand = 4, device_file = 8, prng = 16,
+	any = 0xffff
+    } which;
 
     if (token == "default")
       {
-	default_token = true;
+	which = any;
 	fname = "/dev/urandom";
-#if defined _GLIBCXX_USE_CRT_RAND_S
-	which = rand_s;
-#elif defined USE_RDSEED
-	which = rdseed;
-#elif defined USE_RDRAND
-	which = rdrand;
-#elif defined _GLIBCXX_USE_DEV_RANDOM
-	which = device_file;
-#else
-# error "either define USE_MT19937 above or set the default device here"
-#endif
       }
 #ifdef USE_RDSEED
     else if (token == "rdseed")
@@ -191,22 +201,25 @@ namespace std _GLIBCXX_VISIBILITY(default)
 	which = device_file;
       }
 #endif // _GLIBCXX_USE_DEV_RANDOM
+#ifdef USE_LCG
+    else if (token == "prng")
+      which = prng;
+#endif
     else
       std::__throw_runtime_error(
 	  __N("random_device::random_device(const std::string&):"
 	      " unsupported token"));
 
-    switch (which)
-    {
 #ifdef _GLIBCXX_USE_CRT_RAND_S
-      case rand_s:
+    if (which | rand_s)
     {
       _M_func = &__winxp_rand_s;
       return;
     }
 #endif // _GLIBCXX_USE_CRT_RAND_S
+
 #ifdef USE_RDSEED
-      case rdseed:
+    if (which | rdseed)
     {
       unsigned int eax, ebx, ecx, edx;
       // Check availability of cpuid and, for now at least, also the
@@ -231,15 +244,11 @@ namespace std _GLIBCXX_VISIBILITY(default)
 	      return;
 	    }
 	}
-	// If rdseed was explicitly requested then we're done here.
-	if (!default_token)
-	  break;
-	// Otherwise fall through to try the next available option.
-	[[gnu::fallthrough]];
     }
 #endif // USE_RDSEED
+
 #ifdef USE_RDRAND
-      case rdrand:
+    if (which | rdrand)
     {
       unsigned int eax, ebx, ecx, edx;
       // Check availability of cpuid and, for now at least, also the
@@ -255,15 +264,11 @@ namespace std _GLIBCXX_VISIBILITY(default)
 	      return;
 	    }
 	}
-	// If rdrand was explicitly requested then we're done here.
-	if (!default_token)
-	  break;
-	// Otherwise fall through to try the next available option.
-	[[gnu::fallthrough]];
     }
 #endif // USE_RDRAND
+
 #ifdef _GLIBCXX_USE_DEV_RANDOM
-      case device_file:
+    if (which | device_file)
     {
 #ifdef USE_POSIX_FILE_IO
       _M_fd = ::open(fname, O_RDONLY);
@@ -278,12 +283,25 @@ namespace std _GLIBCXX_VISIBILITY(default)
       if (_M_file)
 	return;
 #endif // USE_POSIX_FILE_IO
-	[[gnu::fallthrough]];
     }
 #endif // _GLIBCXX_USE_DEV_RANDOM
-      default:
-      { }
+
+#ifdef USE_LCG
+    // Either "prng" was requested explicitly, or "default" was requested
+    // but nothing above worked. Use PRNG with lame seed. This should be
+    // the last option, so we try everything else first for "default".
+    if (which | prng)
+    {
+      static_assert(sizeof(lcg_type) <= sizeof(_M_fd), "");
+      static_assert(alignof(lcg_type) <= alignof(_M_fd), "");
+      unsigned seed = reinterpret_cast<uintptr_t>(this) >> 2;
+      seed ^= static_cast<unsigned>(std::time(nullptr));
+      _M_file = ::new(&_M_fd) lcg_type(seed);
+      _M_func = &__lcg;
+      return;
     }
+#endif
+
     std::__throw_runtime_error(
 	__N("random_device::random_device(const std::string&):"
 	    " device not available"));
@@ -297,7 +315,7 @@ namespace std _GLIBCXX_VISIBILITY(default)
   {
 #ifdef USE_MT19937
     unsigned long seed = 5489UL;
-    if (token != "default" && token != "mt19937")
+    if (token != "default" && token != "mt19937" && token != "prng")
       {
 	const char* nptr = token.c_str();
 	char* endptr;
@@ -351,10 +369,8 @@ namespace std _GLIBCXX_VISIBILITY(default)
     return _M_mt();
 #else
 
-#if defined USE_RDRAND || defined USE_RDSEED || defined _GLIBCXX_USE_CRT_RAND_S
     if (_M_func)
-      return _M_func(nullptr);
-#endif
+      return _M_func(_M_file);
 
     result_type ret;
     void* p = &ret;
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc
index 8bfb1fa71bf..80cb912b587 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc
@@ -16,14 +16,21 @@
 // <http://www.gnu.org/licenses/>.
 
 // { dg-do run { target c++11 } }
-// { dg-require-effective-target random_device }
 
 #include <random>
 #include <testsuite_hooks.h>
+#include <testsuite_random.h>
 
 void
 test01()
 {
+  if (__gnu_test::random_device_available("mt19937"))
+  {
+    // std::random_device uses a Mersenne Twister with default seed,
+    // and the test below will fail.  No point trying to test it.
+    return;
+  }
+
   unsigned v1[3], v2[3];
   std::random_device d1, d2;
   for (auto& v : v1)
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc
index c77917fb680..77bb7fa2d54 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc
@@ -25,18 +25,9 @@
 #include <memory>
 #include <thread>
 #include <cstdio>
+#include <testsuite_random.h>
 
-bool
-random_device_available(const char* token) noexcept
-{
-  try {
-    std::random_device dev(token);
-    return true;
-  } catch (...) {
-    std::printf("random_device(\"%s\") not available\n", token);
-    return false;
-  }
-}
+using __gnu_test::random_device_available;
 
 void read_random_device(const char* token, int iterations)
 {
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc
index ef35773e009..489a0a0965f 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc
@@ -1,6 +1,5 @@
 // { dg-options "-D_GLIBCXX_USE_CXX11_ABI=0" }
 // { dg-do run { target c++11 } }
-// { dg-require-effective-target random_device }
 // { dg-require-cstdint "" }
 //
 // Copyright (C) 2019-2021 Free Software Foundation, Inc.
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc
index 72756e245d5..79e044ece42 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc
@@ -1,5 +1,4 @@
 // { dg-do run { target c++11 } }
-// { dg-require-effective-target random_device }
 // { dg-require-cstdint "" }
 //
 // 2008-11-24  Edward M. Smith-Rowland <3dw4rd@verizon.net>
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
index defb8d58c58..aeb7403e830 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
@@ -25,6 +25,7 @@
 #include <random>
 #include <stdexcept>
 #include <testsuite_hooks.h>
+#include <testsuite_random.h>
 
 void
 test01()
@@ -50,41 +51,27 @@ test03()
 {
   // At least one of these tokens should be valid.
   const std::string tokens[] = {
-    "rdseed", "rdrand", "rand_s", "/dev/urandom", "/dev/random", "mt19937"
+    "rdseed", "rdrand", "rand_s", "/dev/urandom", "/dev/random", "mt19937",
+    "prng"
   };
   int count = 0;
   for (const std::string& token : tokens)
   {
-    try
-    {
-      std::random_device x(token);
+    if (__gnu_test::random_device_available(token))
       ++count;
   }
-    catch (const std::runtime_error&)
-    {
-    }
-  }
   VERIFY( count != 0 );
 }
 
 void
 test04()
 {
-  bool can_use_mt19937 = true;
-  std::random_device::result_type xval;
-  try
+  if (__gnu_test::random_device_available("mt19937"))
   {
     std::random_device x("mt19937");
-    xval = x();
-  }
-  catch (const std::runtime_error&)
-  {
-    can_use_mt19937 = false;
-  }
+    std::random_device::result_type xval = x();
 
     // If "mt19937" is a valid token then numeric seeds should be too.
-  if (can_use_mt19937)
-  {
     std::random_device x1("0");
     std::random_device x2("1234");
     std::random_device x3("0xc0fefe");
diff --git a/libstdc++-v3/testsuite/util/testsuite_random.h b/libstdc++-v3/testsuite/util/testsuite_random.h
index 0b670bfb771..c8323078492 100644
--- a/libstdc++-v3/testsuite/util/testsuite_random.h
+++ b/libstdc++-v3/testsuite/util/testsuite_random.h
@@ -197,6 +197,19 @@ namespace __gnu_test
   }
 #endif
 
+  // Check whether TOKEN can construct a std::random_device successfully.
+  inline bool
+  random_device_available(const std::string& token) noexcept
+  {
+    try {
+      std::random_device dev(token);
+      return true;
+    } catch (...) {
+      std::printf("random_device(\"%s\") not available\n", token);
+      return false;
+    }
+  }
+
 } // namespace __gnu_test
 
 #endif // #ifndef _GLIBCXX_TESTSUITE_RANDOM_H

Re: require et random_device for cons token test

[Alexandre Oliva]

On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:

> This works for me on x86_64-linux and powerpc64le-linux, and also on
> x86_64-linux when I kluge the config macros so that the new code path
> gets used. Does this work for VxWorks?

Thanks.  I (trivially) backported it to apply on our gcc-10 tree, and
tested that on x86_64-vx7r2, and I confirm it works there too.

However, I suspect there's a series of typos in the patch.  You appear
to be using the 'which' enum variable for bit testing, but with '|'
rather than '&'.

Unless I'm missing something in my reading of the modified code, this
may cause a backend different from that requested by the token to be
selected, but it doesn't look like we have any test that detects this
problem.

-- 
Alexandre Oliva, happy hacker  https://FSFLA.org/blogs/lxo/
   Free Software Activist         GNU Toolchain Engineer
        Vim, Vi, Voltei pro Emacs -- GNUlius Caesar

Re: require et random_device for cons token test

[Jonathan Wakely]

On 25/03/21 07:17 -0300, Alexandre Oliva wrote:
>On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
>
>> This works for me on x86_64-linux and powerpc64le-linux, and also on
>> x86_64-linux when I kluge the config macros so that the new code path
>> gets used. Does this work for VxWorks?
>
>Thanks.  I (trivially) backported it to apply on our gcc-10 tree, and
>tested that on x86_64-vx7r2, and I confirm it works there too.
>
>However, I suspect there's a series of typos in the patch.  You appear
>to be using the 'which' enum variable for bit testing, but with '|'
>rather than '&'.

Oops, that's what I get for a last-minute rewrite without proper
testing. I originally had:

   if (which == blah || which == any)

and then borked it in an attempt to use & instead.

I'll fix that locally too.

>Unless I'm missing something in my reading of the modified code, this
>may cause a backend different from that requested by the token to be
>selected, but it doesn't look like we have any test that detects this
>problem.

I don't see how it's possible to detect, without something like a
modified system that replaces /dev/random with a non-random source so
you can verify that std::random_device("/dev/random") and
std::random_device("rdseed") don't use the same source.

I suppose we could check the number of open file descriptors as a proxy
for "something is reading /dev/random", but there's no way to check
that std::random_device(tok1) and std::random_device(tok2) use
different sources of randomness.

Re: require et random_device for cons token test

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On 25/03/21 11:57 +0000, Jonathan Wakely wrote:
>On 25/03/21 07:17 -0300, Alexandre Oliva wrote:
>>On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
>>
>>>This works for me on x86_64-linux and powerpc64le-linux, and also on
>>>x86_64-linux when I kluge the config macros so that the new code path
>>>gets used. Does this work for VxWorks?
>>
>>Thanks.  I (trivially) backported it to apply on our gcc-10 tree, and
>>tested that on x86_64-vx7r2, and I confirm it works there too.
>>
>>However, I suspect there's a series of typos in the patch.  You appear
>>to be using the 'which' enum variable for bit testing, but with '|'
>>rather than '&'.
>
>Oops, that's what I get for a last-minute rewrite without proper
>testing. I originally had:
>
>  if (which == blah || which == any)
>
>and then borked it in an attempt to use & instead.
>
>I'll fix that locally too.

Here's what I've pushed to trunk.

Tested x86_64-linux, powerpc64le-linux, x86_64-w64-mingw.



[-- Attachment #2: patch.txt --]
[-- Type: text/x-patch, Size: 17560 bytes --]

commit 5f070ba29803c99a5fe94ed7632d7b8c55593df3
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Fri Mar 26 18:39:49 2021

    libstdc++: Add PRNG fallback to std::random_device
    
    This makes std::random_device usable on VxWorks when running on older
    x86 hardware. Since the r10-728 fix for PR libstdc++/85494 the library
    will use the new code unconditionally on x86, but the cpuid checks for
    RDSEED and RDRAND can fail at runtime, depending on the hardware where
    the code is executing. If the OS does not provide /dev/urandom then this
    means the std::random_device constructor always fails. In previous
    releases if /dev/urandom is unavailable then std::mt19937 was used
    unconditionally.
    
    This patch adds a fallback for the case where the runtime cpuid checks
    for x86 hardware instructions fail, and no /dev/urandom is available.
    When this happens a std::linear_congruential_engine object will be used,
    with a seed based on hashing the engine's address and the current time.
    Distinct std::random_device objects will use different seeds, unless an
    object is created and destroyed and a new object created at the same
    memory location within the clock tick. This is not great, but is better
    than always throwing from the constructor, and better than always using
    std::mt19937 with the same seed (as GCC 9 and earlier do).
    
    libstdc++-v3/ChangeLog:
    
            * src/c++11/random.cc (USE_LCG): Define when a pseudo-random
            fallback is needed.
            [USE_LCG] (bad_seed, construct_lcg_at, destroy_lcg_at, __lcg):
            New helper functions and callback.
            (random_device::_M_init): Add 'prng' and 'all' enumerators.
            Replace switch with fallthrough with a series of 'if' statements.
            [USE_LCG]: Construct an lcg_type engine and use __lcg when cpuid
            checks fail.
            (random_device::_M_init_pretr1) [USE_MT19937]: Accept "prng"
            token.
            (random_device::_M_getval): Check for callback unconditionally
            and always pass _M_file pointer.
            * testsuite/26_numerics/random/random_device/85494.cc: Remove
            effective-target check. Use new random_device_available helper.
            * testsuite/26_numerics/random/random_device/94087.cc: Likewise.
            * testsuite/26_numerics/random/random_device/cons/default-cow.cc:
            Remove effective-target check.
            * testsuite/26_numerics/random/random_device/cons/default.cc:
            Likewise.
            * testsuite/26_numerics/random/random_device/cons/token.cc: Use
            new random_device_available helper. Test "prng" token.
            * testsuite/util/testsuite_random.h (random_device_available):
            New helper function.

diff --git a/libstdc++-v3/src/c++11/random.cc b/libstdc++-v3/src/c++11/random.cc
index 1092299e56d..44b9f30e4a9 100644
--- a/libstdc++-v3/src/c++11/random.cc
+++ b/libstdc++-v3/src/c++11/random.cc
@@ -66,14 +66,23 @@
 # include <stdlib.h>
 #endif
 
-#if defined USE_RDRAND || defined USE_RDSEED \
-  || defined _GLIBCXX_USE_CRT_RAND_S || defined _GLIBCXX_USE_DEV_RANDOM
+#if defined _GLIBCXX_USE_CRT_RAND_S || defined _GLIBCXX_USE_DEV_RANDOM
+// The OS provides a source of randomness we can use.
 # pragma GCC poison _M_mt
+#elif defined USE_RDRAND || defined USE_RDSEED
+// Hardware instructions might be available, but use cpuid checks at runtime.
+# pragma GCC poison _M_mt
+// If the runtime cpuid checks fail we'll use a linear congruential engine.
+# define USE_LCG 1
 #else
 // Use the mt19937 member of the union, as in previous GCC releases.
 # define USE_MT19937 1
 #endif
 
+#ifdef USE_LCG
+# include <chrono>
+#endif
+
 namespace std _GLIBCXX_VISIBILITY(default)
 {
   namespace
@@ -136,6 +145,53 @@ namespace std _GLIBCXX_VISIBILITY(default)
       return val;
     }
 #endif
+
+#ifdef USE_LCG
+    // TODO: use this to seed std::mt19937 engine too.
+    unsigned
+    bad_seed(void* p) noexcept
+    {
+      // Poor quality seed based on hash of the current time and the address
+      // of the object being seeded. Better than using the same default seed
+      // for every object though.
+      const uint64_t bits[] = {
+	(uint64_t) chrono::system_clock::now().time_since_epoch().count(),
+	(uint64_t) reinterpret_cast<uintptr_t>(p)
+      };
+      auto bytes = reinterpret_cast<const unsigned char*>(bits);
+      // 32-bit FNV-1a hash
+      uint32_t h = 2166136261u;
+      for (unsigned i = 0; i < sizeof(bits); ++i)
+	{
+	  h ^= *bytes++;
+	  h *= 16777619u;
+	}
+      return h;
+    }
+
+    // Same as std::minstd_rand0 but using unsigned not uint_fast32_t.
+    using lcg_type
+      = linear_congruential_engine<unsigned, 16807UL, 0UL, 2147483647UL>;
+
+    inline lcg_type*
+    construct_lcg_at(void* addr) noexcept
+    {
+      return ::new(addr) lcg_type(bad_seed(addr));
+    }
+
+    inline void
+    destroy_lcg_at(void* addr) noexcept
+    {
+      static_cast<lcg_type*>(addr)->~lcg_type();
+    }
+
+    unsigned int
+    __lcg(void* ptr) noexcept
+    {
+      auto& lcg = *static_cast<lcg_type*>(ptr);
+      return lcg();
+    }
+#endif
   }
 
   void
@@ -152,25 +208,16 @@ namespace std _GLIBCXX_VISIBILITY(default)
     _M_fd = -1;
 
     const char* fname [[gnu::unused]] = nullptr;
-    bool default_token [[gnu::unused]] = false;
 
-    enum { rand_s, rdseed, rdrand, device_file } which;
+    enum {
+	rand_s = 1, rdseed = 2, rdrand = 4, device_file = 8, prng = 16,
+	any = 0xffff
+    } which;
 
     if (token == "default")
       {
-	default_token = true;
+	which = any;
 	fname = "/dev/urandom";
-#if defined _GLIBCXX_USE_CRT_RAND_S
-	which = rand_s;
-#elif defined USE_RDSEED
-	which = rdseed;
-#elif defined USE_RDRAND
-	which = rdrand;
-#elif defined _GLIBCXX_USE_DEV_RANDOM
-	which = device_file;
-#else
-# error "either define USE_MT19937 above or set the default device here"
-#endif
       }
 #ifdef USE_RDSEED
     else if (token == "rdseed")
@@ -191,99 +238,104 @@ namespace std _GLIBCXX_VISIBILITY(default)
 	which = device_file;
       }
 #endif // _GLIBCXX_USE_DEV_RANDOM
+#ifdef USE_LCG
+    else if (token == "prng")
+      which = prng;
+#endif
     else
       std::__throw_runtime_error(
 	  __N("random_device::random_device(const std::string&):"
 	      " unsupported token"));
 
-    switch (which)
-    {
 #ifdef _GLIBCXX_USE_CRT_RAND_S
-      case rand_s:
-      {
-	_M_func = &__winxp_rand_s;
-	return;
-      }
-#endif // _GLIBCXX_USE_CRT_RAND_S
-#ifdef USE_RDSEED
-      case rdseed:
-      {
-	unsigned int eax, ebx, ecx, edx;
-	// Check availability of cpuid and, for now at least, also the
-	// CPU signature for Intel and AMD.
-	if (__get_cpuid_max(0, &ebx) > 0
-	    && (ebx == signature_INTEL_ebx || ebx == signature_AMD_ebx))
-	  {
-	    // CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18]
-	    __cpuid_count(7, 0, eax, ebx, ecx, edx);
-	    if (ebx & bit_RDSEED)
-	      {
-#ifdef USE_RDRAND
-		// CPUID.01H:ECX.RDRAND[bit 30]
-		__cpuid(1, eax, ebx, ecx, edx);
-		if (ecx & bit_RDRND)
-		  {
-		    _M_func = &__x86_rdseed_rdrand;
-		    return;
-		  }
-#endif
-		_M_func = &__x86_rdseed;
-		return;
-	      }
-	  }
-	// If rdseed was explicitly requested then we're done here.
-	if (!default_token)
-	  break;
-	// Otherwise fall through to try the next available option.
-	[[gnu::fallthrough]];
-      }
-#endif // USE_RDSEED
-#ifdef USE_RDRAND
-      case rdrand:
-      {
-	unsigned int eax, ebx, ecx, edx;
-	// Check availability of cpuid and, for now at least, also the
-	// CPU signature for Intel and AMD.
-	if (__get_cpuid_max(0, &ebx) > 0
-	    && (ebx == signature_INTEL_ebx || ebx == signature_AMD_ebx))
-	  {
-	    // CPUID.01H:ECX.RDRAND[bit 30]
-	    __cpuid(1, eax, ebx, ecx, edx);
-	    if (ecx & bit_RDRND)
-	      {
-		_M_func = &__x86_rdrand;
-		return;
-	      }
-	  }
-	// If rdrand was explicitly requested then we're done here.
-	if (!default_token)
-	  break;
-	// Otherwise fall through to try the next available option.
-	[[gnu::fallthrough]];
-      }
-#endif // USE_RDRAND
-#ifdef _GLIBCXX_USE_DEV_RANDOM
-      case device_file:
-      {
-#ifdef USE_POSIX_FILE_IO
-	_M_fd = ::open(fname, O_RDONLY);
-	if (_M_fd != -1)
-	  {
-	    // Set _M_file to non-null so that _M_fini() will do clean up.
-	    _M_file = &_M_fd;
-	    return;
-	  }
-#else // USE_POSIX_FILE_IO
-	_M_file = static_cast<void*>(std::fopen(fname, "rb"));
-	if (_M_file)
-	  return;
-#endif // USE_POSIX_FILE_IO
-	[[gnu::fallthrough]];
-      }
-#endif // _GLIBCXX_USE_DEV_RANDOM
-      default:
-      { }
+    if (which & rand_s)
+    {
+      _M_func = &__winxp_rand_s;
+      return;
     }
+#endif // _GLIBCXX_USE_CRT_RAND_S
+
+#ifdef USE_RDSEED
+    if (which & rdseed)
+    {
+      unsigned int eax, ebx, ecx, edx;
+      // Check availability of cpuid and, for now at least, also the
+      // CPU signature for Intel and AMD.
+      if (__get_cpuid_max(0, &ebx) > 0
+	  && (ebx == signature_INTEL_ebx || ebx == signature_AMD_ebx))
+	{
+	  // CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18]
+	  __cpuid_count(7, 0, eax, ebx, ecx, edx);
+	  if (ebx & bit_RDSEED)
+	    {
+#ifdef USE_RDRAND
+	      // CPUID.01H:ECX.RDRAND[bit 30]
+	      __cpuid(1, eax, ebx, ecx, edx);
+	      if (ecx & bit_RDRND)
+		{
+		  _M_func = &__x86_rdseed_rdrand;
+		  return;
+		}
+#endif
+	      _M_func = &__x86_rdseed;
+	      return;
+	    }
+	}
+    }
+#endif // USE_RDSEED
+
+#ifdef USE_RDRAND
+    if (which & rdrand)
+    {
+      unsigned int eax, ebx, ecx, edx;
+      // Check availability of cpuid and, for now at least, also the
+      // CPU signature for Intel and AMD.
+      if (__get_cpuid_max(0, &ebx) > 0
+	  && (ebx == signature_INTEL_ebx || ebx == signature_AMD_ebx))
+	{
+	  // CPUID.01H:ECX.RDRAND[bit 30]
+	  __cpuid(1, eax, ebx, ecx, edx);
+	  if (ecx & bit_RDRND)
+	    {
+	      _M_func = &__x86_rdrand;
+	      return;
+	    }
+	}
+    }
+#endif // USE_RDRAND
+
+#ifdef _GLIBCXX_USE_DEV_RANDOM
+    if (which & device_file)
+    {
+#ifdef USE_POSIX_FILE_IO
+      _M_fd = ::open(fname, O_RDONLY);
+      if (_M_fd != -1)
+	{
+	  // Set _M_file to non-null so that _M_fini() will do clean up.
+	  _M_file = &_M_fd;
+	  return;
+	}
+#else // USE_POSIX_FILE_IO
+      _M_file = static_cast<void*>(std::fopen(fname, "rb"));
+      if (_M_file)
+	return;
+#endif // USE_POSIX_FILE_IO
+    }
+#endif // _GLIBCXX_USE_DEV_RANDOM
+
+#ifdef USE_LCG
+    // Either "prng" was requested explicitly, or "default" was requested
+    // but nothing above worked, use a PRNG.
+    if (which & prng)
+    {
+      static_assert(sizeof(lcg_type) <= sizeof(_M_fd), "");
+      static_assert(alignof(lcg_type) <= alignof(_M_fd), "");
+      _M_file = construct_lcg_at(&_M_fd);
+      _M_func = &__lcg;
+      return;
+    }
+#endif
+
     std::__throw_runtime_error(
 	__N("random_device::random_device(const std::string&):"
 	    " device not available"));
@@ -297,7 +349,7 @@ namespace std _GLIBCXX_VISIBILITY(default)
   {
 #ifdef USE_MT19937
     unsigned long seed = 5489UL;
-    if (token != "default" && token != "mt19937")
+    if (token != "default" && token != "mt19937" && token != "prng")
       {
 	const char* nptr = token.c_str();
 	char* endptr;
@@ -335,6 +387,14 @@ namespace std _GLIBCXX_VISIBILITY(default)
     if (!_M_file)
       return;
 
+#if USE_LCG
+    if (_M_func == &__lcg)
+      {
+	destroy_lcg_at(_M_file);
+	return;
+      }
+#endif
+
 #ifdef USE_POSIX_FILE_IO
     ::close(_M_fd);
     _M_fd = -1;
@@ -351,10 +411,8 @@ namespace std _GLIBCXX_VISIBILITY(default)
     return _M_mt();
 #else
 
-#if defined USE_RDRAND || defined USE_RDSEED || defined _GLIBCXX_USE_CRT_RAND_S
     if (_M_func)
-      return _M_func(nullptr);
-#endif
+      return _M_func(_M_file);
 
     result_type ret;
     void* p = &ret;
@@ -430,5 +488,11 @@ namespace std _GLIBCXX_VISIBILITY(default)
     0x9d2c5680UL, 15,
     0xefc60000UL, 18, 1812433253UL>;
 #endif // USE_MT19937
+
+#ifdef USE_LCG
+  template class
+    linear_congruential_engine<unsigned, 16807UL, 0UL, 2147483647UL>;
+  template struct __detail::_Mod<unsigned, 2147483647UL, 16807UL, 0UL>;
+#endif
 }
 #endif // _GLIBCXX_USE_C99_STDINT_TR1
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc
index 8bfb1fa71bf..80cb912b587 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/85494.cc
@@ -16,14 +16,21 @@
 // <http://www.gnu.org/licenses/>.
 
 // { dg-do run { target c++11 } }
-// { dg-require-effective-target random_device }
 
 #include <random>
 #include <testsuite_hooks.h>
+#include <testsuite_random.h>
 
 void
 test01()
 {
+  if (__gnu_test::random_device_available("mt19937"))
+  {
+    // std::random_device uses a Mersenne Twister with default seed,
+    // and the test below will fail.  No point trying to test it.
+    return;
+  }
+
   unsigned v1[3], v2[3];
   std::random_device d1, d2;
   for (auto& v : v1)
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc
index c77917fb680..7ff672e7024 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/94087.cc
@@ -25,18 +25,9 @@
 #include <memory>
 #include <thread>
 #include <cstdio>
+#include <testsuite_random.h>
 
-bool
-random_device_available(const char* token) noexcept
-{
-  try {
-    std::random_device dev(token);
-    return true;
-  } catch (...) {
-    std::printf("random_device(\"%s\") not available\n", token);
-    return false;
-  }
-}
+using __gnu_test::random_device_available;
 
 void read_random_device(const char* token, int iterations)
 {
@@ -59,5 +50,7 @@ int main() {
       for (auto& w : workers)
 	w.join();
     }
+    else
+      std::printf("random_device(\"%s\") not available\n", dev);
   }
 }
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc
index ef35773e009..489a0a0965f 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default-cow.cc
@@ -1,6 +1,5 @@
 // { dg-options "-D_GLIBCXX_USE_CXX11_ABI=0" }
 // { dg-do run { target c++11 } }
-// { dg-require-effective-target random_device }
 // { dg-require-cstdint "" }
 //
 // Copyright (C) 2019-2021 Free Software Foundation, Inc.
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc
index 72756e245d5..79e044ece42 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/default.cc
@@ -1,5 +1,4 @@
 // { dg-do run { target c++11 } }
-// { dg-require-effective-target random_device }
 // { dg-require-cstdint "" }
 //
 // 2008-11-24  Edward M. Smith-Rowland <3dw4rd@verizon.net>
diff --git a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
index defb8d58c58..aeb7403e830 100644
--- a/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
+++ b/libstdc++-v3/testsuite/26_numerics/random/random_device/cons/token.cc
@@ -25,6 +25,7 @@
 #include <random>
 #include <stdexcept>
 #include <testsuite_hooks.h>
+#include <testsuite_random.h>
 
 void
 test01()
@@ -50,19 +51,14 @@ test03()
 {
   // At least one of these tokens should be valid.
   const std::string tokens[] = {
-    "rdseed", "rdrand", "rand_s", "/dev/urandom", "/dev/random", "mt19937"
+    "rdseed", "rdrand", "rand_s", "/dev/urandom", "/dev/random", "mt19937",
+    "prng"
   };
   int count = 0;
   for (const std::string& token : tokens)
   {
-    try
-    {
-      std::random_device x(token);
+    if (__gnu_test::random_device_available(token))
       ++count;
-    }
-    catch (const std::runtime_error&)
-    {
-    }
   }
   VERIFY( count != 0 );
 }
@@ -70,21 +66,12 @@ test03()
 void
 test04()
 {
-  bool can_use_mt19937 = true;
-  std::random_device::result_type xval;
-  try
+  if (__gnu_test::random_device_available("mt19937"))
   {
     std::random_device x("mt19937");
-    xval = x();
-  }
-  catch (const std::runtime_error&)
-  {
-    can_use_mt19937 = false;
-  }
+    std::random_device::result_type xval = x();
 
-  // If "mt19937" is a valid token then numeric seeds should be too.
-  if (can_use_mt19937)
-  {
+    // If "mt19937" is a valid token then numeric seeds should be too.
     std::random_device x1("0");
     std::random_device x2("1234");
     std::random_device x3("0xc0fefe");
diff --git a/libstdc++-v3/testsuite/util/testsuite_random.h b/libstdc++-v3/testsuite/util/testsuite_random.h
index 0b670bfb771..8fba2688bc0 100644
--- a/libstdc++-v3/testsuite/util/testsuite_random.h
+++ b/libstdc++-v3/testsuite/util/testsuite_random.h
@@ -197,6 +197,18 @@ namespace __gnu_test
   }
 #endif
 
+  // Check whether TOKEN can construct a std::random_device successfully.
+  inline bool
+  random_device_available(const std::string& token) noexcept
+  {
+    try {
+      std::random_device dev(token);
+      return true;
+    } catch (...) {
+      return false;
+    }
+  }
+
 } // namespace __gnu_test
 
 #endif // #ifndef _GLIBCXX_TESTSUITE_RANDOM_H

Re: require et random_device for cons token test

[Alexandre Oliva]

On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:

> diff --git a/libstdc++-v3/testsuite/util/testsuite_random.h b/libstdc++-v3/testsuite/util/testsuite_random.h
> index 0b670bfb771..c8323078492 100644
> --- a/libstdc++-v3/testsuite/util/testsuite_random.h
> +++ b/libstdc++-v3/testsuite/util/testsuite_random.h
> @@ -197,6 +197,19 @@ namespace __gnu_test
>    }
>  #endif
 
> +  // Check whether TOKEN can construct a std::random_device successfully.
> +  inline bool
> +  random_device_available(const std::string& token) noexcept
> +  {
> +    try {
> +      std::random_device dev(token);
> +      return true;
> +    } catch (...) {
> +      std::printf("random_device(\"%s\") not available\n", token);

Another nit: I'm seeing line noise (do people still use this term? :-)
in libstdc++.log where unavailable random_device tokens should appear.
I suspect the token has to be converted to a C string in the call above.

-- 
Alexandre Oliva, happy hacker  https://FSFLA.org/blogs/lxo/
   Free Software Activist         GNU Toolchain Engineer
        Vim, Vi, Voltei pro Emacs -- GNUlius Caesar

Re: require et random_device for cons token test

[Jonathan Wakely]

On 25/03/21 08:03 -0300, Alexandre Oliva wrote:
>On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
>
>> diff --git a/libstdc++-v3/testsuite/util/testsuite_random.h b/libstdc++-v3/testsuite/util/testsuite_random.h
>> index 0b670bfb771..c8323078492 100644
>> --- a/libstdc++-v3/testsuite/util/testsuite_random.h
>> +++ b/libstdc++-v3/testsuite/util/testsuite_random.h
>> @@ -197,6 +197,19 @@ namespace __gnu_test
>>    }
>>  #endif
>
>> +  // Check whether TOKEN can construct a std::random_device successfully.
>> +  inline bool
>> +  random_device_available(const std::string& token) noexcept
>> +  {
>> +    try {
>> +      std::random_device dev(token);
>> +      return true;
>> +    } catch (...) {
>> +      std::printf("random_device(\"%s\") not available\n", token);
>
>Another nit: I'm seeing line noise (do people still use this term? :-)
>in libstdc++.log where unavailable random_device tokens should appear.
>I suspect the token has to be converted to a C string in the call above.

Doh, the function took a const char* then I changed it to std::string
because some of the callers wanted to pass a std::string. I'll fix it
in my branch.

Re: require et random_device for cons token test

[Alexandre Oliva]

On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:

> Does vxworks provide any platform-specific source of randomness, like
> Linux getrandom(2) or BSD arc4random(3) or Windows rand_s? If yes, we
> should add support for that (in the next stage 1).

There appears to be a randNumGenCtl syscall that appears to be relevant
to that end, and randAdd to seed and randBytes to obtain random bytes in
vxRandLib.  I couldn't find documentation on how to use it, but there
seems to be some code using it in openssl.  Sorry, I don't know a lot
about vxworks.

-- 
Alexandre Oliva, happy hacker  https://FSFLA.org/blogs/lxo/
   Free Software Activist         GNU Toolchain Engineer
        Vim, Vi, Voltei pro Emacs -- GNUlius Caesar

Re: require et random_device for cons token test

[Jonathan Wakely]

On 25/03/21 08:00 -0300, Alexandre Oliva wrote:
>On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
>
>> Does vxworks provide any platform-specific source of randomness, like
>> Linux getrandom(2) or BSD arc4random(3) or Windows rand_s? If yes, we
>> should add support for that (in the next stage 1).
>
>There appears to be a randNumGenCtl syscall that appears to be relevant
>to that end, and randAdd to seed and randBytes to obtain random bytes in
>vxRandLib.  I couldn't find documentation on how to use it, but there
>seems to be some code using it in openssl.  Sorry, I don't know a lot
>about vxworks.

Thanks! We could look into that in stage 1.

Re: require et random_device for cons token test

[Jonathan Wakely]

[-- Attachment #1: Type: text/plain, Size: 820 bytes --]

On Thu, 25 Mar 2021 at 11:38, Jonathan Wakely wrote:

> On 25/03/21 08:00 -0300, Alexandre Oliva wrote:
> >On Mar 24, 2021, Jonathan Wakely <jwakely@redhat.com> wrote:
> >
> >> Does vxworks provide any platform-specific source of randomness, like
> >> Linux getrandom(2) or BSD arc4random(3) or Windows rand_s? If yes, we
> >> should add support for that (in the next stage 1).
> >
> >There appears to be a randNumGenCtl syscall that appears to be relevant
> >to that end, and randAdd to seed and randBytes to obtain random bytes in
> >vxRandLib.  I couldn't find documentation on how to use it, but there
> >seems to be some code using it in openssl.  Sorry, I don't know a lot
> >about vxworks.
>
> Thanks! We could look into that in stage 1.
>

Maybe something like this, completely untested and not ready to commit.

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 2431 bytes --]

diff --git a/libstdc++-v3/src/c++11/random.cc b/libstdc++-v3/src/c++11/random.cc
index 70fd520077a..eb092cd0950 100644
--- a/libstdc++-v3/src/c++11/random.cc
+++ b/libstdc++-v3/src/c++11/random.cc
@@ -75,6 +75,14 @@
 # include <unistd.h>
 #endif
 
+#if __has_include(<_vxworks-versions.h>)
+# include <_vxworks-versions.h>
+# if _VXWORKS_MAJOR_GE(7) && __has_include(<randomNumGen.h>)
+#  include <randomNumGen.h>
+#  define USE_RANDBYTES 1
+# endif
+#endif
+
 #if defined _GLIBCXX_USE_CRT_RAND_S || defined _GLIBCXX_USE_DEV_RANDOM \
   || _GLIBCXX_HAVE_GETENTROPY
 // The OS provides a source of randomness we can use.
@@ -221,6 +229,29 @@ namespace std _GLIBCXX_VISIBILITY(default)
     }
 #endif
 
+#if USE_RANDBYTES
+    unsigned int
+    __vx_randBytes(void*)
+    {
+      int retries = 10;
+      unsigned int val;
+      auto bytes = reinterpret_cast<unsigned char*>(&val);
+      while (retries-- > 0)
+	{
+	  RANDOM_NUM_GEN_STATUS status = randStatus();
+	  if (status == RANDOM_NUM_GEN_ENOUGH_ENTROPY
+	     || status == RANDOM_NUM_GEN_MAX_ENTROPY)
+	    {
+	      if (randBytes(bytes, sizeof(val)) == OK)
+		return val;
+	    }
+	  else
+	    taskDelay(5);
+	}
+      std::__throw_runtime_error(__N("random_device: randBytes failed"));
+    }
+#endif
+
 #ifdef USE_LCG
     // TODO: use this to seed std::mt19937 engine too.
     unsigned
@@ -271,6 +302,7 @@ namespace std _GLIBCXX_VISIBILITY(default)
     enum Which : unsigned {
       device_file = 1, prng = 2, rand_s = 4, getentropy = 8, arc4random = 16,
       rdseed = 64, rdrand = 128, darn = 256, rndr = 512,
+      randBytes = 1024,
       any = 0xffff
     };
 
@@ -326,6 +358,11 @@ namespace std _GLIBCXX_VISIBILITY(default)
 	return getentropy;
 #endif
 
+#if USE_RANDBYTES
+      if (func == __vx_randBytes)
+	return randBytes;
+#endif
+
 #ifdef USE_LCG
       if (func == &__lcg)
 	return prng;
@@ -506,6 +543,14 @@ namespace std _GLIBCXX_VISIBILITY(default)
       }
 #endif // _GLIBCXX_HAVE_GETENTROPY
 
+#if USE_RANDBYTES
+    if (which & randBytes)
+      {
+	_M_func = __vx_randBytes;
+	return;
+      }
+#endif
+
 #ifdef _GLIBCXX_USE_DEV_RANDOM
     if (which & device_file)
     {
@@ -666,6 +711,8 @@ namespace std _GLIBCXX_VISIBILITY(default)
     case rand_s:
     case prng:
       return 0.0;
+    case randBytes: // XXX is this a real source of entropy or PRNG?
+      return 0.0;
     case device_file:
       // handled below
       break;

Re: require et random_device for cons token test

[Jonathan Wakely]

On 24/03/21 08:59 +0000, Jonathan Wakely via Libstdc++ wrote:
>On 24/03/21 03:53 -0300, Alexandre Oliva wrote:
>>
>>On target systems that don't support any random_device, not even the
>>default one,
>
>It should be impossible to have no random_device. As a fallback a
>pseudo random number generator should be used.
>
>>other random_device constructor tests are disabled by
>>dg-require-effective-target random_device.  The token.cc also
>>exercises the default constructor, in a way that doesn't expect an
>>exception to be raised, but it's not guarded by the same requirement.
>>
>>Other potentially-raising ctors in token.cc expect exceptions and
>>handle them, but the ("default")-constructed one does not, so the
>>program terminates and the test fails without exercising the other
>>constructor variants.
>
>Why does the "default" token cause an exception?
>
>>This patch arranges to disable the test altogether when the
>>random_device feature is not available.  A reasonable alternative
>>would be to install a std::runtime_error handler around the test01
>>body, so that we exercise at least the exception raising, but then
>>test03 would have to be relaxed, since without even "default", it
>>likely wouldn't meet the tested requirement there.
>>
>>Regstrapped on x86_64-linux-gnu and cross-tested for x86_64-vx7r2 along
>>with other patches, mostly for the testsuite.  Ok to install?
>
>No, that would disable the test for Windows, where it works OK.
>
>The 'random_device' effective-target is poorly named, it checks
>whether the _GLIBCXX_USE_RANDOM_TR1 macro is defined, which is even
>more poorly named. That macro is defined when /dev/random and
>/dev/urandom are available. But we support std::random_device without
>those files (e.g. on Windows).

And that macro is defined according to the same conditions as
_GLIBCXX_USE_DEV_RANDOM which is tested via #ifdef in cons/token.cc.
So the test already guards the parts that should depend on the
_GLIBCXX_USE_RANDOM_TR1 macro. Everything else in the test *should*
work unconditionally. In theory. We should find out why they don't
(and probably fix the library code, not disable the test).

Several of the other tests in that directory should *not* require that
effective-target. They should now work unconditionally too.

Tangentially, it would also be good to improve the definition of the
effective-target so it is true for Windows and other targets that are
guaranteed to have a useful (non-pseudo RNG) std::random_device.

Re: require et random_device for cons token test

[Koning, Paul]

> On Mar 24, 2021, at 4:59 AM, Jonathan Wakely via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
> 
> On 24/03/21 03:53 -0300, Alexandre Oliva wrote:
>> 
>> On target systems that don't support any random_device, not even the
>> default one,
> 
> It should be impossible to have no random_device.

Not true; deeply embedded systems might not have one.  Among GCC platforms, pdp11 doesn't have one, and at least some vax platforms probably don't either.

> As a fallback a
> pseudo random number generator should be used.

Presumably yes -- it seems unlikely that GCC tests depend on cryptographic strength of the random nummber generator.  If a PRNG is used then the classic FORTRAN "random" function would serve.  

	paul

Re: require et random_device for cons token test

[Jonathan Wakely]

On 24/03/21 13:22 +0000, Koning, Paul via Libstdc++ wrote:
>
>
>> On Mar 24, 2021, at 4:59 AM, Jonathan Wakely via Gcc-patches <gcc-patches@gcc.gnu.org> wrote:
>>
>> On 24/03/21 03:53 -0300, Alexandre Oliva wrote:
>>>
>>> On target systems that don't support any random_device, not even the
>>> default one,
>>
>> It should be impossible to have no random_device.
>
>Not true; deeply embedded systems might not have one.  Among GCC platforms, pdp11 doesn't have one, and at least some vax platforms probably don't either.

I'm talking about the C++ std::random_device type, not the /dev/random
block device.

>> As a fallback a
>> pseudo random number generator should be used.
>
>Presumably yes -- it seems unlikely that GCC tests depend on cryptographic strength of the random nummber generator.  If a PRNG is used then the classic FORTRAN "random" function would serve.

Libstdc++ already contains several PNRGs, we don't need to use
Fortran's.