Re: [PATCH] Add example to rand.3

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1542 bytes --]

Hi Jonny,

On 12/26/22 22:50, Jonny Grant wrote:
> Hi Alejandro

Please send also to my email.

> Please find below a patch.
> 
> 2022-12-26  Jonathan Grant <jg@jguk.org>
> 	* man3/rand.3: Add example to rand.3 seed with time(NULL)
> 
> 
>  From 2d4501354ea6c465173fe6c089dfbcc80393a644 Mon Sep 17 00:00:00 2001
> From: Jonathan Grant <jg@jguk.org>
> Date: Mon, 26 Dec 2022 21:48:17 +0000
> Subject: [PATCH] add rand.3 example
> 
> Signed-off-by: Jonathan Grant <jg@jguk.org>

time(NULL) is not too good.  If you call it several times per second, you'll 
find that it only changes the seed every second.  There are better ways to 
produce a good seed.

However, I prefer suggesting arc4random() rather than workarounding rand(3) to 
get good results.

Florian, did you already merge arc4random() to glibc?

Cheers,

Alex

> ---
>   man3/rand.3 | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
> 
> diff --git a/man3/rand.3 b/man3/rand.3
> index 572471749..c1542fb56 100644
> --- a/man3/rand.3
> +++ b/man3/rand.3
> @@ -164,6 +164,20 @@ when good randomness is needed.
>   .BR random (3)
>   instead.)
>   .SH EXAMPLES
> +
> +A possibly useful seed value would be by calling
> +.BR rand ()
> +with the result of
> +
> +.BR time ()
> +as that varies with every call
> +
> +.EX
> +srand((unsigned int)time(NULL));
> +.EE
> +.in
> +.PP
> +
>   POSIX.1-2001 gives the following example of an implementation of
>   .BR rand ()
>   and

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Cristian Rodríguez]

On Mon, Dec 26, 2022 at 7:29 PM Alejandro Colomar via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> Hi Jonny,
>
l it several times per second, you'll
> find that it only changes the seed every second.  There are better ways to
> produce a good seed.
>
> However, I prefer suggesting arc4random() rather than workarounding rand(3) to
> get good results.
>
> Florian, did you already merge arc4random() to glibc?

arc4random is already on libc.. now to make it a replacement for
everything random  one has to hope for
https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/log/?h=vdso
to be merged into the mainline kernel.
That will make it as fast as possible and really,really cheap to call.
Now it is just a wrapper that calls getrandom syscall.

WRT the rand() example suggestion..it is bad.. The only addition I
will make to this man page is strongly discouraging its use.
Suggesting to use ar4random if available or one of
https://prng.di.unimi.it/ PRNG if not for crypto.

Re: [PATCH] Add example to rand.3

[Jonny Grant]

Hi Alex

On 26/12/2022 22:29, Alejandro Colomar wrote:
> Hi Jonny,
> 
> On 12/26/22 22:50, Jonny Grant wrote:
>> Hi Alejandro
> 
> Please send also to my email.
> 
>> Please find below a patch.
>>
>> 2022-12-26  Jonathan Grant <jg@jguk.org>
>>     * man3/rand.3: Add example to rand.3 seed with time(NULL)
>>
>>
>>  From 2d4501354ea6c465173fe6c089dfbcc80393a644 Mon Sep 17 00:00:00 2001
>> From: Jonathan Grant <jg@jguk.org>
>> Date: Mon, 26 Dec 2022 21:48:17 +0000
>> Subject: [PATCH] add rand.3 example
>>
>> Signed-off-by: Jonathan Grant <jg@jguk.org>
> 
> time(NULL) is not too good.  If you call it several times per second, you'll find that it only changes the seed every second.  There are better ways to produce a good seed.
> 
> However, I prefer suggesting arc4random() rather than workarounding rand(3) to get good results.
> 
> Florian, did you already merge arc4random() to glibc?

Hopefully arc4random will come soon. Maybe rand.3 could then be updated to SEE ALSO that.

I would only mention to call srand once to seed, but you're right there are lots of other ways to get a seed.
Jonny

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1600 bytes --]

Hi Jonny,

On 12/27/22 22:37, Jonny Grant wrote:
> Hi Alex
> 
> On 26/12/2022 22:29, Alejandro Colomar wrote:
>> Hi Jonny,
>>
>> On 12/26/22 22:50, Jonny Grant wrote:
>>> Hi Alejandro
>>
>> Please send also to my email.
>>
>>> Please find below a patch.
>>>
>>> 2022-12-26  Jonathan Grant <jg@jguk.org>
>>>      * man3/rand.3: Add example to rand.3 seed with time(NULL)
>>>
>>>
>>>   From 2d4501354ea6c465173fe6c089dfbcc80393a644 Mon Sep 17 00:00:00 2001
>>> From: Jonathan Grant <jg@jguk.org>
>>> Date: Mon, 26 Dec 2022 21:48:17 +0000
>>> Subject: [PATCH] add rand.3 example
>>>
>>> Signed-off-by: Jonathan Grant <jg@jguk.org>
>>
>> time(NULL) is not too good.  If you call it several times per second, you'll find that it only changes the seed every second.  There are better ways to produce a good seed.
>>
>> However, I prefer suggesting arc4random() rather than workarounding rand(3) to get good results.
>>
>> Florian, did you already merge arc4random() to glibc?
> 
> Hopefully arc4random will come soon. Maybe rand.3 could then be updated to SEE ALSO that. >
> I would only mention to call srand once to seed, but you're right there are lots of other ways to get a seed.
> Jonny

But consider the following case:

You're testing some program with random data in a loop.  And the program 
executes in less than a second.  Then srand(3) will be called several times per 
second with the same seed, and you'll get useless results.  I faced that exact 
situation a few years ago :)

Cheers,

Alex


-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2044 bytes --]

Hi Christian,

On 12/27/22 14:07, Cristian Rodríguez wrote:
> On Mon, Dec 26, 2022 at 7:29 PM Alejandro Colomar via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
>>
>> Hi Jonny,
>>
> l it several times per second, you'll
>> find that it only changes the seed every second.  There are better ways to
>> produce a good seed.
>>
>> However, I prefer suggesting arc4random() rather than workarounding rand(3) to
>> get good results.
>>
>> Florian, did you already merge arc4random() to glibc?
> 
> arc4random is already on libc.. now to make it a replacement for
> everything random  one has to hope for
> https://git.kernel.org/pub/scm/linux/kernel/git/crng/random.git/log/?h=vdso
> to be merged into the mainline kernel.
> That will make it as fast as possible and really,really cheap to call.

Great!  When that is merged, please ping me.

If someone wants to contribute a manual page for the arc4random family of 
functions, please speak; otherwise I can start writing something.

> Now it is just a wrapper that calls getrandom syscall.
> 
> WRT the rand() example suggestion..it is bad.. The only addition I
> will make to this man page is strongly discouraging its use.
> Suggesting to use ar4random if available or one of
> https://prng.di.unimi.it/ PRNG if not for crypto.

The only problem with arc4random() is the lack of repeatability.  When testing a 
program with random data, you'll need repeatable results.  For that, rand(3) 
Just Works.  When you want unpredictable results, you just seed it with some 
really random value, and you're fine.  You need to be careful to not introduce 
bias, but there's nothing better in libc.  It would be nice if libc provided a 
rand_uniform(3) variant of rand(3), BTW.

So, rand(3) is not to be deprecated, although it must be used with care.  I 
would recommend the following, per Jonny's report:

     if (repeatable)
         srand(seed)
     else
         srand(arc4random());

Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 942 bytes --]

At 2022-12-28T00:33:13+0100, Alejandro Colomar wrote:
> The only problem with arc4random() is the lack of repeatability.  When
> testing a program with random data, you'll need repeatable results.
> For that, rand(3) Just Works.  When you want unpredictable results,
> you just seed it with some really random value, and you're fine.  You
> need to be careful to not introduce bias, but there's nothing better
> in libc.  It would be nice if libc provided a rand_uniform(3) variant
> of rand(3), BTW.

Permit me to counsel against that last proposed name.  In probability
theory "uniform" is already widely and well understood to indicate the
nature of the distribution.  A "uniform" distribution is one in which
each outcome is precisely equally likely.

rand_deterministic() would get the right idea across but is lengthy.

Regards,
Branden

P.S. I asked Bertrand to tag groff 1.23.0.rc2 earlier today.  Happy
     belated Boxing Day.  ;-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2348 bytes --]

Hi Branden,

On 12/28/22 01:00, G. Branden Robinson wrote:
> At 2022-12-28T00:33:13+0100, Alejandro Colomar wrote:
>> The only problem with arc4random() is the lack of repeatability.  When
>> testing a program with random data, you'll need repeatable results.
>> For that, rand(3) Just Works.  When you want unpredictable results,
>> you just seed it with some really random value, and you're fine.  You
>> need to be careful to not introduce bias, but there's nothing better
>> in libc.  It would be nice if libc provided a rand_uniform(3) variant
>> of rand(3), BTW.
> 
> Permit me to counsel against that last proposed name.  In probability
> theory "uniform" is already widely and well understood to indicate the
> nature of the distribution.  A "uniform" distribution is one in which
> each outcome is precisely equally likely.

That's exactly what I miss from the rand(3) interface; a variant that produces a 
value with uniform probability up to some maximum.  There's the same for the 
arc4random() family of functions:

<https://www.gnu.org/software/libc/manual/html_mono/libc.html#index-arc4random_005funiform>

For example I used something similar to this a long time ago:

     #include <stdbit.h>
     #include <stdint.h>
     #include <stdlib.h>

     uint32_t
     rand_uniform(uint32_t upper_bound)
     {
         uint32_t  r;

         do {
             r = rand();
             r %= stdc_bit_ceil(upper_bound);
         } while (r >= upper_bound);

         return r;
     }

(I used C23 syntax for rounding up to a power of 2, to avoid some magic macro. 
That line is not necessary, but improves performance considerably for small 
upper bounds.  I don't know if it's available already in GCC or Clang.)

That function is API-compatible with arc4random_uniform(), but it works with 
rand(3), so it can give repeatable results.  If I didn't screw it, it gives 
uniform results.

> 
> rand_deterministic() would get the right idea across but is lengthy.
> 
> Regards,
> Branden

Cheers,

Alex

> 
> P.S. I asked Bertrand to tag groff 1.23.0.rc2 earlier today.  Happy
>       belated Boxing Day.  ;-)

I saw :)  BTW, the Ada thingy, if it's opinionated as you said, it will help 
considerably to make it easier to read ;)

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Cristian Rodríguez]

On Tue, Dec 27, 2022 at 9:41 PM Alejandro Colomar
<alx.manpages@gmail.com> wrote:
>
> Hi Branden,
>
> On 12/28/22 01:00, G. Branden Robinson wrote:
> > At 2022-12-28T00:33:13+0100, Alejandro Colomar wrote:

> (I used C23 syntax for rounding up to a power of 2, to avoid some magic macro.
> That line is not necessary, but improves performance considerably for small
> upper bounds.  I don't know if it's available already in GCC or Clang.)

at least current gcc13 does not implement N3022 yet :-| that actually
a nice and looong needed addition (like 30 years late)

Re: [PATCH] Add example to rand.3

[Jonny Grant]

On 27/12/2022 23:11, Alejandro Colomar wrote:
> Hi Jonny,
> 
> On 12/27/22 22:37, Jonny Grant wrote:
>> Hi Alex
>>
>> On 26/12/2022 22:29, Alejandro Colomar wrote:
>>> Hi Jonny,
>>>
>>> On 12/26/22 22:50, Jonny Grant wrote:
>>>> Hi Alejandro
>>>
>>> Please send also to my email.
>>>
>>>> Please find below a patch.
>>>>
>>>> 2022-12-26  Jonathan Grant <jg@jguk.org>
>>>>      * man3/rand.3: Add example to rand.3 seed with time(NULL)
>>>>
>>>>
>>>>   From 2d4501354ea6c465173fe6c089dfbcc80393a644 Mon Sep 17 00:00:00 2001
>>>> From: Jonathan Grant <jg@jguk.org>
>>>> Date: Mon, 26 Dec 2022 21:48:17 +0000
>>>> Subject: [PATCH] add rand.3 example
>>>>
>>>> Signed-off-by: Jonathan Grant <jg@jguk.org>
>>>
>>> time(NULL) is not too good.  If you call it several times per second, you'll find that it only changes the seed every second.  There are better ways to produce a good seed.
>>>
>>> However, I prefer suggesting arc4random() rather than workarounding rand(3) to get good results.
>>>
>>> Florian, did you already merge arc4random() to glibc?
>>
>> Hopefully arc4random will come soon. Maybe rand.3 could then be updated to SEE ALSO that. >
>> I would only mention to call srand once to seed, but you're right there are lots of other ways to get a seed.
>> Jonny
> 
> But consider the following case:
> 
> You're testing some program with random data in a loop.  And the program executes in less than a second.  Then srand(3) will be called several times per second with the same seed, and you'll get useless results.  I faced that exact situation a few years ago :)

You're completely right. It's a real issue if software starts multiple times per second, or executes in less than one second and then runs again. Our software always runs for at least minutes, maybe another code suggestion for a seed would be good instead, like arc4random. I do like that rand() offers a reproducible sequence, useful when in some other software we logged the seed value used. random.4 - /dev/random would be a better seed than time(NULL) if running the program multiple times per second. Anyway, rand() is only pseudo-random, utilising /dev/random would be really much more random, and I like that the seed is saved between reboots.

Cheers, Jonny

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1028 bytes --]

Hi Jonny,

On 12/28/22 21:51, Jonny Grant wrote:
> You're completely right. It's a real issue if software starts multiple times per second, or executes in less than one second and then runs again. Our software always runs for at least minutes, maybe another code suggestion for a seed would be good instead, like arc4random. I do like that rand() offers a reproducible sequence, useful when in some other software we logged the seed value used. random.4 - /dev/random would be a better seed than time(NULL) if running the program multiple times per second. Anyway, rand() is only pseudo-random, utilising /dev/random would be really much more random, and I like that the seed is saved between reboots.

Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc. 
libbsd already provides arc4random on GNU/Linux systems, so I can already 
recommend using arc4random to seed srand(3).

I'll prepare a patch...

Cheers,

Alex

> 
> Cheers, Jonny
> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2045 bytes --]

Hi Jonny,

On 12/28/22 21:56, Alejandro Colomar wrote:
> Hi Jonny,
> 
> On 12/28/22 21:51, Jonny Grant wrote:
>> You're completely right. It's a real issue if software starts multiple times 
>> per second, or executes in less than one second and then runs again. Our 
>> software always runs for at least minutes, maybe another code suggestion for a 
>> seed would be good instead, like arc4random. I do like that rand() offers a 
>> reproducible sequence, useful when in some other software we logged the seed 
>> value used. random.4 - /dev/random would be a better seed than time(NULL) if 
>> running the program multiple times per second. Anyway, rand() is only 
>> pseudo-random, utilising /dev/random would be really much more random, and I 
>> like that the seed is saved between reboots.
> 
> Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc. 
> libbsd already provides arc4random on GNU/Linux systems, so I can already 
> recommend using arc4random to seed srand(3).
> 
> I'll prepare a patch...
> 

I will probably apply the following patch.  Do you have any comments?  Does it 
provide the information you wanted to add?

Cheers,

Alex

diff --git a/man3/rand.3 b/man3/rand.3
index 572471749..350a875d8 100644
--- a/man3/rand.3
+++ b/man3/rand.3
@@ -190,6 +190,9 @@ .SH EXAMPLES
  pseudo-random sequence produced by
  .BR rand ()
  when given a particular seed.
+When the seed is
+.IR \-1 ,
+the program uses a random seed.
  .PP
  .in +4n
  .\" SRC BEGIN (rand.c)
@@ -211,7 +214,11 @@ .SH EXAMPLES
      seed = atoi(argv[1]);
      nloops = atoi(argv[2]);

-    srand(seed);
+    if (seed == -1)
+        srand(src4random());
+    else
+        srand(seed);
+
      for (unsigned int j = 0; j < nloops; j++) {
          r =  rand();
          printf("%d\en", r);
@@ -223,5 +230,6 @@ .SH EXAMPLES
  .\" SRC END
  .in
  .SH SEE ALSO
+.BR arc4random (3bsd),
  .BR drand48 (3),
  .BR random (3)



-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2294 bytes --]



On 12/28/22 22:03, Alejandro Colomar wrote:
> Hi Jonny,
> 
> On 12/28/22 21:56, Alejandro Colomar wrote:
>> Hi Jonny,
>>
>> On 12/28/22 21:51, Jonny Grant wrote:
>>> You're completely right. It's a real issue if software starts multiple times 
>>> per second, or executes in less than one second and then runs again. Our 
>>> software always runs for at least minutes, maybe another code suggestion for 
>>> a seed would be good instead, like arc4random. I do like that rand() offers a 
>>> reproducible sequence, useful when in some other software we logged the seed 
>>> value used. random.4 - /dev/random would be a better seed than time(NULL) if 
>>> running the program multiple times per second. Anyway, rand() is only 
>>> pseudo-random, utilising /dev/random would be really much more random, and I 
>>> like that the seed is saved between reboots.
>>
>> Ahh, I didn't connect the dots the other day!  We don't need to wait for 
>> glibc. libbsd already provides arc4random on GNU/Linux systems, so I can 
>> already recommend using arc4random to seed srand(3).
>>
>> I'll prepare a patch...
>>
> 
> I will probably apply the following patch.  Do you have any comments?  Does it 
> provide the information you wanted to add?
> 
> Cheers,
> 
> Alex
> 
> diff --git a/man3/rand.3 b/man3/rand.3
> index 572471749..350a875d8 100644
> --- a/man3/rand.3
> +++ b/man3/rand.3
> @@ -190,6 +190,9 @@ .SH EXAMPLES
>   pseudo-random sequence produced by
>   .BR rand ()
>   when given a particular seed.
> +When the seed is
> +.IR \-1 ,
> +the program uses a random seed.
>   .PP
>   .in +4n
>   .\" SRC BEGIN (rand.c)
> @@ -211,7 +214,11 @@ .SH EXAMPLES
>       seed = atoi(argv[1]);
>       nloops = atoi(argv[2]);
> 
> -    srand(seed);
> +    if (seed == -1)
> +        srand(src4random());

Oops, typo there

> +    else
> +        srand(seed);
> +
>       for (unsigned int j = 0; j < nloops; j++) {
>           r =  rand();
>           printf("%d\en", r);
> @@ -223,5 +230,6 @@ .SH EXAMPLES
>   .\" SRC END
>   .in
>   .SH SEE ALSO
> +.BR arc4random (3bsd),
>   .BR drand48 (3),
>   .BR random (3)
> 
> 
> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Cristian Rodríguez]

On Wed, Dec 28, 2022 at 5:57 PM Alejandro Colomar via Libc-alpha
<libc-alpha@sourceware.org> wrote:

> Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc.
> libbsd already provides arc4random on GNU/Linux systems, so I can already
> recommend using arc4random to seed srand(3).
>
> I'll prepare a patch...
>

I would suggest avoiding that, as it suffers from all the problems
previously discussed on this list , on the relevant arc4random thread

tl;dr , it can't be safe without kernel support, as you need to know
when to drop the buffer. (on fork, on resume plus on $deity knows
condition for which there is no kernel notification about)

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1510 bytes --]

Hi Cristian,

On 12/28/22 22:04, Cristian Rodríguez wrote:
> On Wed, Dec 28, 2022 at 5:57 PM Alejandro Colomar via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
> 
>> Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc.
>> libbsd already provides arc4random on GNU/Linux systems, so I can already
>> recommend using arc4random to seed srand(3).
>>
>> I'll prepare a patch...
>>
> 
> I would suggest avoiding that, as it suffers from all the problems
> previously discussed on this list , on the relevant arc4random thread
> 
> tl;dr , it can't be safe without kernel support, as you need to know
> when to drop the buffer. (on fork, on resume plus on $deity knows
> condition for which there is no kernel notification about)

I don't consider rand(3) anything safe or really random, but rather a tool for 
simulations or debugging where repeatability matters more than anything else.

For anything where randomness matters, arc4random(3) is the way to go.  But 
randomness is not always what you want.  What would you recommend for some 
software where you need to test/simulate many different cases, where you want to 
be able to repeat the tests?

Maybe we should make clear that rand(3) should never be used unless you want 
repeatable results.  And when randomness matters in some cases, you could add a 
macro wrapping both rand(3) and arc4random(3), and use the appropriate one.

Cheers,

Alex


-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 997 bytes --]



On 12/28/22 22:04, Cristian Rodríguez wrote:
> On Wed, Dec 28, 2022 at 5:57 PM Alejandro Colomar via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
> 
>> Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc.
>> libbsd already provides arc4random on GNU/Linux systems, so I can already
>> recommend using arc4random to seed srand(3).
>>
>> I'll prepare a patch...
>>
> 
> I would suggest avoiding that, as it suffers from all the problems
> previously discussed on this list , on the relevant arc4random thread
> 
> tl;dr , it can't be safe without kernel support, as you need to know
> when to drop the buffer. (on fork, on resume plus on $deity knows
> condition for which there is no kernel notification about)

Maybe I'd remove the SEE ALSO reference to arc4random(3bsd) then, so that I 
don't "recommend" using it.  I'll add arc4random(3) to SEE ALSO when we have a 
manual page for the function.

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Jonny Grant]

On 28/12/2022 21:11, Alejandro Colomar wrote:
> Hi Cristian,
> 
> On 12/28/22 22:04, Cristian Rodríguez wrote:
>> On Wed, Dec 28, 2022 at 5:57 PM Alejandro Colomar via Libc-alpha
>> <libc-alpha@sourceware.org> wrote:
>>
>>> Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc.
>>> libbsd already provides arc4random on GNU/Linux systems, so I can already
>>> recommend using arc4random to seed srand(3).
>>>
>>> I'll prepare a patch...
>>>
>>
>> I would suggest avoiding that, as it suffers from all the problems
>> previously discussed on this list , on the relevant arc4random thread
>>
>> tl;dr , it can't be safe without kernel support, as you need to know
>> when to drop the buffer. (on fork, on resume plus on $deity knows
>> condition for which there is no kernel notification about)
> 
> I don't consider rand(3) anything safe or really random, but rather a tool for simulations or debugging where repeatability matters more than anything else.
> 
> For anything where randomness matters, arc4random(3) is the way to go.  But randomness is not always what you want.  What would you recommend for some software where you need to test/simulate many different cases, where you want to be able to repeat the tests?

Can generate our stream of randomness values in a file and then the software can just run with those (like a one-time pad (OTP)).
 
> Maybe we should make clear that rand(3) should never be used unless you want repeatable results.  And when randomness matters in some cases, you could add a macro wrapping both rand(3) and arc4random(3), and use the appropriate one.

Sounds reasonable to have the ability to run both ways if useful.
Jonny

Re: [PATCH] Add example to rand.3

[Jonny Grant]

On 28/12/2022 21:04, Alejandro Colomar wrote:
> 
> 
> On 12/28/22 22:03, Alejandro Colomar wrote:
>> Hi Jonny,
>>
>> On 12/28/22 21:56, Alejandro Colomar wrote:
>>> Hi Jonny,
>>>
>>> On 12/28/22 21:51, Jonny Grant wrote:
>>>> You're completely right. It's a real issue if software starts multiple times per second, or executes in less than one second and then runs again. Our software always runs for at least minutes, maybe another code suggestion for a seed would be good instead, like arc4random. I do like that rand() offers a reproducible sequence, useful when in some other software we logged the seed value used. random.4 - /dev/random would be a better seed than time(NULL) if running the program multiple times per second. Anyway, rand() is only pseudo-random, utilising /dev/random would be really much more random, and I like that the seed is saved between reboots.
>>>
>>> Ahh, I didn't connect the dots the other day!  We don't need to wait for glibc. libbsd already provides arc4random on GNU/Linux systems, so I can already recommend using arc4random to seed srand(3).
>>>
>>> I'll prepare a patch...
>>>
>>
>> I will probably apply the following patch.  Do you have any comments?  Does it provide the information you wanted to add?
>>
>> Cheers,
>>
>> Alex
>>
>> diff --git a/man3/rand.3 b/man3/rand.3
>> index 572471749..350a875d8 100644
>> --- a/man3/rand.3
>> +++ b/man3/rand.3
>> @@ -190,6 +190,9 @@ .SH EXAMPLES
>>   pseudo-random sequence produced by
>>   .BR rand ()
>>   when given a particular seed.
>> +When the seed is
>> +.IR \-1 ,
>> +the program uses a random seed.
>>   .PP
>>   .in +4n
>>   .\" SRC BEGIN (rand.c)
>> @@ -211,7 +214,11 @@ .SH EXAMPLES
>>       seed = atoi(argv[1]);
>>       nloops = atoi(argv[2]);
>>
>> -    srand(seed);
>> +    if (seed == -1)
>> +        srand(src4random());
> 
> Oops, typo there
> 
>> +    else
>> +        srand(seed);
>> +
>>       for (unsigned int j = 0; j < nloops; j++) {
>>           r =  rand();
>>           printf("%d\en", r);
>> @@ -223,5 +230,6 @@ .SH EXAMPLES
>>   .\" SRC END
>>   .in
>>   .SH SEE ALSO
>> +.BR arc4random (3bsd),
>>   .BR drand48 (3),
>>   .BR random (3)

That example program looks good, thank you for taking the time to prepare that patch.
Jonny

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1106 bytes --]

Hi,

On 12/28/22 22:25, Jonny Grant wrote:
> That example program looks good, thank you for taking the time to prepare that patch.
> Jonny

Thanks!

This is my v2:

It doesn't recommend libbsd's arc4random().
It also shows that arc4random is just a way to get a random seed, and prints it, 
but then it's as "bad" as any other run of srand(3) + rand(3).

Cheers,

Alex

diff --git a/man3/rand.3 b/man3/rand.3
index 572471749..08ad0a822 100644
--- a/man3/rand.3
+++ b/man3/rand.3
@@ -190,6 +190,9 @@ .SH EXAMPLES
  pseudo-random sequence produced by
  .BR rand ()
  when given a particular seed.
+When the seed is
+.IR \-1 ,
+the program uses a random seed.
  .PP
  .in +4n
  .\" SRC BEGIN (rand.c)
@@ -211,7 +214,13 @@ .SH EXAMPLES
      seed = atoi(argv[1]);
      nloops = atoi(argv[2]);

+    if (seed == -1) {
+        seed = arc4random();
+        printf("seed: %u\en", seed);
+    }
+
      srand(seed);
+
      for (unsigned int j = 0; j < nloops; j++) {
          r =  rand();
          printf("%d\en", r);


-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 990 bytes --]

On Wed, 28 Dec 2022, Cristian Rodríguez via Libc-alpha wrote:

> On Tue, Dec 27, 2022 at 9:41 PM Alejandro Colomar
> <alx.manpages@gmail.com> wrote:
> >
> > Hi Branden,
> >
> > On 12/28/22 01:00, G. Branden Robinson wrote:
> > > At 2022-12-28T00:33:13+0100, Alejandro Colomar wrote:
> 
> > (I used C23 syntax for rounding up to a power of 2, to avoid some magic macro.
> > That line is not necessary, but improves performance considerably for small
> > upper bounds.  I don't know if it's available already in GCC or Clang.)
> 
> at least current gcc13 does not implement N3022 yet :-| that actually
> a nice and looong needed addition (like 30 years late)

I expect to implement these functions in due course for glibc (not GCC, 
since these are library functions, with the usual requirements for 
definitions with external linkage, though most would also have inline 
header implementations based on existing longstanding built-in functions).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1286 bytes --]

Hey Joseph!

On 12/30/22 19:15, Joseph Myers wrote:
> On Wed, 28 Dec 2022, Cristian Rodríguez via Libc-alpha wrote:
> 
>> On Tue, Dec 27, 2022 at 9:41 PM Alejandro Colomar
>> <alx.manpages@gmail.com> wrote:
>>>
>>> Hi Branden,
>>>
>>> On 12/28/22 01:00, G. Branden Robinson wrote:
>>>> At 2022-12-28T00:33:13+0100, Alejandro Colomar wrote:
>>
>>> (I used C23 syntax for rounding up to a power of 2, to avoid some magic macro.
>>> That line is not necessary, but improves performance considerably for small
>>> upper bounds.  I don't know if it's available already in GCC or Clang.)
>>
>> at least current gcc13 does not implement N3022 yet :-| that actually
>> a nice and looong needed addition (like 30 years late)
> 
> I expect to implement these functions in due course for glibc (not GCC,
> since these are library functions, with the usual requirements for
> definitions with external linkage, though most would also have inline
> header implementations based on existing longstanding built-in functions).

Is there any builtin for stdc_bit_ceil()?  I've tried to search for it, but 
didn't find it.  I came to the conclusion that I need to write the ugly code 
around __builtin_clz().

Cheers,

Alex
> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Joseph Myers]

On Fri, 30 Dec 2022, Alejandro Colomar via Libc-alpha wrote:

> > I expect to implement these functions in due course for glibc (not GCC,
> > since these are library functions, with the usual requirements for
> > definitions with external linkage, though most would also have inline
> > header implementations based on existing longstanding built-in functions).
> 
> Is there any builtin for stdc_bit_ceil()?  I've tried to search for it, but
> didn't find it.  I came to the conclusion that I need to write the ugly code
> around __builtin_clz().

I don't think there's such a built-in function, so yes, the inline 
function would need to use __builtin_clz.  (Even the stdc_leading_zeros 
inline functions will need to do more than just call __builtin_clz - they 
need to check for zero, since the standard functions have a defined result 
for a zero argument whereas __builtin_clz doesn't.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [PATCH] Add example to rand.3

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 1094 bytes --]

Hi Joseph,

On 12/30/22 19:50, Joseph Myers wrote:
> On Fri, 30 Dec 2022, Alejandro Colomar via Libc-alpha wrote:
> 
>>> I expect to implement these functions in due course for glibc (not GCC,
>>> since these are library functions, with the usual requirements for
>>> definitions with external linkage, though most would also have inline
>>> header implementations based on existing longstanding built-in functions).
>>
>> Is there any builtin for stdc_bit_ceil()?  I've tried to search for it, but
>> didn't find it.  I came to the conclusion that I need to write the ugly code
>> around __builtin_clz().
> 
> I don't think there's such a built-in function, so yes, the inline
> function would need to use __builtin_clz.  (Even the stdc_leading_zeros
> inline functions will need to do more than just call __builtin_clz - they
> need to check for zero, since the standard functions have a defined result
> for a zero argument whereas __builtin_clz doesn't.)

Thanks for confirming.  That was my guess :|

Cheers,

Alex

> 

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Add example to rand.3

[Cristian Rodríguez]

On Fri, Dec 30, 2022 at 3:15 PM Joseph Myers <joseph@codesourcery.com> wrote:

> I expect to implement these functions in due course for glibc (not GCC,
> since these are library functions, with the usual requirements for
> definitions with external linkage, though most would also have inline
> header implementations based on existing longstanding built-in functions).

Unless I am seriously missing something, the compiler is required to
provide an implementation
of them as it is a freestanding header..right ?

Re: [PATCH] Add example to rand.3

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 2358 bytes --]

On Fri, 30 Dec 2022, Cristian Rodríguez via Libc-alpha wrote:

> On Fri, Dec 30, 2022 at 3:15 PM Joseph Myers <joseph@codesourcery.com> wrote:
> 
> > I expect to implement these functions in due course for glibc (not GCC,
> > since these are library functions, with the usual requirements for
> > definitions with external linkage, though most would also have inline
> > header implementations based on existing longstanding built-in functions).
> 
> Unless I am seriously missing something, the compiler is required to
> provide an implementation
> of them as it is a freestanding header..right ?

"freestanding" versus "hosted" isn't the right distinction for which 
component in a (compiler + library) implementation provides which pieces 
any more.  The actual distinction is closer to whether a feature involves 
library functions (with external linkage): if it does, it belongs in the 
library (glibc), whereas if a header doesn't involve such functions, it 
belongs in the compiler (GCC), along with the library support (in libgcc) 
for language features (i.e. library code that can be needed even by 
programs that don't include any standard library header or call any 
function from the standard library).  My list of things to be done for C2x 
in GCC includes updating the GCC documentation (standards.texi) to reflect 
this: that is, to say that GCC provides the compiler part of a 
freestanding or hosted implementation, and library support for language 
features and headers not involving external linkage functions, to be used 
together with a corresponding library implementation - not that it's a 
freestanding implementation by itself.

(It wasn't quite right for C11 either; GCC provides <stdatomic.h> and 
libatomic, and while there are a few atomic_* functions with external 
linkage, the bulk of the operations in <stdatomic.h> are only generic 
functions with no defined external-linkage functions underlying them, 
while it's been documented for a long time that a corresponding library is 
always needed to provided memcpy, memmove and memset as those may be 
generated from use of language features without being called explicitly by 
the user's code.  But C2x makes it a lot more visible that the previously 
documented rule of thumb for where a feature goes is no longer 
appropriate.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Internal organization of "the implementation" (was: [PATCH] Add example to rand.3)

[Alejandro Colomar]

[-- Attachment #1.1: Type: text/plain, Size: 2796 bytes --]

Hi Joseph,

On 12/30/22 22:08, Joseph Myers wrote:
> On Fri, 30 Dec 2022, Cristian Rodríguez via Libc-alpha wrote:
> 
>> On Fri, Dec 30, 2022 at 3:15 PM Joseph Myers <joseph@codesourcery.com> wrote:
>>
>>> I expect to implement these functions in due course for glibc (not GCC,
>>> since these are library functions, with the usual requirements for
>>> definitions with external linkage, though most would also have inline
>>> header implementations based on existing longstanding built-in functions).
>>
>> Unless I am seriously missing something, the compiler is required to
>> provide an implementation
>> of them as it is a freestanding header..right ?
> 
> "freestanding" versus "hosted" isn't the right distinction for which
> component in a (compiler + library) implementation provides which pieces
> any more.  The actual distinction is closer to whether a feature involves
> library functions (with external linkage): if it does, it belongs in the
> library (glibc), whereas if a header doesn't involve such functions, it
> belongs in the compiler (GCC), along with the library support (in libgcc)
> for language features (i.e. library code that can be needed even by
> programs that don't include any standard library header or call any
> function from the standard library).  My list of things to be done for C2x
> in GCC includes updating the GCC documentation (standards.texi) to reflect
> this: that is, to say that GCC provides the compiler part of a
> freestanding or hosted implementation, and library support for language
> features and headers not involving external linkage functions, to be used
> together with a corresponding library implementation - not that it's a
> freestanding implementation by itself.
> 
> (It wasn't quite right for C11 either; GCC provides <stdatomic.h> and
> libatomic, and while there are a few atomic_* functions with external
> linkage, the bulk of the operations in <stdatomic.h> are only generic
> functions with no defined external-linkage functions underlying them,
> while it's been documented for a long time that a corresponding library is
> always needed to provided memcpy, memmove and memset as those may be
> generated from use of language features without being called explicitly by
> the user's code.  But C2x makes it a lot more visible that the previously
> documented rule of thumb for where a feature goes is no longer
> appropriate.)
> 

Why is this duplication of responsibility for libc functions?  Why isn't there a 
smaller say libmem that provides a unique implementation for these memcpy(), 
memmove(), memset() functions, and both gcc and glibc (and even the kernel 
could, with static linking) depend on it?

Cheers,

Alex

-- 
<http://www.alejandro-colomar.es/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Internal organization of "the implementation" (was: [PATCH] Add example to rand.3)

[Joseph Myers]

On Fri, 30 Dec 2022, Alejandro Colomar via Libc-alpha wrote:

> Why is this duplication of responsibility for libc functions?  Why isn't there

There isn't duplication.  Each function is the responsibility of exactly 
one of the compiler and the libc; GCC's documentation, once updated for 
C2x, will reflect which parts it expects to provide and which it expects a 
corresponding libc to provide.

> a smaller say libmem that provides a unique implementation for these memcpy(),
> memmove(), memset() functions, and both gcc and glibc (and even the kernel
> could, with static linking) depend on it?

The design follows from the traditional structure of libc on Unix systems 
(containing both functions closely related to the OS and functions that 
only depend on the CPU architecture and not the OS) and GCC's early role 
as an alternative to the system compiler there.  It's possible to 
structure libraries in lots of different ways (I don't know if BSD systems 
actually share parts of libc with the kernel, but they certainly tend to 
have closer integration of code across the whole system, for example).  
Non-C-family languages are probably a better place for exploring the 
design space of different divisions among compilers and libraries, than C 
libraries that are expected to fit with existing tooling, build systems 
and operating systems (and certainly the extreme case of a very large 
number of separately maintained tiny libraries in some such language 
environments shows up its own problems with dependency management).

In practice, while there's sometimes sharing at the level of a third-party 
source code repository providing code that can be used in multiple places 
(see Arm's optimized routines, for example), including code in shared 
libraries and using features such as IFUNC, along with e.g. some objects 
having multiple entry points and other libc-internal symbol handling 
requiring close coordination among the build of all linked objects, makes 
it hard to share at the object code level.  And we've tended lately to 
move *away* from separate small shared libraries - rather, integrating 
formerly separate libraries such as libpthread into the single shared 
libc.

-- 
Joseph S. Myers
joseph@codesourcery.com