Re: RNG question

[Fabian Bastin <fabian.bastin@gmail.com>]

Just a comment. When testing random numbers generators, you should 
consider TestU01: http://www.iro.umontreal.ca/~simardr/testu01/tu01.html

This software is more recent than Diehard, has a more exhaustive list of 
generators and statistical tests, so it is reasonable to expect that 
some users would like to use it instead of Diehard. Just consider this 
citation: "It should be obvious that TESTU01 has supplanted DIEHARD, and 
TESTU01 is now the. standard for testing uniform RNGs" (B. D. 
McCullough, A review of TESTU01, Journal of Applied Econometrics 21(5), 
2006).

IMHO, in view of current research in this field, it is clear that new 
rngs will be proposed, so the list cannot be frozen (for instance, why 
not include the generator MRG32K3A of L'Ecuyer, as it is a well know 
reference, cited for instance by Law in his book Simulation Modeling and 
Analysis?), and not be limited to some test code since new software can 
be proposed at any time.

Therefore, I think that any arbitrary limitation should be avoided, 
except no reasonable alternative exists.

Fabian

Robert G. Brown wrote:
> I'm preparing to address an rng testing issue in dieharder to fix the
> following bug/feature.  dieharder tests all the gsl generators as
> embedded tests (making it easy for end users to study all its generators
> and assess suitability for various tasks or demonstrate the weaknesses
> of the classical weak generators).  It then uses the gsl harness to add
> more rngs to test -- it is actually quite easy to wrap any new candidate
> rng in the gsl rng format and just add it to the list that can be
> invoked to directly test as opposed to test via file input (which is
> much more limited).
> 
> However, when I add my own gsl rng types with a loop at the end of the
> gsl-provided list, they pick up sequential numbers.  This means that if
> any new gsl routines are added (as has happened a few times) all the
> non-gsl routines have their numbers bumped.  If I add or rearrange any
> of my own (since some of them group by type, it makes more sense to keep
> them "together" in number-space) this also can change the number of
> existing rngs.
> 
> I didn't view this as a problem during design, and the default behavior
> of dieharder without arguments is to spit out a list of all its known
> rngs just so people could see what was what.  HOWEVER, users have
> started to make scripts that tests certain rngs, by number, and when the
> numbers bump as described above it breaks their scripts.
> 
> There are obviously several ways I can fix it so this never happens
> again, but I thought before I implemented any of them I'd ask at least
> if it is now expected that the rngs in the gsl are "frozen", or if there
> is an ongoing possibility that more will be added?  I'm guessing the
> latter -- if somebody invents a really great one you can hardly not
> include it in the GSL, and I've got a few that I might be able to
> contribute back eventually as well, if there is any interest.
> 
> The other question I have is that at one point in time the maximum
> number of rngs one could have was restricted by a macro in the sources
> to be 100 (if I recall correctly -- I have a remark to that effect in my
> own code's comments).  One solution to the dilemma above is to create
> "ranges" of numbers -- 0-99 for gsl rngs, 100-199 for dieharder added
> rngs, 200+ for user added rngs.  This would, however, require that the
> macro/variable's value be bumped in the gsl to maybe 500 or 1000.
> Otherwise I think it is not impossible that dieharder will exhaust the
> current gsl space in the next few years, as people keep contributing new
> rngs at a slow but steady pace, and I've got a small stack of them
> standing by to add when I next get a chance.
> 
> Comments?  Answers?
> 
>    rgb
>