[Bug c++/107660] New: Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[Bug c++/107660] New: Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[t3suzuki at ucsd dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

            Bug ID: 107660
           Summary: Running binaries compiled with g++11 or later produces
                    different results than g++ version 10 or earlier
           Product: gcc
           Version: 12.2.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c++
          Assignee: unassigned at gcc dot gnu.org
          Reporter: t3suzuki at ucsd dot edu
  Target Milestone: ---

[gcc version]
$ ./g++ -v
Using built-in specs.
COLLECT_GCC=./g++
COLLECT_LTO_WRAPPER=/home/tomoya-s/gcc-12.2.0/install/libexec/gcc/x86_64-pc-linux-gnu/12.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ./configure --prefix=/home/tomoya-s/gcc-12.2.0/install
--disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (GCC)

[the command line that triggers the bug]
% git clone https://github.com/sbeamer/gapbs.git
% cd gapbs
% make bfs
% ./bfs -g 10 -n 1

[the output by using g++-12.2.0 (or g++-11) ]
Generate Time:       0.00282
Build Time:          0.00044
Graph has 1024 nodes and 10496 undirected edges for degree: 10
Source:                  204
    i                0.00001
   td          2     0.00000
   td         59     0.00000
    e                0.00000
   bu        711     0.00003
   bu        123     0.00001
   bu          0     0.00001
    c                0.00000
Trial Time:          0.00017
Average Time:        0.00017

[the output by using g++-10 or earlier or clang++]
Generate Time:       0.00797
Build Time:          0.00102
Graph has 1024 nodes and 10496 undirected edges for degree: 10
Source:                  204
    i                0.00001
   td         15     0.00001
    e                0.00000
   bu        615     0.00004
   bu        264     0.00001
   bu          1     0.00001
    c                0.00000
   td          0     0.00000
Trial Time:          0.00017
Average Time:        0.00017

Of course, the execution time is different.

We expect the numbers 2 and 59 and 711 after td and bu to be the same.
This is because the input and the process itself are the same, and there is no
randomness inside the process (random seed is fixed).

Note that this program uses OpenMP, but even if we disable it, the results will
be different depending on the g++ version as well.

[Bug c++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Last reconfirmed|                            |2022-11-12
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |WAITING

--- Comment #1 from Jonathan Wakely <redi at gcc dot gnu.org> ---
As requested when creating the bug, please read https://gcc.gnu.org/bugs/ and
provide the missing information.

You're asking us to download and analyze a whole codebase we've never seen
before, which means your bug report will probably not get much attention.

[Bug c++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[t3suzuki at ucsd dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #2 from Tomoya Suzuki <t3suzuki at ucsd dot edu> ---
I Sorry I didn't follow your bug report guidelines. After this, I investigated
the cause by myself, and it seems that there is a difference in the output of
the std::shuffle function. Even if the input and random number configurations
of the shuffle function are the same, can the output differ depending on the
gcc version?

[Bug c++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[t3suzuki at ucsd dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #3 from Tomoya Suzuki <t3suzuki at ucsd dot edu> ---
Perhaps I should ask the question in libc instead of gcc?

[Bug c++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #4 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
std::mt19937

[Bug libstdc++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #5 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
(In reply to Andrew Pinski from comment #4)
> std::mt19937

r11-4535-g822c1d21a3c710

Hmm, this might be expected ....

[Bug libstdc++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #6 from Jonathan Wakely <redi at gcc dot gnu.org> ---
It's expected and required by the standard that std::mt19937 produces the same
numbers every time. And if I run a test for GCC 10 and GCC 11 I see the same
number produced for the first 2.14 billion results.

[Bug libstdc++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[t3suzuki at ucsd dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #7 from Tomoya Suzuki <t3suzuki at ucsd dot edu> ---
The following code appears to me to work differently with gcc10 and gcc11.

#include <vector>
#include <algorithm>
#include <random>

int kRandSeed = 27491095;

int
main()
{
  const int n = 16384;
  std::vector<int> v(n);
  std::mt19937 rng(kRandSeed);
  //std::minstd_rand rng;

  for (int i=0; i < n; i++)
    v[i] = i;

  std::shuffle(v.begin(), v.end(), rng);

  for (int i=0; i < n; i++)
    printf("%d\n", v[i]);
}

[Bug libstdc++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |UNCONFIRMED
     Ever confirmed|1                           |0

--- Comment #8 from Jonathan Wakely <redi at gcc dot gnu.org> ---
Thanks for the reduced example code, I can see what the problem is now.

The mt19937 engine produces consistent results, but the commit that Andrew
linked to alters the output of std::uniform_int_distribution, due to a
different (much faster) algorithm used to downscale the random bits produced by
std::mt19937 to the number of elements being shuffled. The only requirement on
the uniform_int_distribution is that it produces values uniformly distributed
between the minimum and maximum values. Unlike std::mt19937 it doesn't have to
produce a specific sequence of values.

I supposed we could add a macro that disables the new fast implementation, for
users who prefer to get the same sequence of values as GCC 10, even though it's
slower.

[Bug libstdc++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[t3suzuki at ucsd dot edu]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

--- Comment #9 from Tomoya Suzuki <t3suzuki at ucsd dot edu> ---
Thanks for clarifying the cause. I don't think you need to add the macro just
for me, gcc11 behavior is not critical for me.

[Bug libstdc++/107660] Running binaries compiled with g++11 or later produces different results than g++ version 10 or earlier

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107660

Jonathan Wakely <redi at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |WONTFIX

--- Comment #10 from Jonathan Wakely <redi at gcc dot gnu.org> ---
OK, thanks for clarifying. Let's close this as WONTFIX then, but I might
revisit it in future if others need stable results from the distribution.