[Bug c/115345] New: [12/13/14 REGRESSION] / Different outputs compared to GCC 11- and MSVC/Clang

[Bug c/115345] New: [12/13/14 REGRESSION] / Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

            Bug ID: 115345
           Summary: [12/13/14 REGRESSION] / Different outputs compared to
                    GCC 11- and MSVC/Clang
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: djordje.baljozovic at ac dot rwth-aachen.de
  Target Milestone: ---

Created attachment 58344
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58344&action=edit
Crash report

Our team works on a large code base written primarily in C:
https://github.com/IUPAC-InChI/InChI .
After compiling the project with MSVC, GCC and Clang/LLVM, we noticed that
there are discrepancies in outputs when the project is compiled with GCC
versions 14.0-14.1/13.0-13.3/12.0-12.4 -- in short GCC versions 12+.
The issue occurs when the GCC 12+ compiled project cannot process 10 specific
inputs (.sdf/.mol files) which are processed without any problems with GCC
11-/MSVC/Clang.
After running dbg with one of these inputs, we got a crash report which I am
attaching to this issue (file name: inchi_stacktrace_core_dump.txt).

We had put some "control" code in strutil.c which creates an output file
(gcc_crash_report.txt) with relevant values of all variables within this
function:

...
#if (GCC_DEBUG)
    FILE* gcc_fptr;
    gcc_fptr = fopen("gcc_crash_report.txt", "w");
    if (!gcc_fptr)
    {
        printf("\n>>>>>Error opening gcc_crash_report.txt file!<<<<<\n");
    }
#endif

    for (j = 0; j < num_at; j++)
    {
        if (!nNewCompNumber[j])
        {
            /* mark starting with at[j] */
            int fst_at = 0, nxt_at = 0, cur_at = j;
            num_components++;

            /* first time at at[j] */
            fst_at = cur_at;
            nNewCompNumber[fst_at] = (AT_NUMB) num_components;

            bool cur_neq_fst = true;

            /* find next neighbor */
            do
            {
#if (GCC_DEBUG)
                fprintf(gcc_fptr, "\ncur_at = %d, nxt_at = %d, iNeigh[cur_at] =
%d, at[cur_at].val = %d ", cur_at, nxt_at, iNeigh[cur_at], at[cur_at].valence);
#endif
                if (iNeigh[cur_at] < at[cur_at].valence)
                {
                    int ineigh_incr = (int)iNeigh[cur_at];
                    nxt_at = at[cur_at].neighbor[ineigh_incr];
                    iNeigh[cur_at]++;
#if (GCC_DEBUG)
                    fprintf(gcc_fptr, "/ ineigh_incr = %d, new_nxt_at = %d,
iNeigh[cur_at]_inc = %d, nNewCompNumber[nxt_at] = %d ", ineigh_incr, nxt_at,
iNeigh[cur_at], nNewCompNumber[nxt_at]);
#endif

                    if (!nNewCompNumber[nxt_at])
                    {
                        /* forward edge: found new atom */
                        nNewCompNumber[nxt_at] = (AT_NUMB) num_components;
                        nPrevAtom[nxt_at] = (AT_NUMB) cur_at;
                        cur_at = nxt_at;
                    }
                }
                else if (cur_at == fst_at)
                {
                    cur_neq_fst = false;
                    /* break;  done */
                }
                else
                {
                    cur_at = nPrevAtom[cur_at]; /* retract */
                }
            } while (cur_neq_fst);
#if (GCC_DEBUG)
            fprintf(gcc_fptr, "\n");
#endif
        }
    }

#if (GCC_DEBUG)
    if (gcc_fptr)
    {
        fclose(gcc_fptr);
    }
#endif
...

MSVC, Clang and GCC 11- compiled project and GCC 12+ compiled project produce
totally different gcc_crash_report.txt contents for the same input .mol/.sdf
file.
This example shows that the variables in question get totally different values
when compiled with GCC 12+, or, in some cases, do not get values at all.

The described issue seems to be indicating an existence of a very serious bug,
which needs to be fixed immediately.

At this moment, we are recommending our users to compile the code from the
source using GCC 11- or MSVC/Clang, but this is obviously not a solution.

I can give some examples but only one attachment is allowed here.
Please feel free to let me know if any further 

Thank you in advance for your assistance.
George

[Bug c/115345] [12/13/14 REGRESSION] / Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                URL|                            |https://github.com/IUPAC-In
                   |                            |ChI/InChI

--- Comment #1 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
I am attaching an example of the differences in gcc_crash_report.txt for the
input file sid_453841144.sdf in a separate attachment.

[Bug c/115345] [12/13/14 REGRESSION] / Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #2 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
Created attachment 58345
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58345&action=edit
Example of differences in output

[Bug c/115345] [12/13/14 REGRESSION] / Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #3 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Djordje Baljozovic from comment #1)
> I am attaching an example of the differences in gcc_crash_report.txt for the
> input file sid_453841144.sdf in a separate attachment.

See attachment below.

[Bug c/115345] [12/13/14 REGRESSION] / Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #4 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Djordje Baljozovic from comment #3)
> (In reply to Djordje Baljozovic from comment #1)
> > I am attaching an example of the differences in gcc_crash_report.txt for the
> > input file sid_453841144.sdf in a separate attachment.
> 
> See attachment below.

Now it's above :) .

[Bug c/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2024-06-04
            Summary|[12/13/14 REGRESSION] /     |[12/13/14/15 Regression]
                   |Different outputs compared  |Different outputs compared
                   |to GCC 11- and MSVC/Clang   |to GCC 11- and MSVC/Clang
             Status|UNCONFIRMED                 |WAITING
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Please see https://gcc.gnu.org/bugs/ for what we need, we need a self-contained
preprocessed testcase, which you haven't provided.
You should start by compiling the code with -fsanitize=undefined and/or
-fsanitize=address, see if it reports anything, if not, create a minimal
self-contained reproducer.

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #6 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Jakub Jelinek from comment #5)
> Please see https://gcc.gnu.org/bugs/ for what we need, we need a
> self-contained preprocessed testcase, which you haven't provided.
> You should start by compiling the code with -fsanitize=undefined and/or
> -fsanitize=address, see if it reports anything, if not, create a minimal
> self-contained reproducer.

Hi,
You are pointing me to use fuzzer for our code, which is already done by
Google's oss-fuzz project.
No issue in the code has been detected in the part of the code I sent.

Also, the code itself is not an issue -- it is the GCC executables generated
from the very same code that make all the differences, i.e. if the project/code
is compiled with GCC 12-14 on one side and GCC 0-11, MSVC (all versions) and
Clang (all versions) on the other.

Since the code base is rather big, and all functions are intertwined, the only
thing I can do is:
1. attach EXE/ELF files and one of the input files which causes the problem
2. create two Docker containers -- one with GCC 11 and one with GCC 12/13/14.
3. I can send you instructions how to compile one part of the project from our
GitHub repo.

As mentioned in my initial post, gdb crash report is given in the attachment
https://gcc.gnu.org/bugzilla/attachment.cgi?id=58344 .

Hope this helps.
Sincerely,
George

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |wrong-code

--- Comment #7 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
A few questions, does `-fsanitize=undefined -fsanitize=address` report
anything? Does it work at -O0 and not just -O3? Does adding
-fno-strict-aliasing to the command line "fix" the crash? Are there any
warnings with `-Wextra -Wall` that might be causing an issue?

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #8 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Also, if building with GCC 11 works and with GCC 12 doesn't, can you build two
trees, one with either compiler and then bisect first among the shared
libraries or binaries (find out which shared library or binary is affected),
among the static libraries (GCC 11 and 12 ought to be ABI compatible, so try
linking some *.a libraries from one build and other from another build, among
the object files (start with linking half of the objects built by one compiler
and another half by another compiler and depending on if it works as expected
or not change the set of objects to link).
If you get to one object file which when compiled with GCC 11 works even when
everything else has been compiled with GCC 12 or vice versa, you can then try
different options on that one (e.g. the -fno-strict-aliasing Andrew mentioned,
-O0 etc.), and/or
bisect inside of the source using #pragma GCC optimize(0) etc.
Anyway, if there is a single translation unit like that and ideally if you find
out which function is problematic, trying to stub functions it calls and add a
caller which calls that single function with the right arguments turns it into
a self-contained reproducer.

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #9 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Andrew Pinski from comment #7)
> A few questions, does `-fsanitize=undefined -fsanitize=address` report
> anything? Does it work at -O0 and not just -O3? Does adding
> -fno-strict-aliasing to the command line "fix" the crash? Are there any
> warnings with `-Wextra -Wall` that might be causing an issue?

Have not tested -O0 and -fno-strict-aliasing; will let you know if this fixed
the problem.
No warnings with -Wextra -Wall to my knowledge.

Sincerely,
George

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #10 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Jakub Jelinek from comment #8)
> Also, if building with GCC 11 works and with GCC 12 doesn't, can you build
> two trees, one with either compiler and then bisect first among the shared
> libraries or binaries (find out which shared library or binary is affected),
> among the static libraries (GCC 11 and 12 ought to be ABI compatible, so try
> linking some *.a libraries from one build and other from another build,
> among the object files (start with linking half of the objects built by one
> compiler and another half by another compiler and depending on if it works
> as expected or not change the set of objects to link).
> If you get to one object file which when compiled with GCC 11 works even
> when everything else has been compiled with GCC 12 or vice versa, you can
> then try different options on that one (e.g. the -fno-strict-aliasing Andrew
> mentioned, -O0 etc.), and/or
> bisect inside of the source using #pragma GCC optimize(0) etc.
> Anyway, if there is a single translation unit like that and ideally if you
> find out which function is problematic, trying to stub functions it calls
> and add a caller which calls that single function with the right arguments
> turns it into a self-contained reproducer.

Good idea(s). Let me try all Andrew's options firstly, and get back to you.
Thanks,
George

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #11 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Djordje Baljozovic from comment #9)
> (In reply to Andrew Pinski from comment #7)
> > A few questions, does `-fsanitize=undefined -fsanitize=address` report
> > anything? Does it work at -O0 and not just -O3? Does adding
> > -fno-strict-aliasing to the command line "fix" the crash? Are there any
> > warnings with `-Wextra -Wall` that might be causing an issue?
> 
> Have not tested -O0 and -fno-strict-aliasing; will let you know if this
> fixed the problem.
> No warnings with -Wextra -Wall to my knowledge.
> 
> Sincerely,
> George

Hi Andrew and Jakub,
The results are more than interesting:

1. -fno-strict-aliasing: none of the inputs processed (with O3)
2. O0: all but one input processed
3. O3: none of the inputs processed
4. O1 and O2: all inputs processed without any issues -- this did it.

Now the question is: how on Earth did O1/O2 do the trick, and not O0?!

Once again, thanks a lot for your detailed and quick responses.
George
P.S. I will keep @Jakub's bisect idea in mind if something like this happens in
the future.

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #12 from Richard Biener <rguenth at gcc dot gnu.org> ---
(In reply to Djordje Baljozovic from comment #11)
> (In reply to Djordje Baljozovic from comment #9)
> > (In reply to Andrew Pinski from comment #7)
> > > A few questions, does `-fsanitize=undefined -fsanitize=address` report
> > > anything? Does it work at -O0 and not just -O3? Does adding
> > > -fno-strict-aliasing to the command line "fix" the crash? Are there any
> > > warnings with `-Wextra -Wall` that might be causing an issue?
> > 
> > Have not tested -O0 and -fno-strict-aliasing; will let you know if this
> > fixed the problem.
> > No warnings with -Wextra -Wall to my knowledge.
> > 
> > Sincerely,
> > George
> 
> Hi Andrew and Jakub,
> The results are more than interesting:
> 
> 1. -fno-strict-aliasing: none of the inputs processed (with O3)
> 2. O0: all but one input processed
> 3. O3: none of the inputs processed
> 4. O1 and O2: all inputs processed without any issues -- this did it.
> 
> Now the question is: how on Earth did O1/O2 do the trick, and not O0?!

Can you check whether -O0 works with the other compilers?  It feels like
you might be triggering some undefined behavior in your code.

If you have a short running example that breaks with -O0 it might be
also interesting to run it through valgrind to spot use-after-free
or uninitialized use issues.

> Once again, thanks a lot for your detailed and quick responses.
> George
> P.S. I will keep @Jakub's bisect idea in mind if something like this happens
> in the future.

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #13 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Richard Biener from comment #12)
> (In reply to Djordje Baljozovic from comment #11)
> > (In reply to Djordje Baljozovic from comment #9)
> > > (In reply to Andrew Pinski from comment #7)
> > > > A few questions, does `-fsanitize=undefined -fsanitize=address` report
> > > > anything? Does it work at -O0 and not just -O3? Does adding
> > > > -fno-strict-aliasing to the command line "fix" the crash? Are there any
> > > > warnings with `-Wextra -Wall` that might be causing an issue?
> > > 
> > > Have not tested -O0 and -fno-strict-aliasing; will let you know if this
> > > fixed the problem.
> > > No warnings with -Wextra -Wall to my knowledge.
> > > 
> > > Sincerely,
> > > George
> > 
> > Hi Andrew and Jakub,
> > The results are more than interesting:
> > 
> > 1. -fno-strict-aliasing: none of the inputs processed (with O3)
> > 2. O0: all but one input processed
> > 3. O3: none of the inputs processed
> > 4. O1 and O2: all inputs processed without any issues -- this did it.
> > 
> > Now the question is: how on Earth did O1/O2 do the trick, and not O0?!
> 
> Can you check whether -O0 works with the other compilers?  It feels like
> you might be triggering some undefined behavior in your code.
> 
> If you have a short running example that breaks with -O0 it might be
> also interesting to run it through valgrind to spot use-after-free
> or uninitialized use issues.
> 
> > Once again, thanks a lot for your detailed and quick responses.
> > George
> > P.S. I will keep @Jakub's bisect idea in mind if something like this happens
> > in the future.

Hi Richard,
Yes, I think that testing O0 with other compilers makes sense.
BTW, do you think that running the code through valgrind can identify some
issues which are not detected by fuzzers?
Thanks,
George

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[sjames at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

Sam James <sjames at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sjames at gcc dot gnu.org

--- Comment #14 from Sam James <sjames at gcc dot gnu.org> ---
It's worth keeping in mind:
* Fuzzers are distinct from sanitizers (fuzzers run with random or ~random
input and often use sanitizers to help find issues)
* Fuzzers might not be hitting the code you're having trouble with
* The implementations of sanitizers in gcc vs clang are different (and AFAIK
oss-fuzz uses clang). They share libsanitizer but when to add instrumentation
varies.
* Valgrind can find things that some sanitizers can't.

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

--- Comment #15 from Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> ---
(In reply to Sam James from comment #14)
> It's worth keeping in mind:
> * Fuzzers are distinct from sanitizers (fuzzers run with random or ~random
> input and often use sanitizers to help find issues)
> * Fuzzers might not be hitting the code you're having trouble with
> * The implementations of sanitizers in gcc vs clang are different (and AFAIK
> oss-fuzz uses clang). They share libsanitizer but when to add
> instrumentation varies.
> * Valgrind can find things that some sanitizers can't.

Hi Sam,
Thanks for this useful information.
To be perfectly honest, I did not even run address/unknown GCC sanitisers as I
thought that Clang and GCC sanitizers are identical (I can confirm that
oss-fuzz uses Clang sanitisers).

Will try Valgrind as I have already detected a share of issues with this
inherited code just by reading it, which did not show up in any
fuzzers'/sanitisers' reports.
Thanks,
George

[Bug middle-end/115345] [12/13/14/15 Regression] Different outputs compared to GCC 11- and MSVC/Clang

[djordje.baljozovic at ac dot rwth-aachen.de]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=115345

Djordje Baljozovic <djordje.baljozovic at ac dot rwth-aachen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED