[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[debian-gcc at lists dot debian dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366


debian-gcc at lists dot debian dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |wrong-code

[Bug optimization/11366] New: [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[debian-gcc at lists dot debian dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366

           Summary: [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key
                    generation and/or validation in XFree86 at -O2
           Product: gcc
           Version: 3.3
            Status: UNCONFIRMED
          Severity: critical
          Priority: P2
         Component: optimization
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: debian-gcc at lists dot debian dot org
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: i386-linux / powerpc-linux
  GCC host triplet: i386-linux / powerpc-linux
GCC target triplet: i386-linux / powerpc-linux

This is not a complete bug report. Any hints and help to track down this report
are welcome.

[forwarded from http://bugs.debian.org/196090]

Rechecked with 3.3 CVS 20030627, with HEAD xfree86 does finish the build due to
some ICE. Setting the optimization to -O2 -fno-strict-aliasing doesn't help
either. Compiling with -O1 doesn't show the error. Seen at least on i386-linux
and powerpc-linux.

The symptom can be found in /var/log/xdm.log:
AUDIT: Fri Jun 27 20:28:48 2003: 18641 X: client 2 rejected from local host
  Auth name: XDM-AUTHORIZATION-1 ID: -1
Xlib: connection to ":0.0" refused by server
Xlib: Invalid XDM-AUTHORIZATION-1 key (failed key comparison)
/usr/X11R6/bin/xrdb: Can't open display ':0'


Bug submitter writes:

When testing XFree86 4.2.1-7 on my PowerPC I found that compiling the
tree with -O2 (per Debian Policy) and GCC 3.3 caused the X server to
break such that XDM-AUTHORIZATION-1 Xauth tokens could not be validated.

I have not narrowed this down.  I do know that using the 4.2.1-6 server
with the 4.2.1-7 libraries worked fine, so I suspect a miscompilation on
the server side, but it is conceivable that the problem is really on the
client/library side and that GCC 3.3 is exposing a very subtle error in
the code.  Or something like that.

Compiling with -O instead of -O2 made the problem go away, and that's
the optimization level that was used for the 4.2.1-7 packages for
PowerPC which were uploaded.

Anyway, I have heard rumors that the GCC team knows that there are
problems with this optimization level on PowerPC in GCC 3.3.  Something
about a bunch of new subtargets being added to the ppc/ELF backend in
this release?

Please nag me for more information if too much time passes and I have
not followed up.

Alternatively some person might be willing to track this down for me
using the recipe above.  Just take the 4.2.1-7 source package and remove
the special case optimization stepdown for powerpc from the debian/rules
file.

Compile XFree86, install the resulting packages, and use xdm to
(attempt) to start a session.  You'll also notice that the bustage keeps
the xdm greeter from working properly.


Jack Howard adds:
   Franz Sirl, the ppclinux devtool maintainer, has kindly
built the current rawhide XFree86 4.3.0 srpms against 
gcc-3.3 from 6/18/03 on Yellow Dog Linux. He is unable to
reproduce the authentication problems we see on debian
under either xdm or kdm. Perhaps we build XFree86 differently
enough from RedHat that only we tickled the gcc bug. However
I did find out that the gcc 3.3 that RedHat uses is not
a patched version of the gcc 3.3 branch but rather a special
rh branch of gcc-3.3. One wonders if they would see the
problem if they used the stock gcc-3.3 branch instead and
if there are any fixes trapped in the gcc-3.3 rh branch that
haven't been put into the main gcc-3.3 branch.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[reichelt at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From reichelt at gcc dot gnu dot org  2003-07-04 11:38 -------
Might be related to PR 11198.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[doko at cs dot tu-berlin dot de]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From doko at cs dot tu-berlin dot de  2003-07-09 07:31 -------
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

reichelt at gcc dot gnu dot org writes:
> Might be related to PR 11198.

No. Rechecked with 3.3 CVS 20030708.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[sirl at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From sirl at gcc dot gnu dot org  2003-07-14 18:17 -------
Out of interest, since you are still using XFree86-4.2.1, have you made sure 
you compiled XFree86 with -fno-strict-aliasing, a thing that is done 
automatically since 4.3.0?

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[james at nocrew dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From james at nocrew dot org  2003-07-14 18:44 -------
Subject: Re:  [3.3 regression] miscompiles
 XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

"sirl at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:

> ------- Additional Comments From sirl at gcc dot gnu dot org  2003-07-14 18:17 -------
> Out of interest, since you are still using XFree86-4.2.1, have you made sure 
> you compiled XFree86 with -fno-strict-aliasing, a thing that is done 
> automatically since 4.3.0? 

Yes, I tried this; it doesn't help.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[sirl at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From sirl at gcc dot gnu dot org  2003-07-14 19:14 -------
I have no other idea then, 4.2 already had -fno-merge-constants and I can't 
think of another promising option to try. Maybe -fno-strength-reduce? 
Otherwise you have to narrow down the bugreport to a single miscompiled file, 
we won't be able to help you otherwise.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[doko at cs dot tu-berlin dot de]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From doko at cs dot tu-berlin dot de  2003-07-15 05:46 -------
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

sirl at gcc dot gnu dot org writes:
> I have no other idea then, 4.2 already had -fno-merge-constants and
> I can't think of another promising option to try. Maybe
> -fno-strength-reduce? Otherwise you have to narrow down the
> bugreport to a single miscompiled file, we won't be able to help you
> otherwise.

Daniel, Branden,

I see, that on Debian only -O2 is used, not any of the mentioned
flags.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[pinskia at physics dot uc dot edu]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From pinskia at physics dot uc dot edu  2003-07-16 16:08 -------
Can you also try with -fno-gcse, since there was a bug in gcse where it would miscompile 
a bitfields.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[pme at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366


pme at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pme at gcc dot gnu dot org


------- Additional Comments From pme at gcc dot gnu dot org  2003-07-17 02:52 -------
Ideally, what we'd really need from the Debian team are:

1) the file that gets miscompiled, and
2) the -f flag which causes it

(1) is essential.  (2) would be nice but not crucial.  The "easy" way to
find (2) is to compile with "-fFOO" where FOO is one of the flags
turned on by -O2, as listed in the manual.  Iterate over the list until
you find the FOO that causes the breakage.  Building with "-O2 -fno-FOO"
should give you all the /other/ benefits as well as working code.

This goes much faster, of course, if you can identify (1), then you only
have to recompile that file.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[pinskia at physics dot uc dot edu]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366


pinskia at physics dot uc dot edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.4                         |3.3.1


------- Additional Comments From pinskia at physics dot uc dot edu  2003-07-17 03:08 -------
I would also use -W -Wall and see if there is any warnings that could be causing this.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[mmitchel at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366


mmitchel at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|3.3.1                       |3.3.2


------- Additional Comments From mmitchel at gcc dot gnu dot org  2003-07-20 00:46 -------
Until we have a test case for this PR, there is nothing that we can do.

Therefore, I've postponed it until GCC 3.3.2.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[branden at debian dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From branden at debian dot org  2003-07-24 18:49 -------
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

On Tue, Jul 15, 2003 at 07:45:02AM +0200, Matthias Klose wrote:
> sirl at gcc dot gnu dot org writes:
> > I have no other idea then, 4.2 already had -fno-merge-constants and
> > I can't think of another promising option to try. Maybe
> > -fno-strength-reduce? Otherwise you have to narrow down the
> > bugreport to a single miscompiled file, we won't be able to help you
> > otherwise.
> 
> Daniel, Branden,
> 
> I see, that on Debian only -O2 is used, not any of the mentioned
> flags.

Can someone give me an idea of what the proper followup to this message
might be?  Is more information required?

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[pme at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From pme at gcc dot gnu dot org  2003-07-24 19:01 -------
> Can someone give me an idea of what the proper followup to this message
> might be?  Is more information required?

Yes, more information is required.  See comment #8 and comment #9.

The "other mentioned flags" are the flags mentioned in comments 3, 4, 5, and 7.
I believe those are a subset of the possible ones mentioned in comment 8.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[debian-gcc at lists dot debian dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From debian-gcc at lists dot debian dot org  2003-08-13 07:45 -------
It looks like this is fixed with Jim Wilson's alias.c patch(?), at least I am
unable to reproduce the failure on the 20030812 3.3 branch.

Checked on i386-linux, need to recheck on powerpc.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[dhazeghi at yahoo dot com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366


dhazeghi at yahoo dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING


------- Additional Comments From dhazeghi at yahoo dot com  2003-08-25 00:40 -------
Any news about this on powerpc? Thanks.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[branden at debian dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366



------- Additional Comments From branden at debian dot org  2003-09-02 09:11 -------
Subject: Re:  [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

This problem appears to have been fixed by:

gcc-3.3 (1:3.3.2ds0-0pre0) unstable; urgency=medium

  * Update to gcc-3.3.2 CVS 20030812.
    - Fixes generation of wrong code for XDM-AUTHORIZATION-1 key generation
      and/or validation. Closes: #196090.
  * Update NEWS files.
  * Change ix86 default CPU type for code generation:
    - i386-linux       -> i486-linux
    - i386-gnu         -> i586-gnu
    - i386-freebsd-gnu -> i486-freebsd-gnu
    Use -march=i386 to target i386 CPUs.

 -- Matthias Klose <doko@debian.org>  Tue, 12 Aug 2003 10:31:28 +0200

Thanks for fixing it.

[Bug optimization/11366] [3.3 regression] miscompiles XDM-AUTHORIZATION-1 key generation and/or validation in XFree86 at -O2

[pinskia at gcc dot gnu dot org]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11366


pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |DUPLICATE


------- Additional Comments From pinskia at gcc dot gnu dot org  2003-09-02 22:46 -------
Thanks for filing this bug, I am going to mark this as a dup of bug 11319 which is meta bug for 
the aliasing bugs.

*** This bug has been marked as a duplicate of 11319 ***