public inbox for gcc-bugs@sourceware.org
help / color / mirror / Atom feed
* [Bug target/39118]  New: x86_64 red zone violation
@ 2009-02-06  5:52 ian at airs dot com
  2009-02-06  5:53 ` [Bug target/39118] " ian at airs dot com
                   ` (26 more replies)
  0 siblings, 27 replies; 28+ messages in thread
From: ian at airs dot com @ 2009-02-06  5:52 UTC (permalink / raw)
  To: gcc-bugs

This is a bug report for gcc 4_3 branch.  I will attach a test case, slightly
reduced from zlib code.  When compiling this test case for the x86_64-linux
target with -O2 -fomit-frame-pointer, I see this at the start of the function:

adler32:
        pushq   %rbp
        movq    %rdi, %rax
        andl    $65535, %edi
        shrq    $16, %rax
        movq    %rsp, %rbp
        pushq   %r15
        andl    $65535, %eax
        movl    %edx, -140(%rbp)

After %rsp was copied to %rbp, %r15 was pushed.  So now %rsp is 8 bytes less
than %rbp.  The red zone is 128 bytes, so this means that any reference down to
-136(%rbp) is valid.  However, the code actually stores a value into
-140(%rbp).  This is invalid, and can cause subtle and unpredictable bugs
depending upon when the kernel interrupts this code.

This error happens because the scheduler moves this movl instruction ahead of
some other pushq instructions.  If all the pushq instructions happened first,
the stack pointer would have been decremented sufficiently that the reference
to -140(%rbp) would be safely within the red zone.

4.4 generates completely different, and seemingly better, code for this test
case.  However, I don't see anything in the 4.4 source code which would prevent
this same bug from occurring given the right test case.  So this is potentially
a serious error for the 4.4 release as well, unless it has been fixed by some
scheduling dependency that I haven't uncovered.  However, I have not been able
to find a test case.


-- 
           Summary: x86_64 red zone violation
           Product: gcc
           Version: 4.3.4
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: ian at airs dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=39118


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2009-02-17 19:50 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-02-06  5:52 [Bug target/39118] New: x86_64 red zone violation ian at airs dot com
2009-02-06  5:53 ` [Bug target/39118] " ian at airs dot com
2009-02-06 13:45 ` ubizjak at gmail dot com
2009-02-06 14:14 ` ian at airs dot com
2009-02-06 14:50 ` ubizjak at gmail dot com
2009-02-06 15:05 ` ubizjak at gmail dot com
2009-02-06 16:44 ` hjl dot tools at gmail dot com
2009-02-06 17:25 ` ian at airs dot com
2009-02-08 21:01 ` [Bug target/39118] [4.3/4.4 Regression] " ubizjak at gmail dot com
2009-02-09 21:22 ` jsm28 at gcc dot gnu dot org
2009-02-10 16:13 ` uros at gcc dot gnu dot org
2009-02-10 16:13 ` uros at gcc dot gnu dot org
2009-02-10 16:14 ` ubizjak at gmail dot com
2009-02-10 19:56 ` ian at airs dot com
2009-02-10 21:04 ` ian at airs dot com
2009-02-10 21:11 ` ian at airs dot com
2009-02-11  7:09 ` ubizjak at gmail dot com
2009-02-11  8:24 ` ubizjak at gmail dot com
2009-02-11  8:32 ` ubizjak at gmail dot com
2009-02-11 11:00 ` jakub at gcc dot gnu dot org
2009-02-11 11:43 ` uros at gcc dot gnu dot org
2009-02-11 11:54 ` uros at gcc dot gnu dot org
2009-02-11 11:55 ` ubizjak at gmail dot com
2009-02-11 14:49 ` ian at airs dot com
2009-02-17 18:40 ` rguenth at gcc dot gnu dot org
2009-02-17 18:45 ` hjl dot tools at gmail dot com
2009-02-17 19:00 ` rguenther at suse dot de
2009-02-17 19:50 ` rguenther at suse dot de

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).