[Bug c/52236] New: segfault in program that results from compiling sha512-hash.c

[Bug c/52236] New: segfault in program that results from compiling sha512-hash.c

[zooko at zooko dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52236

             Bug #: 52236
           Summary: segfault in program that results from compiling
                    sha512-hash.c
    Classification: Unclassified
           Product: gcc
           Version: 4.7.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: zooko@zooko.com


Using this version of gcc-4.7.0 prerelease which is shipped in Fedora Rawhide:
4.7.0 20120126 (Red Hat 4.7.0-0.10), the resulting code segfaults. Valgrind
reports this:

==9709== Invalid read of size 1
==9709==    at 0xB4FEE70: crypto_hash_sha512 (sha512-hash.c:40)
==9709==    by 0xB4F8F23: crypto_sign_publickey (ed25519.c:30)
==9709==    by 0xB4F7EEB: ed25519_publickey (ed25519module.c:48)
==9709==    by 0x4F0A153: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4F0B7C0: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E9C2BA: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E78A1D: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E86EEF: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E78A1D: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4ECBC41: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4ECB8DB: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E78A1D: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==9709==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==9709== 
{
   <insert_a_suppression_name_here>
   Memcheck:Addr1
   fun:crypto_hash_sha512
   fun:crypto_sign_publickey
   fun:ed25519_publickey
   fun:PyEval_EvalFrameEx
   fun:PyEval_EvalCodeEx
   obj:/usr/lib64/libpython2.7.so.1.0
   fun:PyObject_Call
   obj:/usr/lib64/libpython2.7.so.1.0
   fun:PyObject_Call
   obj:/usr/lib64/libpython2.7.so.1.0
   obj:/usr/lib64/libpython2.7.so.1.0
   fun:PyObject_Call
}
==9709== 
==9709== Process terminating with default action of signal 11 (SIGSEGV)
==9709==  Access not within mapped region at address 0x0
==9709==    at 0xB4FEE70: crypto_hash_sha512 (sha512-hash.c:40)
==9709==    by 0xB4F8F23: crypto_sign_publickey (ed25519.c:30)
==9709==    by 0xB4F7EEB: ed25519_publickey (ed25519module.c:48)
==9709==    by 0x4F0A153: PyEval_EvalFrameEx (in
/usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4F0B7C0: PyEval_EvalCodeEx (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E9C2BA: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E78A1D: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E86EEF: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E78A1D: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4ECBC41: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4ECB8DB: ??? (in /usr/lib64/libpython2.7.so.1.0)
==9709==    by 0x4E78A1D: PyObject_Call (in /usr/lib64/libpython2.7.so.1.0)

More detail, including access to a buildbot which can reliably reproduce the
problem on Fedora Rawhide and demonstrate no such problem on several other
platforms, is here: https://tahoe-lafs.org/trac/pycryptopp/ticket/80

[Bug c/52236] segfault in program that results from compiling sha512-hash.c

[zooko at zooko dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52236

--- Comment #1 from Zooko Wilcox-O'Hearn <zooko at zooko dot com> 2012-02-13 17:03:49 UTC ---
The host is linux x86_64. More information about the platform is queried
automatically by the buildbot and archived here:
https://tahoe-lafs.org/buildbot-pycryptopp/builders/Ruben%20Fedora%20syslib/builds/44/steps/show-tool-versions/logs/stdio

platform: Linux-3.2.1-8.fc17.x86_64-x86_64-with-fedora-17-Rawhide
machine:  x86_64
linux_distribution: ('Fedora', '17', 'Rawhide')

[Bug c/52236] segfault in program that results from compiling sha512-hash.c

[zooko at zooko dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52236

--- Comment #2 from Zooko Wilcox-O'Hearn <zooko at zooko dot com> 2012-02-13 17:17:37 UTC ---
I opened a ticket on launchpad.net with which to track the progress of this
issue across multiple other issue trackers: pycryptopp, GCC, Fedora, and
possibly DJB's "nacl" crypto library if there is any way to track such issues
other than emailing the author.
https://bugs.launchpad.net/pycryptopp/+bug/931542

[Bug c/52236] segfault in program that results from compiling sha512-hash.c

[jakub at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52236

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2012-02-13
                 CC|                            |jakub at gcc dot gnu.org
     Ever Confirmed|0                           |1

--- Comment #3 from Jakub Jelinek <jakub at gcc dot gnu.org> 2012-02-13 17:40:38 UTC ---
Please read http://gcc.gnu.org/bugs.html, you should provide a self-contained
and if possible small testcase, it could very well be a bug in the application
you are using.  If you suspect a gcc bug, you can use use either a debugger or
brute-force - e.g. binary search in between objects compiled with various
compilation flags or various versions of the compiler (-O0 vs. standard flags,
or standard flags + -fno-strict-aliasing, etc.).

[Bug middle-end/52236] segfault in program that results from compiling sha512-hash.c

[zooko at zooko dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52236

Zooko Wilcox-O'Hearn <zooko at zooko dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |INVALID

--- Comment #4 from Zooko Wilcox-O'Hearn <zooko at zooko dot com> 2012-02-15 21:22:28 UTC ---
Thanks to Samuel Neves, the bug was identified in the application (pycryptopp),
not in gcc. Thanks!