[Bug c/44736] New: Overeager -O1 optimization results in incorrect code generation

[Bug c/44736] New: Overeager -O1 optimization results in incorrect code generation

[opensource3141 at gmail dot com]

I have found what appears to be a pretty serious issue with the new
optimization algorithms in GCC 4.5.0.  The following code (with appropriate
comments inline) demonstrate the problem.  Based on a quick search, I don't
think this bug has been reported yet.  Note that I have assigned the component
to C since I wasn't sure which other component -- rtl-optimization or
tree-optimization or some other component entirely -- was the best component.

I'm fairly certain that the problem is target independent since I see it with
GCC 4.5.0 for mingw-w64 targets.  When compiling with my system GCC (version
4.4.3) or a self-compiled GCC 4.4.4, there are no problems.

Please let me know if you need any other information from me.

---
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <malloc.h>

void *malloc_hook (size_t, const void *);
void  free_hook   (void *, const void *);

void *(* _old_malloc_hook) (size_t, const void *);
void  (* _old_free_hook)   (void *, const void *);

static int _allocCount;

int main (int argc, char *argv[])
{
    _allocCount    = 0;

    // Save any existing malloc hooks
    _old_malloc_hook = __malloc_hook;
    _old_free_hook   = __free_hook;

    // Add our function as the primary __malloc_hook
    __malloc_hook    = malloc_hook;
    __free_hook      = free_hook;

    // Invoke malloc, which should invoke our function
    char *str = (char *)malloc(4);

    strcpy(str, "foo");
    printf("str = %s\n", str);
    printf("allocCount = %d\n", _allocCount);

    return 0;
}

void *malloc_hook (size_t size, const void *caller)
{
    void *result;

    // Unhook ourselves before calling malloc,
    // or else there will be an infinite recursion!
    //
    // *** NOTE THIS IS THE CODE THAT GETS ERRONEOUSLY
    // OPTIMIZED OUT RESULTING IN STACK OVERFLOW. ***
    __malloc_hook = _old_malloc_hook;
    __free_hook   = _old_free_hook;

    // Since malloc is a function and the optimizer has
    // no idea what it does with __malloc_hook, it has
    // to ensure that the above code is generated, or
    // else there will be big problems.
    result = malloc(size);

    _allocCount++;

    // Save the current malloc hooks in case
    // they changed as a result of calling malloc
    _old_malloc_hook = __malloc_hook;
    _old_free_hook   = __free_hook;

    // Add ourselves back as the hook
    __malloc_hook = malloc_hook;
    __free_hook   = free_hook;

    return result;
}

void free_hook (void *ptr, const void *caller)
{
    // Same as above but for free()
    __malloc_hook = _old_malloc_hook;
    __free_hook   = _old_free_hook;

    free(ptr);

    _allocCount--;

    _old_malloc_hook = __malloc_hook;
    _old_free_hook   = __free_hook;

    __malloc_hook = malloc_hook;
    __free_hook   = free_hook;
}
---

] x86_64-linux-gcc -g -O1 -o malloc_hook malloc_hook.c && malloc_hook
Segmentation fault (core dumped)


Here is the configuration options for GCC:
---
Configured with: ../../gcc-4.5.0/configure
--prefix=/usr/local/gcc/x86_64-linux-gcc
--with-sysroot=/usr/local/gcc/x86_64-linux-gcc/x86_64-linux/sdk
--program-prefix=x86_64-linux- --enable-languages=c,c++ --build=i386-linux
--host=i386-linux --target=x86_64-linux --enable-multilib
--enable-targets=x86_64-linux,i386-linux --disable-nls --enable-shared
--without-x
---

Note that the behavior also is exhibited when GCC is invoked with --sysroot=/
(basically using the system libraries instead of the sdk).


-- 
           Summary: Overeager -O1 optimization results in incorrect code
                    generation
           Product: gcc
           Version: 4.5.0
            Status: UNCONFIRMED
          Severity: critical
          Priority: P3
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: opensource3141 at gmail dot com
 GCC build triplet: i386-linux
  GCC host triplet: i386-linux
GCC target triplet: x86_64-linux


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44736

[Bug tree-optimization/44736] Overeager -O1 optimization results in incorrect code generation

[pinskia at gcc dot gnu dot org]

------- Comment #1 from pinskia at gcc dot gnu dot org  2010-06-30 23:42 -------
Stupid glibc extensions.

__malloc_hook and  __free_hook are not part of standard C90/C99.  

-fno-builtin-malloc will disable this optimization.  Basically malloc cannot
touch global memory as far as the compiler knows so it optimizes out the extra
stores.


-- 

pinskia at gcc dot gnu dot org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|critical                    |normal
          Component|c                           |tree-optimization


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44736

[Bug tree-optimization/44736] Overeager -O1 optimization results in incorrect code generation

[opensource3141 at gmail dot com]

------- Comment #2 from opensource3141 at gmail dot com  2010-07-01 00:32 -------
Thanks for the lightening fast response.  I wouldn't have known to look there,
especially since older GCC versions did not have this problem.

Is it because 4.5.0 has better optimizations such that the code surrounding
this malloc is now optimized away in this situation?

Finally, is this issue going to be resolved within GCC, or is the permanent fix
going to be to ask developers to use -fno-builtin-malloc?

Thanks again.


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44736

[Bug tree-optimization/44736] Overeager -O1 optimization results in incorrect code generation

[rguenth at gcc dot gnu dot org]

------- Comment #3 from rguenth at gcc dot gnu dot org  2010-07-01 08:20 -------
(In reply to comment #2)
> Thanks for the lightening fast response.  I wouldn't have known to look there,
> especially since older GCC versions did not have this problem.
> 
> Is it because 4.5.0 has better optimizations such that the code surrounding
> this malloc is now optimized away in this situation?
> 
> Finally, is this issue going to be resolved within GCC, or is the permanent fix
> going to be to ask developers to use -fno-builtin-malloc?

We are going to ask developers to use -fno-builtin-malloc for now.  I also
think this is a glibc bug which should mark the hook variables volatile.

So, can you file a bug in the glibc bugzilla as well?

> Thanks again.
> 


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44736

[Bug tree-optimization/44736] Overeager -O1 optimization results in incorrect code generation

[opensource3141 at gmail dot com]

------- Comment #4 from opensource3141 at gmail dot com  2010-07-01 16:52 -------
(In reply to comment #3)
> We are going to ask developers to use -fno-builtin-malloc for now.  I also
> think this is a glibc bug which should mark the hook variables volatile.

I tested using the volatile keyword inside the glibc header malloc.h, and that
also serves as a workaround.  However, I believe the meaning of volatile is
something a bit different, and using the qualifier would prevent GCC from
optimizing accesses to the hook variables independent of calls to malloc.

The fundamental issue seems to be that there is a mismatch between how malloc
behaves and how GCC thinks it behaves.  It sounds like the best way to resolve
that mismatch is to use -fno-builtin-malloc, the main drawback being that the
developer has to be consciously aware of this when building the code. 
Otherwise, there will be a nasty surprise.

> So, can you file a bug in the glibc bugzilla as well?

I have filed a bug there and referenced this one:
http://sources.redhat.com/bugzilla/show_bug.cgi?id=11781


-- 


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44736