[Bug tree-optimization/57742] memset(malloc(n),0,n) -> calloc(n,1)

["glisse at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org>]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57742

--- Comment #7 from Marc Glisse <glisse at gcc dot gnu.org> ---
(In reply to Richard Biener from comment #5)
> We have walk_aliased_vdefs for this.  Basically the first callback
> you receive has to be the malloc, otherwise there is an aliasing
> stmt inbetween.  Initialize the ao_ref with ao_ref_init_from_ptr_and_size.

Hmm, there is a problem with that: I don't get a callback for malloc.
stmt_may_clobber_ref_p_1 only looks at the lhs of a call statement if it isn't
an SSA_NAME, so it considers that p=malloc(n) does not clobber MEM_REF[p]. This
kind of makes sense, it creates this memory, which is different from
clobbering. I can look at the def_stmt of the first argument of memset to find
the malloc, at least, but that doesn't help me with the memory checks.

Also, for this testcase:
void* f(int n,double*d){
  int* p=__builtin_malloc(n);
  ++*d;
  __builtin_memset(p,0,n);
  return p;
}
I actually get a callback for the store in *d, which gcc believes might alias
:-(

For this example:
void g(int*);
void* f(int n){
  int* p=__builtin_malloc(n);
  for(int i=0;i<10000;++i){
    __builtin_memset(p,0,n);
    g(p);
    p[5]=10;
  }
  return p;
}
if I modify the aliasing machinery to make it believe that p=malloc does alias,
malloc is the first callback. I haven't added the dominance checks, but I
assume they will tell me that malloc dominates memset and memset postdominates
malloc, although I still shouldn't do the transformation.

Pretty depressed at this point...