How can I modify the source of new,malloc

How can I modify the source of new,malloc

[Zvi Vered]

Hello,

I'm using crosstool-ng 5.1.2 to create gcc 4.3.2
The project started 3 years ago. This is the reason for the old version.
I modify the code of new,malloc so it will print a message each time
they are called.
Where is the code of those routines ?
Are they part of glibc ?

I have a huge application (not written by me).
I want to track all places where a dynamic allocation is done.
Is there an alternative to my way ?

Thanks,
Zvika

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: How can I modify the source of new,malloc

[Mirko Banchi]

You could write your own shared object with your malloc implementation  
and use LD_PRELOAD var.

Mirko

Il giorno 25/giu/2012, alle ore 23.55, Zvi Vered <veredz72@gmail.com>  
ha scritto:

> Hello,
>
> I'm using crosstool-ng 5.1.2 to create gcc 4.3.2
> The project started 3 years ago. This is the reason for the old  
> version.
> I modify the code of new,malloc so it will print a message each time
> they are called.
> Where is the code of those routines ?
> Are they part of glibc ?
>
> I have a huge application (not written by me).
> I want to track all places where a dynamic allocation is done.
> Is there an alternative to my way ?
>
> Thanks,
> Zvika
>
> --
> For unsubscribe information see http://sourceware.org/lists.html#faq
>

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: How can I modify the source of new,malloc

[Yann E. MORIN]

Zvi, Mirko, All,

[Please, do not top-post...]

On Tuesday 26 June 2012 00:28:09 Mirko Banchi wrote:
> Il giorno 25/giu/2012, alle ore 23.55, Zvi Vered <veredz72@gmail.com>  
> ha scritto:
> > I have a huge application (not written by me).
> > I want to track all places where a dynamic allocation is done.
> > Is there an alternative to my way ?
> You could write your own shared object with your malloc implementation  
> and use LD_PRELOAD var.

As Mirko suggested, I'd use a loader trick to pre-load a shared lib that
overrides the malloc() and free() functions. Something like (untested!):

void* malloc( size_t size )
{
    static void* real_malloc;

    if( ! real_malloc ) {
        real_malloc = dlsym( RTLD_NEXT, "malloc" );
        if( ! real_malloc ) {
            panic_and_exit();
        }
    }
    fprintf( stderr, "Allocating %d bytes\n", size );
    return real_malloc( size );
}

Then, replacing printf with calls to backtrace(3) and backtrace_symbols_fd(3),
you can know the caller (function, offset), if you have the debug symbols in
the binaries (libs and executable), in which case you can post-process that
to find the actual  file, function and line at which the call to malloc is
made.

(Note: do not use backtrace_symbols(3), as it calls malloc! So you have
to use backtrace_symbols_fd(3)).

Then, you can write such wrappers to free(3) and so on...

Note however, that if you can recompile your application, you may want to
link with a library like DUMA or dmalloc instead of using the above trick.
  http://duma.sourceforge.net/
  http://dmalloc.com/

Which can be installed in the toolshain's sysroot by crosstool-NG ( I leave
it up to you to find in which sub-menu these libs' options live! ;-) )

Cheers,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: How can I modify the source of new,malloc

[Joel Sherrill]

On 06/26/2012 03:14 PM, Yann E. MORIN wrote:
> Zvi, Mirko, All,
>
> [Please, do not top-post...]
>
> On Tuesday 26 June 2012 00:28:09 Mirko Banchi wrote:
>> Il giorno 25/giu/2012, alle ore 23.55, Zvi Vered<veredz72@gmail.com>
>> ha scritto:
>>> I have a huge application (not written by me).
>>> I want to track all places where a dynamic allocation is done.
>>> Is there an alternative to my way ?
>> You could write your own shared object with your malloc implementation
>> and use LD_PRELOAD var.
> As Mirko suggested, I'd use a loader trick to pre-load a shared lib that
> overrides the malloc() and free() functions. Something like (untested!):
>
> void* malloc( size_t size )
> {
>      static void* real_malloc;
>
>      if( ! real_malloc ) {
>          real_malloc = dlsym( RTLD_NEXT, "malloc" );
>          if( ! real_malloc ) {
>              panic_and_exit();
>          }
>      }
>      fprintf( stderr, "Allocating %d bytes\n", size );
>      return real_malloc( size );
> }
>
> Then, replacing printf with calls to backtrace(3) and backtrace_symbols_fd(3),
> you can know the caller (function, offset), if you have the debug symbols in
> the binaries (libs and executable), in which case you can post-process that
> to find the actual  file, function and line at which the call to malloc is
> made.
>
> (Note: do not use backtrace_symbols(3), as it calls malloc! So you have
> to use backtrace_symbols_fd(3)).
>
> Then, you can write such wrappers to free(3) and so on...
>
> Note however, that if you can recompile your application, you may want to
> link with a library like DUMA or dmalloc instead of using the above trick.
>    http://duma.sourceforge.net/
>    http://dmalloc.com/
>
> Which can be installed in the toolshain's sysroot by crosstool-NG ( I leave
> it up to you to find in which sub-menu these libs' options live! ;-) )
You may also be able to use the "--wrap=symbol" option on ld. I have
used this to provide malloc and free wrappers.
>
> Cheers,
> Yann E. MORIN.
>


-- 
Joel Sherrill, Ph.D.             Director of Research&   Development
joel.sherrill@OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
     Support Available             (256) 722-9985



--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: How can I modify the source of new,malloc

[Eric Doenges]

On 26.06.2012 22:14, Yann E. MORIN wrote:

> As Mirko suggested, I'd use a loader trick to pre-load a shared lib that
> overrides the malloc() and free() functions. Something like (untested!):
> 
> void* malloc( size_t size )
> {
>     static void* real_malloc;
> 
>     if( ! real_malloc ) {
>         real_malloc = dlsym( RTLD_NEXT, "malloc" );
>         if( ! real_malloc ) {
>             panic_and_exit();
>         }
>     }
>     fprintf( stderr, "Allocating %d bytes\n", size );
>     return real_malloc( size );
> }
> 
> Then, replacing printf with calls to backtrace(3) and backtrace_symbols_fd(3),
> you can know the caller (function, offset), if you have the debug symbols in
> the binaries (libs and executable), in which case you can post-process that
> to find the actual  file, function and line at which the call to malloc is
> made.
> 
> (Note: do not use backtrace_symbols(3), as it calls malloc! So you have
> to use backtrace_symbols_fd(3)).
> 
> Then, you can write such wrappers to free(3) and so on...

Also note that if you wrap calloc() like malloc() above, you will run
into the problem that dlsym calls calloc (at least on all the Linux
systems I have tested on). What I do is to load the pointers to the real
functions in a special initialization function that is marked with the
"constructor" attribute, and call sbrk() instead of the real calloc if
the pointer to the real calloc hasn't been defined yet.

> Note however, that if you can recompile your application, you may want to
> link with a library like DUMA or dmalloc instead of using the above trick.
>   http://duma.sourceforge.net/
>   http://dmalloc.com/

If you really want to go down the wrapper route, I can send you my
wrapper off-list as a starting point. It currently prints out how much
memory was requested (and tracks all allocations, so it can tell you
about possible memory leaks on exit), but since I didn't know about
backtrace(3) and friends it doesn't show you where the call came from (yet).
-- 
Dr. Eric Dönges                            doenges@mvtec.com
MVTec Software GmbH | Neherstr. 1 | 81675 München  | Germany
www.mvtec.com | Tel: +49 89 457695-0 | Fax: +49 89 457695-55
Geschäftsführer: Dr. Wolfgang Eckstein, Dr. Olaf Munkelt
Amtsgericht München HRB 114695



--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: How can I modify the source of new,malloc

[Eric Doenges]

On 27.06.2012 11:16, Eric Doenges wrote:
> On 26.06.2012 22:14, Yann E. MORIN wrote:
> 
>> As Mirko suggested, I'd use a loader trick to pre-load a shared lib that
>> overrides the malloc() and free() functions. Something like (untested!):
>>
>> void* malloc( size_t size )
>> {
>>     static void* real_malloc;
>>
>>     if( ! real_malloc ) {
>>         real_malloc = dlsym( RTLD_NEXT, "malloc" );
>>         if( ! real_malloc ) {
>>             panic_and_exit();
>>         }
>>     }
>>     fprintf( stderr, "Allocating %d bytes\n", size );
>>     return real_malloc( size );
>> }
>>
>> Then, replacing printf with calls to backtrace(3) and backtrace_symbols_fd(3),
>> you can know the caller (function, offset), if you have the debug symbols in
>> the binaries (libs and executable), in which case you can post-process that
>> to find the actual  file, function and line at which the call to malloc is
>> made.
>>
>> (Note: do not use backtrace_symbols(3), as it calls malloc! So you have
>> to use backtrace_symbols_fd(3)).

Just a short addendum: backtrace() calls malloc() internally (see bug
report from 2005 http://sourceware.org/bugzilla/show_bug.cgi?id=956), so
unfortunately you can't use it in your malloc/free wrapper.
-- 
Dr. Eric Dönges                            doenges@mvtec.com
MVTec Software GmbH | Neherstr. 1 | 81675 München  | Germany
www.mvtec.com | Tel: +49 89 457695-0 | Fax: +49 89 457695-55
Geschäftsführer: Dr. Wolfgang Eckstein, Dr. Olaf Munkelt
Amtsgericht München HRB 114695



--
For unsubscribe information see http://sourceware.org/lists.html#faq

Re: How can I modify the source of new,malloc

[Yann E. MORIN]

Eric, All,

On Wednesday 27 June 2012 11:51:14 Eric Doenges wrote:
> On 27.06.2012 11:16, Eric Doenges wrote:
> > On 26.06.2012 22:14, Yann E. MORIN wrote:
> > 
> >> As Mirko suggested, I'd use a loader trick to pre-load a shared lib that
> >> overrides the malloc() and free() functions. Something like (untested!):
> >>
> >> void* malloc( size_t size )
> >> {
> >>     static void* real_malloc;
> >>
> >>     if( ! real_malloc ) {
> >>         real_malloc = dlsym( RTLD_NEXT, "malloc" );
> >>         if( ! real_malloc ) {
> >>             panic_and_exit();
> >>         }
> >>     }
> >>     fprintf( stderr, "Allocating %d bytes\n", size );
> >>     return real_malloc( size );
> >> }
> >>
> >> Then, replacing printf with calls to backtrace(3) and backtrace_symbols_fd(3),
> >> you can know the caller (function, offset), if you have the debug symbols in
> >> the binaries (libs and executable), in which case you can post-process that
> >> to find the actual  file, function and line at which the call to malloc is
> >> made.
> >>
> >> (Note: do not use backtrace_symbols(3), as it calls malloc! So you have
> >> to use backtrace_symbols_fd(3)).
> 
> Just a short addendum: backtrace() calls malloc() internally (see bug
> report from 2005 http://sourceware.org/bugzilla/show_bug.cgi?id=956),

Hmmm. Baaaad... And, as usual, I appreciate very much the comments in
that bug report! ;-p

> so
> unfortunately you can't use it in your malloc/free wrapper.

You probably could, with some tricky tricks (this is not thread-safe, but
no less than was my initial code-snippet):

void* malloc( size_t size )
{
    static void* real_malloc;
    static int in_malloc;

    if( ! real_malloc ) {
        real_malloc = dlsym( RTLD_NEXT, "malloc" );
        if( ! real_malloc ) {
            panic_and_exit();
        }
    }
    if( ! in_malloc ) {
        in_malloc = 1;
        fprintf( stderr, "Allocating %d bytes\n", size );
        backtrace( blablabla );
        in_malloc = 0;
    }
    return real_malloc( size );
}

You could even probably make it thread-safe by using pthread keys, though
how to do it is left as an exercise for the reader. ;-)

And of course, this would break in awfull ways if somehow malloc() becomes
unavailable (because of corruption or whatever), but in this case you're
deeply screwed anyway... :-/

So, if you need to debug memory allocation, I would highly suggest that you
have a look at DUMA or dmalloc.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq