mudflap: how do I give size for 'void *' objects?

mudflap: how do I give size for 'void *' objects?

[Eyal Lebedinsky]

I have a problem where I get a large number of reported violations
when accessing objects acquired through something like
	p = shmat()
where the size of the object is unknown (unless mudflap uses
extra internal knowledge).

I also get a violation on practically every access to a ctype is*()
function, which may be related to a similar issue. I think this
problem is not present in a later libc.

I attempted something like this, but it did not help:

p = (char *)*(MY_STRUCT * (*) (int __shmid, __const void *__shmaddr,
	int __shmflg))(&shmat) (shmid, NULL, 0);

Currently running any of my programs is a pain, generating 100s
of megs of violation reports.

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>

Re: mudflap: how do I give size for 'void *' objects? - example

[Eyal Lebedinsky]

Eyal Lebedinsky wrote:
> 
> I have a problem where I get a large number of reported violations
> when accessing objects acquired through something like
>         p = shmat()
> where the size of the object is unknown (unless mudflap uses
> extra internal knowledge).

Here is a simple program that demonstrates the problem with ctime().
Note how the fprintf() works OK (no violation) but the strlen()
fails. I assume that mudflap has its own strlen() wrapper that
trips on the "foreign" pointer returned from ctime().

  -----------------------------------------------------
#!/bin/sh

mf="/usr/local/gcc-mudflap"

cat >zz.c <<EOF
#include <stdio.h>
#include <time.h>
#include <string.h>

int main ()
{
        time_t  t;
        char    *p;
        int     l;

        t = time (NULL);
        p = ctime (&t);

        fprintf (stderr, "%s", p);

        l = (int)strlen (p);

        return (0);
}
EOF

export LD_LIBRARY_PATH="$mf/lib"

 cc="$mf/bin/i686-pc-linux-gnu-gcc-3.5-tree-ssa -fmudflap -g -O0"
#cc="gcc"

test -f zz && rm zz
$cc -Wall -o zz zz.c
./zz
  -----------------------------------------------------

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>

Re: mudflap: how do I give size for 'void *' objects? - example

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 3354 bytes --]

Hi -

On Sat, Apr 05, 2003 at 04:26:41PM +1000, Eyal Lebedinsky wrote:
> > I have a problem where I get a large number of reported violations
> > when accessing objects acquired through something like
> >         p = shmat()
> > where the size of the object is unknown (unless mudflap uses
> > extra internal knowledge).

In general, libmudflap does not have suitable wrapper functions
already, the application may manually register/unregister
objects using functions declared in mf-runtime.h.

shmat() wrapping will be a bit tricky, because the size of the
newly mapped memory region is not present as a local parameter
of the call.  libmudflap will probably need to intercept shmget()
too just to get this information.

> Here is a simple program that demonstrates the problem with ctime().
> Note how the fprintf() works OK (no violation) but the strlen()
> fails. I assume that mudflap has its own strlen() wrapper that
> trips on the "foreign" pointer returned from ctime().

Right.  Enabling libmudflap heuristics (e.g. "-heur-proc-map") should
cases like this work, if one does not want to write more wrappers.

It would be nice to think of a way of getting such data pointers out of
libraries.  Or compile them in such a way that they are able to register
their individual static variables (but without necessarily the full
mudflap internal checking).


You also wrote:

> And while I am on the ebox, why do I not see a proper stack trace
> with line numbers? What I got for this test is below. The
>         'zz.c:19 (usemem)'
> is correct, but I would have liked to see (and expected)
>         'zz.c:48 (main)'

You're not looking at a stack trace.  These strings are not extracted
from the debugging information, but is rather constructed at compile-time
into literal strings from the trees, to identify the check site or
variable.  To parse the quoted error message:

> mudflap violation 1 (check/read): time=1049530365.371258 ptr=08051f30
> size=1 pc=080487f9 location=`zz.c:19 (usemem)

-> The usemem function at zz.c:19 made a one-byte memory read of the
given address.


>       ./zz(__mf_check+0x245) [0x80487f9]
>       ./zz(main+0x52) [0x8048876]
>       /lib/libc.so.6(__libc_start_main+0xbb) [0x4005714f]

-> The stack traceback, as supplied by glibc, went thusly.  Since usemem
doesn't show up, maybe it was inlined here, or maybe glibc doesn't show
it in the traceback for another reason.


> Nearby object 1: checked region begins 0B into and ends 0B into
> mudflap object 08051f60: name=`malloc region'
> bounds=[08051f30,08051f34] size=5 area=heap check=1r/0w liveness=1
> watching=0

-> This is the first read to a heap region.  It hasn't been written
to, as far as libmudflap knows ("0w"), and it was not registered as
initialized ("area=heap"), so the initialization checking code
detected a violation.  If you just want to shut this check up,
you can use MUDFLAP_OPTIONS="-no-check-initialization", though it
would be better to find out why libmudflap thinks that the buffer
hasn't been written to already.


> alloc time=1049530365.371099 pc=40019f32
>       /usr/local/gcc-mudflap/lib/libmudflap.so.0(__real_malloc+0x142)
> [0x40019f32]
>       ./zz(__mf_check+0xfc) [0x80486b0]
>       ./zz(main+0x25) [0x8048849]
> number of nearby objects: 1

-> This is the glibc stack traceback at the moment of allocation.


- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mudflap: how do I give size for 'void *' objects? - example

[Eyal Lebedinsky]

"Frank Ch. Eigler" wrote:
> 
> Hi -
> 
> On Sat, Apr 05, 2003 at 04:26:41PM +1000, Eyal Lebedinsky wrote:
> > > I have a problem where I get a large number of reported violations
> > > when accessing objects acquired through something like
> > >         p = shmat()
> > > where the size of the object is unknown (unless mudflap uses
> > > extra internal knowledge).
> 
> In general, libmudflap does not have suitable wrapper functions
> already, the application may manually register/unregister
> objects using functions declared in mf-runtime.h.

Good, I now wrapped ctime() and shmat() with __mf_[un]register()
calls and this removed the reports. BTW, for such foreign objects,
how do I set the size when it is unknown (e.g. for 'ctime' I now
cheat and say '64'), and what type do I use (I use GUESS)?

And just to be sure I did not miss a post, the getmem/usemem example
does show a real problem, right? It is this one that gets me mostly,
filling up my logs (it trips a few times per record in a sort
program).

> You're not looking at a stack trace.  These strings are not extracted
> from the debugging information, but is rather constructed at compile-time
> into literal strings from the trees, to identify the check site or
> variable.

So, the "location=`zz.c:19 (usemem)'" is from compile time, but the
following traceback is from glibc runtime? I never saw line numbers
in this trace, only hex offsets, and I do use '-g'. Can I make it
show line numbers (am I missing an option)? Just to indicate what
gives me this trouble, here is an example report which I find very
hard to use:

*******
mudflap violation 2 (check/read): time=1049594102.444443 ptr=415aba22
size=2 pc=
40b39bac location=`(memcmp 1st arg)'
      /usr/local/gcc-mudflap/lib/libmudflap.so.0(__mfwrap_memcmp+0x13c)
[0x40b39
bac]
      /ssa/builds/20030404n/bin/libssaiok.so [0x40a85d51]
      /ssa/builds/20030404n/bin/libssaiok.so [0x40a84c93]
Nearby object 1: checked region begins 444922B into and ends 444923B
into
mudflap object 083ab778: name=`malloc region'
Nearby object 2: checked region begins 444922B into and ends 444923B
into
mudflap object 08398d20: name=`malloc region'

My libssaiok.so is very large, and use extensively, and I do not see
how I can relate the report context to my sources.

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>

Re: mudflap: how do I give size for 'void *' objects? - example

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 2166 bytes --]

Hi -

On Sun, Apr 06, 2003 at 01:32:53PM +1000, Eyal Lebedinsky wrote:
> [...]
> Good, I now wrapped ctime() and shmat() with __mf_[un]register()
> calls and this removed the reports. 

(Patch please?)

> BTW, for such foreign objects,
> how do I set the size when it is unknown (e.g. for 'ctime' I now
> cheat and say '64'), and what type do I use (I use GUESS)?

For ctime, the size is probably 26 or 27, and the type should
be STATIC.  For other functions, it varies case by case.


> And just to be sure I did not miss a post, the getmem/usemem example
> does show a real problem, right? [...]

It shows a real problem in your test program.  You're allocating a piece
of memory with malloc(), so it is uninitialized.  Its contents are
undefined, but you're reading from it.  The libmudflap default option
"-check-initialization" is designed to detect exactly this.  Had you
used calloc(), or wrote to the buffer first, there would be no
violation shown.  (This is a pretty naive algorithm - nowhere near
what valgrind does.)


> So, the "location=`zz.c:19 (usemem)'" is from compile time, but the
> following traceback is from glibc runtime? 

Yes.

> I never saw line numbers in this trace, only hex offsets,
> and I do use '-g'. Can I make it show line numbers (am I
> missing an option)? 

I don't think glibc can do that.  By supplying "--export-dynamic",
your shared library may have more of the symbols glibc may use
to get at least function names.

> [...] here is an example report which I find very hard to use:
> [...]/libmudflap.so.0(__mfwrap_memcmp+0x13c) [0x40b39bac]
>       /ssa/builds/20030404n/bin/libssaiok.so [0x40a85d51]
>       /ssa/builds/20030404n/bin/libssaiok.so [0x40a84c93]

Yeah.  Unfortunately libmudflap and glibc don't use debugging
data to get more precise locations.  I expect we could use an
external "addr2line" child process to map raw PC values to
source:line coordinates.  Care to try writing up such an option
for libmudflap, perhaps based on libjava/gnu/gcj/runtime/NameFinder.java?
(I believe there used to be a C/C++ version of that little function too.)


- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mudflap: how do I give size for 'void *' objects? - example

[Eyal Lebedinsky]

"Frank Ch. Eigler" wrote:
> 
> Hi -
> 
> On Sun, Apr 06, 2003 at 01:32:53PM +1000, Eyal Lebedinsky wrote:
> > [...]
> > Good, I now wrapped ctime() and shmat() with __mf_[un]register()
> > calls and this removed the reports.
> 
> (Patch please?)

Well, the patch is to my code (I already had a ctime wrapper and
now added a __mf_[un]register pair). I do not see how it can be done
as a proper mf runtime as I register the memory with a size known only
to me (the shmat case). But maybe I can inquire the shared memory
system for the size instead? I will check.

BTW, is there a way to ask mudflap to figure the size by itself? I
can then register the ctime result (or strerror etc.) and let mf
runtime do a strlen without triggering an error yet. Maybe
something as simple as this for registering an initialised string:

void
__mf_register_str (void *ptr, int type, const char *name)
{
	__mf_register (ptr, strlen (ptr), type, name);
}

And maybe checking 'ptr' first?

> > BTW, for such foreign objects,
> > how do I set the size when it is unknown (e.g. for 'ctime' I now
> > cheat and say '64'), and what type do I use (I use GUESS)?
> 
> For ctime, the size is probably 26 or 27, and the type should
> be STATIC.  For other functions, it varies case by case.

STATIC? I understand that in a multithreaded situation the ctime area
is allocated per-thread, and may be releases on thread exit, or even
on the next call from the same thread. I think that the correct type
depends on the libc implementation.

> > And just to be sure I did not miss a post, the getmem/usemem example
> > does show a real problem, right? [...]
> 
> It shows a real problem in your test program.  You're allocating a piece
> of memory with malloc(), so it is uninitialized.  Its contents are
> undefined, but you're reading from it.  The libmudflap default option
> "-check-initialization" is designed to detect exactly this.  Had you
> used calloc(), or wrote to the buffer first, there would be no
> violation shown.  (This is a pretty naive algorithm - nowhere near
> what valgrind does.)

OK, I did not realise mudflap generates uninited errors. Where can I
find a list of the mudflap errors and their meaning then? I will check
to see if this is what happens in my real program.

> > [...] here is an example report which I find very hard to use:
> > [...]/libmudflap.so.0(__mfwrap_memcmp+0x13c) [0x40b39bac]
> >       /ssa/builds/20030404n/bin/libssaiok.so [0x40a85d51]
> >       /ssa/builds/20030404n/bin/libssaiok.so [0x40a84c93]
> 
> Yeah.  Unfortunately libmudflap and glibc don't use debugging
> data to get more precise locations.  I expect we could use an
> external "addr2line" child process to map raw PC values to
> source:line coordinates.  Care to try writing up such an option
> for libmudflap, perhaps based on libjava/gnu/gcj/runtime/NameFinder.java?
> (I believe there used to be a C/C++ version of that little function too.)

I now have a perl script that reads the error log and parses the hex
values into file:line (was written by a workmate, uses gdb). It will
be posted soon.

--
Eyal Lebedinsky (eyal@eyal.emu.id.au) <http://samba.org/eyal/>