Re: tracepoint maintainance models

Re: tracepoint maintainance models

[Vara Prasad]

Alan Cox wrote:

>Ar Llu, 2006-09-18 am 12:15 -0400, ysgrifennodd Frank Ch. Eigler:
>  
>
>>>[...] So its L1 misses more register reloads and the like. Sounds
>>>more and more like wasted clock cycles for debug. [...]
>>>      
>>>
>>But it's not just "for debug"!  It is for system administrators,
>>end-users, developers.
>>    
>>
>
>It is for debug. System administrators and developers also do debug,
>they may just use different tools. The percentage of schedule() calls
>executed across every Linux box on the planet where debug is enabled is
>so close to nil its noise. Even with traces that won't change.
>  
>

Precisely the reason this huge thread is arguing why we shouldn't be 
including only static marker mechanism in the kernel tree. We are using 
dynamic probe mechanism which doesn't alter the execution flow or 
prevent compiler in making good optimizations for the most part but 
there are few code paths that are critical in understanding that we are 
not able to use this dynamic method for which we need static markers. As 
Martin pointed out if one is critical about performance they can be 
compiled out.

It is also important to note the amount of $s lost by taking long time 
to find a solution to a problem due to lack of good debugging tools is 
also significant compared to few additional clock cycles machines spend 
due to these static markers.

>  
>
>>Indeed, there will be some non-zero execution-time cost.  We must be
>>willing to pay *something* in order to enable this functionality.
>>    
>>
>
>There is an implementation which requires no penalty is paid. Create a
>new elf section which contains something like
>
>	[address to whack with int3]
>	[or info for jprobes to make better use]
>	[name for debug tools to find]
>	[line number in source to parse the gcc debug data]
>  
>
I am not sure i quiet understand your line number part of the proposal. 
Does this proposal assume we have access to source code while generating 
dynamic probes?

>
>  
>
This still doesn't solve the problem of compiler optimizing such that a 
variable i would like to read in my probe not being available at the 
probe point.

>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>  
>

Re: tracepoint maintainance models

[Alan Cox]

Ar Llu, 2006-09-18 am 12:10 -0700, ysgrifennodd Vara Prasad:
> I am not sure i quiet understand your line number part of the proposal. 
> Does this proposal assume we have access to source code while generating 
> dynamic probes?

Its one route - or we dump it into an ELF section in the binary.

> This still doesn't solve the problem of compiler optimizing such that a 
> variable i would like to read in my probe not being available at the 
> probe point.

Then what we really need by the sound of it is enough gcc smarts to do
something of the form

	.section "debugbits"
	
	.asciiz 'hook_sched'
	.dword l1	# Address to probe
	.word 1		# Argument count
	.dword gcc_magic_whatregister("next"); [ reg num or memory ]
	.dword gcc_magic_whataddress("next"); [ address if exists]


Can gcc do any of that for us today ?

Re: tracepoint maintainance models

[Frank Ch. Eigler]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

Hi -

On Mon, Sep 18, 2006 at 08:49:40PM +0100, Alan Cox wrote:
> [...]
> Then what we really need by the sound of it is enough gcc smarts to do
> something of the form [...]
> 	.section "debugbits"
> [...]
> Can gcc do any of that for us today ?

This is not that different from what gcc does for DWARF.  Trouble is,
there appear to exist optimization transformations which make such
data difficult or impossible to generate.  (In particular, it is
unlikely to be easier to create specialized data like this if the
compiler can't be made to create first-class DWARF for the same probe
points / data values.)

- FChE

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: tracepoint maintainance models

[Vara Prasad]

Alan Cox wrote:

>Ar Llu, 2006-09-18 am 12:10 -0700, ysgrifennodd Vara Prasad:
>  
>
>>I am not sure i quiet understand your line number part of the proposal. 
>>Does this proposal assume we have access to source code while generating 
>>dynamic probes?
>>    
>>
>
>Its one route - or we dump it into an ELF section in the binary.
>  
>
Source code access is not a good solution but ELF section could work.

>  
>
>>This still doesn't solve the problem of compiler optimizing such that a 
>>variable i would like to read in my probe not being available at the 
>>probe point.
>>    
>>
>
>Then what we really need by the sound of it is enough gcc smarts to do
>something of the form
>
>	.section "debugbits"
>	
>	.asciiz 'hook_sched'
>	.dword l1	# Address to probe
>	.word 1		# Argument count
>	.dword gcc_magic_whatregister("next"); [ reg num or memory ]
>	.dword gcc_magic_whataddress("next"); [ address if exists]
>
>
>Can gcc do any of that for us today ?
>
>  
>
No, gcc doesn't do that today.

Re: tracepoint maintainance models

[Steven Rostedt]

Coming into this really late, and I'm still behind in reading this and
related threads, but I want to throw this idea out, and it's getting
late.

On Mon, 2006-09-18 at 13:28 -0700, Vara Prasad wrote:
> Alan Cox wrote:
> 
> >
> >>This still doesn't solve the problem of compiler optimizing such that a 
> >>variable i would like to read in my probe not being available at the 
> >>probe point.
> >>    
> >>
> >
> >Then what we really need by the sound of it is enough gcc smarts to do
> >something of the form
> >
> >	.section "debugbits"
> >	
> >	.asciiz 'hook_sched'
> >	.dword l1	# Address to probe
> >	.word 1		# Argument count
> >	.dword gcc_magic_whatregister("next"); [ reg num or memory ]
> >	.dword gcc_magic_whataddress("next"); [ address if exists]
> >
> >
> >Can gcc do any of that for us today ?
> >
> >  
> >
> No, gcc doesn't do that today.
> 
> 


---- cut here ----
#include <stdio.h>

#define MARK(label, var)			\
	asm ("debug_" #label ":\n"		\
	     ".section .data\n"			\
	     #label "_" #var ": xor %0,%0\n"	\
	     ".previous" : : "r"(var))


static int func(int a)
{
	int y;
	int z;

	y = a;
	MARK(func, y);
	z = y+2;

	return z;

}



static void read_label(void)
{
	extern unsigned short regA;
	unsigned short *r = &regA;
	char *regs[] = {
		"A", "B", "C", "D", "DI", "BP", "SP", "CH"
	};
	int i;
	extern unsigned short func_y;
	extern unsigned long debug_func;

	asm (".section .data\n"
	     "regA: xor %eax,%eax\n"
	     "regB: xor %ebx,%ebx\n"
	     "regC: xor %ecx,%ecx\n"
	     "regD: xor %edx,%edx\n"
	     "regDI: xor %edi,%edi\n"
	     "regBP: xor %ebp,%ebp\n"
	     "regSP: xor %esp,%esp\n"
	     ".previous");

	for (i=0; i < 7; i++) {
		if (r[i] == func_y)
			break;
	}
	if (i < 7)
		printf("func y is in reg %s at %p\n",
		       regs[i],
		       &debug_func);
	else
		printf("func y not found!\n");
}

int main (int argc, char **argv)
{
	int g;
	g = func(argc);
	read_label();
	return g;
}
---- end cut ----

$ gcc -O2 -o mark mark.c
$ ./mark
func y is in reg B at 0x80483ce



Now the question is, isn't MARK() in this code a non intrusive marker?

So couldn't a kprobe be set at "debug_func" and we can find what
register "y" is without adding any overhead to the code being marked?

Obviously, this would need to be done special for each arch.

-- Steve

Re: tracepoint maintainance models

[Frank Ch. Eigler]

Hi -

On Fri, Oct 06, 2006 at 01:33:11AM -0400, Steven Rostedt wrote:
> Coming into this really late, and I'm still behind in reading this and
> related threads, but I want to throw this idea out, and it's getting
> late.
> [...]
> #define MARK(label, var)			\
> 	asm ("debug_" #label ":\n"		\
> 	     ".section .data\n"			\
> 	     #label "_" #var ": xor %0,%0\n"	\
> 	     ".previous" : : "r"(var))
> [...]
> $ gcc -O2 -o mark mark.c
> $ ./mark
> func y is in reg B at 0x80483ce
> [...]

Clever.

> Now the question is, isn't MARK() in this code a non intrusive marker?

Not quite.  The assembly code forces gcc to materialize the data that
it might already have inlined, and to borrow a register for the
duration.  It's still a neat idea though.

- FChE

Re: tracepoint maintainance models

[Steven Rostedt]

On Fri, 6 Oct 2006, Frank Ch. Eigler wrote:

> Hi -
>
> On Fri, Oct 06, 2006 at 01:33:11AM -0400, Steven Rostedt wrote:
> > Coming into this really late, and I'm still behind in reading this and
> > related threads, but I want to throw this idea out, and it's getting
> > late.
> > [...]
> > #define MARK(label, var)			\
> > 	asm ("debug_" #label ":\n"		\
> > 	     ".section .data\n"			\
> > 	     #label "_" #var ": xor %0,%0\n"	\
> > 	     ".previous" : : "r"(var))
> > [...]
> > $ gcc -O2 -o mark mark.c
> > $ ./mark
> > func y is in reg B at 0x80483ce
> > [...]
>
> Clever.
>
> > Now the question is, isn't MARK() in this code a non intrusive marker?
>
> Not quite.  The assembly code forces gcc to materialize the data that
> it might already have inlined, and to borrow a register for the
> duration.  It's still a neat idea though.

Thanks!

You're right, it is intrusive in a way that it does modify the way gcc can
optimize that section of code.  But what I like about this idea, is that
it allows for us to tell gcc that we want this variable inside a register,
and then gcc can do that for us and still optimize around that. We put no
more constraints on the code, except that we want some value in a
register at some given point of execution.  This should only be done for
local variables that are not easily captured in a probe.

Of course with i386's limit on registers, it can put a little strain if we
want more than one variable. But x86_64 will soon be the norm, and the
added registers should help out a lot.

-- Steve

Re: tracepoint maintainance models

[Jeremy Fitzhardinge]

Steven Rostedt wrote:
> Coming into this really late, and I'm still behind in reading this and
> related threads, but I want to throw this idea out, and it's getting
> late.
>
> On Mon, 2006-09-18 at 13:28 -0700, Vara Prasad wrote:
>   
>> Alan Cox wrote:
>>
>>     
>>>> This still doesn't solve the problem of compiler optimizing such that a 
>>>> variable i would like to read in my probe not being available at the 
>>>> probe point.
>>>>    
>>>>
>>>>         
>>> Then what we really need by the sound of it is enough gcc smarts to do
>>> something of the form
>>>
>>> 	.section "debugbits"
>>> 	
>>> 	.asciiz 'hook_sched'
>>> 	.dword l1	# Address to probe
>>> 	.word 1		# Argument count
>>> 	.dword gcc_magic_whatregister("next"); [ reg num or memory ]
>>> 	.dword gcc_magic_whataddress("next"); [ address if exists]
>>>
>>>
>>> Can gcc do any of that for us today ?
>>>
>>>  
>>>
>>>       
>> No, gcc doesn't do that today.
>>
>>
>>     
>
>
> ---- cut here ----
> #include <stdio.h>
>
> #define MARK(label, var)			\
> 	asm ("debug_" #label ":\n"		\
> 	     ".section .data\n"			\
> 	     #label "_" #var ": xor %0,%0\n"	\
> 	     ".previous" : : "r"(var))
>   

That's a nice idea.  As Frank pointed out, it does force things into 
register.  You could use "rm" as a constraint, so you can also get the 
location wherever it exists.  It will still force gcc into keeping the 
value around at all, but presumably if its interesting for a mark, its 
interesting to keep:

	asm volatile("..."		\
	    #label "_" #var ": mov %0,%%eax\n"	\
	".previous" : : "rm" (var))

and, aside from the naming issues, it could be a general expression 
rather than a specific variable.

Of course, this requires a more complex addressing mode decoder, but it 
does give gcc more flexibility.  And in principle this is all redundant, 
since DWARF should be able to encode all this too, and if you make use 
of the variable as an asm argument, gcc really should be outputting the 
debug info about it.

    J