proc based observers

proc based observers

[Andrew Cagney]

Hello,

Looking at the current uses of the core there are a significant number 
of cases where the code is trying to observe the process, rather than 
just a single task.

I'd like to suggest, and now that the task observers are working well, 
that we extend the observer interface so that, in addition to Task 
observers there be Proc wide observers.  For instance:

   Proc.addObserverBLAH()

would notify of any event of any task within the proc.

Thoughts

Andrew

Re: proc based observers

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1795 bytes --]

Hi Andrew,

On Fri, 2007-06-15 at 10:07 -0400, Andrew Cagney wrote:
> Looking at the current uses of the core there are a significant number 
> of cases where the code is trying to observe the process, rather than 
> just a single task.

Do you have some examples of code that does that in frysk at the moment?

> I'd like to suggest, and now that the task observers are working well, 
> that we extend the observer interface so that, in addition to Task 
> observers there be Proc wide observers.  For instance:
> 
>    Proc.addObserverBLAH()
> 
> would notify of any event of any task within the proc.
> 
> Thoughts

Seems like a good idea. In these situations does the caller expect the
observer to be added to the current Tasks of the Proc or all Tasks of
the Proc in the future that might not exist yet?

I can imagine two somewhat separate use cases. 1) "one time trigger"
observer, like and Instruction observer triggered on all current Task
for collecting some info like the stack traces of all the currently
active Tasks of a Proc and 2) "automatic new task tracking observers",
like a syscall or code breakpoint observer that just wants to know when
some syscall or address is hit, but doesn't care about which Task it is,
or whether that Task existed when the observer was installed.

And to make this a little more general, does it make sense to define
"Task Sets" (like TotalView seems to have) that an observer can be added
to and that have their own rules of which Tasks will belong to it. So
you could have "Task Set" based observer where Proc based observers are
the special case of all (future) Task Set of that Proc. For an example: 
http://www.totalviewtech.com/Documentation/latest/html/User_Guide/CreatingGroups.html

Cheers,

Mark

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: proc based observers

[Andrew Cagney]

Mark Wielaard wrote:
> Hi Andrew,
>
> On Fri, 2007-06-15 at 10:07 -0400, Andrew Cagney wrote:
>   
>> Looking at the current uses of the core there are a significant number 
>> of cases where the code is trying to observe the process, rather than 
>> just a single task.
>>     
>
> Do you have some examples of code that does that in frysk at the moment?
>   

I'm not sure what you're asking.

Re: proc based observers

[Mark Wielaard]

On Mon, 2007-06-18 at 12:53 -0400, Andrew Cagney wrote:
> Mark Wielaard wrote:
> > On Fri, 2007-06-15 at 10:07 -0400, Andrew Cagney wrote:  
> >> Looking at the current uses of the core there are a significant number 
> >> of cases where the code is trying to observe the process, rather than 
> >> just a single task.
> >
> > Do you have some examples of code that does that in frysk at the moment?
> >   
> I'm not sure what you're asking.

Since you are looking at uses of the core I was just wondering whether
you could be more specific since I presume you already have the code
references handy that use it. Like: "Look at X which tries to accomplish
Y, by doing A, B and C, which might be easier achieved if we just had K
and did M". To make sure we are really talking about the same scenarios.
Just so there is some context so you can answer my other questions and
suggestions better.

Cheers,

Mark

Re: proc based observers

[Andrew Cagney]

Mark Wielaard wrote:
>> I'd like to suggest, and now that the task observers are working well, 
>> that we extend the observer interface so that, in addition to Task 
>> observers there be Proc wide observers.  For instance:
>>
>>    Proc.addObserverBLAH()
>>
>> would notify of any event of any task within the proc.
>>
>> Thoughts
>>     
>
> Seems like a good idea. In these situations does the caller expect the
> observer to be added to the current Tasks of the Proc or all Tasks of
> the Proc in the future that might not exist yet?
>
>   

We're not building a new bike shed.  Existing code already implements 
proc-wide observers (yes a proc-wide observer really is proc-wide), only 
it uses a clone observer and brute force to track new tasks as they are 
created.  This change will let implement the functionality more directly 
in the core; hopefully a natural refactoring.

Andrew

Re: proc based observers

[Mark Wielaard]

On Mon, 2007-06-18 at 13:47 -0400, Andrew Cagney wrote:
> Mark Wielaard wrote:
> >> I'd like to suggest, and now that the task observers are working well, 
> >> that we extend the observer interface so that, in addition to Task 
> >> observers there be Proc wide observers.  For instance:
> >>
> >>    Proc.addObserverBLAH()
> >>
> >> would notify of any event of any task within the proc.
> >>
> >> Thoughts
> >
> > Seems like a good idea. In these situations does the caller expect the
> > observer to be added to the current Tasks of the Proc or all Tasks of
> > the Proc in the future that might not exist yet?
> >
> We're not building a new bike shed.  Existing code already implements 
> proc-wide observers (yes a proc-wide observer really is proc-wide), only 
> it uses a clone observer and brute force to track new tasks as they are 
> created.  This change will let implement the functionality more directly 
> in the core; hopefully a natural refactoring.

I don't follow the "new bike shed" reference. Maybe this is simple and
obvious to you, it certainly isn't for me. This is why I asked for the
use cases you have. Simply to know what code you want to replace with
what. A simple example might help to really understand what you are
proposing and why my questions and suggestions might not be relevant.

Thanks,

Mark

Re: proc based observers

[Sami Wagiaalla]

> Do you have some examples of code that does that in frysk at the moment?
>   
There are a couple of instances in the gui. Here is an example from 
Ftrace.java:

ProcObserver.ProcTasks tasksObserver = new ProcObserver.ProcTasks()
  {
    public void existingTask (Task task)
    {
      handleTask(task);
    }

    public void taskAdded (Task task)
    {
      handleTask(task);
    }

    public void taskRemoved (Task arg0)
    {
    }

    public void addedTo (Object arg0)
    {
    }

    public void addFailed (Object arg0, Throwable arg1)
    {
    }

    public void deletedFrom (Object arg0)
    {
    }
  };


Then handle task does this:

synchronized void handleTask (Task task)
  {
    task.requestAddSyscallObserver(syscallObserver);
    task.requestAddForkedObserver(forkedObserver);
    Proc proc = task.getProc();
  ....

The history behind this is interesting:

We used to have some proc observers, but we wanted fine grain control so 
everything was moved down to tasks. then we wanted a safe way of getting 
a procs tasks, so we created ProcTasksObserver. And now the common case 
(at least outside of monitor) is to add observers to all tasks.

Re: proc based observers

[Mark Wielaard]

On Mon, 2007-06-18 at 16:47 -0400, Sami Wagiaalla wrote:
> > Do you have some examples of code that does that in frysk at the moment?
> >   
> There are a couple of instances in the gui. Here is an example from 
> Ftrace.java:
> 
> ProcObserver.ProcTasks tasksObserver = new ProcObserver.ProcTasks()
>   {
>     public void existingTask (Task task)
>     {
>       handleTask(task);
>     }
> 
>     public void taskAdded (Task task)
>     {
>       handleTask(task);
>     }
> [...]
>   };
> 
> 
> Then handle task does this:
> 
> synchronized void handleTask (Task task)
>   {
>     task.requestAddSyscallObserver(syscallObserver);
>     task.requestAddForkedObserver(forkedObserver);

Yeah, that is a nice way to easily add observers to all Tasks of a Proc
now and in the future (this should really also be added to fstep, which
currently only traces the main task, I have updated the bug report).

> The history behind this is interesting:
> 
> We used to have some proc observers, but we wanted fine grain control so 
> everything was moved down to tasks. then we wanted a safe way of getting 
> a procs tasks, so we created ProcTasksObserver.

Is there any functional difference between the old and the new way?

Cheers,

Mark

Re: proc based observers

[Sami Wagiaalla]

>> The history behind this is interesting:
>>
>> We used to have some proc observers, but we wanted fine grain control so 
>> everything was moved down to tasks. then we wanted a safe way of getting 
>> a procs tasks, so we created ProcTasksObserver.
>>     
>
> Is there any functional difference between the old and the new way?
>   
Do you mean the old and new way of getting tasks ?

If so, the old way of getting tasks was Proc.getTask(). Problem is while 
you are iterating over that list tasks could be dying and getting created

Re: proc based observers

[Andrew Cagney]

>
> Is there any functional difference between the old and the new way?
>
>   

That depends on how much one wants to split hairs.  The operation 
proc.addObserverXXX will provide equivalent functionality.  Will it be 
100% identical?  Hardly, but 100% identical isn't what is needed.

Andrew

Re: proc based observers

[Mark Wielaard]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

Hi Andrew,


On Tue, 2007-06-19 at 17:39 -0400, Andrew Cagney wrote:
> >
> > Is there any functional difference between the old and the new way?
> >  
> 
> That depends on how much one wants to split hairs.

I wasn't splitting hairs, I was just curious why there was a move from
the old proc based observer methods to the ProcTasksObserver class based
solution. And Sami explained that the old proc getTasks method wasn't
reliable because it had races with Tasks appearing and disappearing
while adding observers. Which makes perfect sense.

>   The operation 
> proc.addObserverXXX will provide equivalent functionality.  Will it be 
> 100% identical?  Hardly, but 100% identical isn't what is needed.

Why is that? What problems that proc.addObserverXXX() solve that cannot
be solved with the current ProcTasksObserver setup? I thought you were
suggesting to just add convenient methods to proc to do the same as we
get from ProcTasksObserver. Isn't that the case?

Cheers,

Mark

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]