MI: "^running" issues

MI: "^running" issues

[Vladimir Prus]

I've just run into a somewhat nasty behaviour of gdb.
Suppose if you've set 5 hardware watchpoints (on x86)
and to -exec-step. The output from gdb will be:
	
	(gdb) -exec-next
	^running
	(gdb)
	34^error,msg="...."

The target is not really running at this point, but not that there's no *stopped
message.  This is because the "^running" is in error -- it's actually output
when we only start processing -exec-next.

I also recall the long-standing bug that "^running" is not output for CLI
commands. 

I really wonder if this is hard to fix? Can we just move the printing of "^running" to
target_resume, to catch all cases? Alternatively, we can notice that "^running" has
double duty if identifying that:

	- async operation has started
	- target has started

We can make "^running" more focuses by either:

	- Introducing "*running" notification, that means the target is running.
	- Making "^running" mean that the target has started. As for async/sync
	commands, gdb prompt will indicate new command will be accepted.
	- Killing the idea of async commands. It does not seem to be very useful
	at this point, so we might be better off ripping it and implementing afresh
	later, when it's understood what they should be.

Thoughts?

- Volodya

Re: MI: "^running" issues

[Nick Roberts]

 > I've just run into a somewhat nasty behaviour of gdb.
 > Suppose if you've set 5 hardware watchpoints (on x86)
 > and to -exec-step. The output from gdb will be:
 > 	
 > 	(gdb) -exec-next
 > 	^running
 > 	(gdb)
 > 	34^error,msg="...."

I can't reproduce this.  It looks like partial recipe.

 >...
 >    - Killing the idea of async commands. It does not seem to be very useful
 > at this point, so we might be better off ripping it and implementing afresh
 > later, when it's understood what they should be.

I'm working in the opposite direction: trying to get GDB to work asynchronously.
I don't see the point of ripping out what is already there.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: MI: "^running" issues

[Vladimir Prus]

On Wednesday 05 September 2007 09:23:48 Nick Roberts wrote:
>  > I've just run into a somewhat nasty behaviour of gdb.
>  > Suppose if you've set 5 hardware watchpoints (on x86)
>  > and to -exec-step. The output from gdb will be:
>  > 	
>  > 	(gdb) -exec-next
>  > 	^running
>  > 	(gdb)
>  > 	34^error,msg="...."
> 
> I can't reproduce this.  It looks like partial recipe.

I suppose I can try to make a full recipe -- but only if you're intending to fix that ;-)
 
>  >...
>  >    - Killing the idea of async commands. It does not seem to be very useful
>  > at this point, so we might be better off ripping it and implementing afresh
>  > later, when it's understood what they should be.
> 
> I'm working in the opposite direction: trying to get GDB to work asynchronously.
> I don't see the point of ripping out what is already there.

Then, can you please provide a specification of what "asynchronously" means, and
why it's good for users and frontends?

Thanks,
Volodya

Re: MI: "^running" issues

[Nick Roberts]

 > > I can't reproduce this.  It looks like partial recipe.
 > 
 > I suppose I can try to make a full recipe -- but only if you're intending to
 > fix that ;-)

I can't offer to fix something that I can't even reproduce.  But in any case
you appear to be proposing to `fix' it by moving "^running" to target_resume.
I think we need to establish that the bug is real first.


 > > I'm working in the opposite direction: trying to get GDB to work
 > > asynchronously.  I don't see the point of ripping out what is already
 > > there.
 > 
 > Then, can you please provide a specification of what "asynchronously" means,
 > and why it's good for users and frontends?

To me, it means that GDB can process user input even when the program being
debugged is executing.  Apple use it in Xcode.  It means the frontend can
interrogate GDB at any time.  Also CLI commands give MI async output as
mentioned previously.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: MI: "^running" issues

[Eli Zaretskii]

> From: Vladimir Prus <ghost@cs.msu.su>
> Date: Wed, 5 Sep 2007 09:38:45 +0400
> Cc: gdb@sources.redhat.com
> 
> >  >    - Killing the idea of async commands. It does not seem to be very useful
> >  > at this point, so we might be better off ripping it and implementing afresh
> >  > later, when it's understood what they should be.
> > 
> > I'm working in the opposite direction: trying to get GDB to work asynchronously.
> > I don't see the point of ripping out what is already there.
> 
> Then, can you please provide a specification of what "asynchronously" means, and
> why it's good for users and frontends?

Asynchronous execution means that some GDB commands can be run while
GDB waits for the target to stop.  It is good for users because you
can do something useful while you wait for the target to stop.

Re: MI: "^running" issues

[Vladimir Prus]

Eli Zaretskii wrote:

>> From: Vladimir Prus <ghost@cs.msu.su>
>> Date: Wed, 5 Sep 2007 09:38:45 +0400
>> Cc: gdb@sources.redhat.com
>> 
>> >  >    - Killing the idea of async commands. It does not seem to be very
>> >  >    useful
>> >  > at this point, so we might be better off ripping it and implementing
>> >  > afresh later, when it's understood what they should be.
>> > 
>> > I'm working in the opposite direction: trying to get GDB to work
>> > asynchronously. I don't see the point of ripping out what is already
>> > there.
>> 
>> Then, can you please provide a specification of what "asynchronously"
>> means, and why it's good for users and frontends?
> 
> Asynchronous execution means that some GDB commands can be run while
> GDB waits for the target to stop.  It is good for users because you
> can do something useful while you wait for the target to stop.

Unfortunately, I never saw more concrete details. What commands are actually
meaningful to emit while target are running and are those commands of big
enough value to user to warrant extensive coding? Can interested parties document
those commands at GDB Wiki, or even in email?

- Volodya

Re: MI: "^running" issues

[Eli Zaretskii]

> From:  Vladimir Prus <ghost@cs.msu.su>
> Date:  Wed, 05 Sep 2007 22:41:38 +0400
> 
> > Asynchronous execution means that some GDB commands can be run while
> > GDB waits for the target to stop.  It is good for users because you
> > can do something useful while you wait for the target to stop.
> 
> Unfortunately, I never saw more concrete details.

That's because introduction of asynchronous execution into GDB was
never finished.

> What commands are actually meaningful to emit while target are
> running

Anything that does not need the target itself, or modifies its state.
For example, "help".  A less trivial example is "info break" (to see
what breakpoints were already hit during execution up to now, in case
your "commands" for the breakpoints continue the target).

Note that I'm not actually saying these commands will work
asynchronously in the current GDB, as the implementation of async
execution was never finished, AFAIK.

> and are those commands of big enough value to user to warrant
> extensive coding?

"Big enough" is in the eyes of the beholder.  In principle, every
command in GDB that could take a long time could run in the
background, and GDB could be free to accept other commands in the
meantime.  So you are in effect questioning the value of
multithreading.

As another data point, the people who wrote the infrastructure for the
async execution were two long-time and experienced GDB users and
developers, and they obviously thought it was worth coding.

> Can interested parties document those commands at GDB Wiki, or even
> in email?

The list depends on what we decide to code, so I don't think anyone
can publish it now.

Re: MI: "^running" issues

[Vladimir Prus]

On Thursday 06 September 2007 07:19:11 Eli Zaretskii wrote:
> > From:  Vladimir Prus <ghost@cs.msu.su>
> > Date:  Wed, 05 Sep 2007 22:41:38 +0400
> > 
> > > Asynchronous execution means that some GDB commands can be run while
> > > GDB waits for the target to stop.  It is good for users because you
> > > can do something useful while you wait for the target to stop.
> > 
> > Unfortunately, I never saw more concrete details.
> 
> That's because introduction of asynchronous execution into GDB was
> never finished.
> 
> > What commands are actually meaningful to emit while target are
> > running
> 
> Anything that does not need the target itself, or modifies its state.
> For example, "help".  

I hope you'll agree that "help" is not important enough to warrant
extensive gdb work, so let's drop this example immediately.

> A less trivial example is "info break" (to see 
> what breakpoints were already hit during execution up to now, in case
> your "commands" for the breakpoints continue the target).

Technically speaking, you don't need async for that -- you can interrupt
the target, provide output, and then go on. Making this async will maybe
cut some fraction of section from the run time, why do we care?

> Note that I'm not actually saying these commands will work
> asynchronously in the current GDB, as the implementation of async
> execution was never finished, AFAIK.
> 
> > and are those commands of big enough value to user to warrant
> > extensive coding?
> 
> "Big enough" is in the eyes of the beholder.  In principle, every
> command in GDB that could take a long time could run in the
> background, and GDB could be free to accept other commands in the
> meantime.  

Except for commands that run target, I know of only various get-symbol-list
commands that can take large time, and target memory access (load, disass,
raw memory access).  Are there some other commands?

While those indeed can be parallelized to some degree, I don't particularly see
the end-user use-cases where that's useful.

> So you are in effect questioning the value of 
> multithreading.

Multithreading is not a silver bullet. It should be used when appropriate.
Are you sure gdb code is thread-safe in any degree? 

> As another data point, the people who wrote the infrastructure for the
> async execution were two long-time and experienced GDB users and
> developers, and they obviously thought it was worth coding.

This is argument by authority, I don't accept it.

> > Can interested parties document those commands at GDB Wiki, or even
> > in email?
> 
> The list depends on what we decide to code, so I don't think anyone
> can publish it now.

I think at least some design should be present before coding, so it would
be reasonable that anybody who submits some async patches be ready to
explain the final goal -- what commands will be async, will it be just async,
or multithreaded, and what will be essential benefits for CLI user and for
frontends.

The way it stands now is more like "let's develop async mode, and then see
what commands we can run in async mode, and then see what benefit that
will have".

- Volodya

Re: MI: "^running" issues

[Fabian Cenedese]

>> > What commands are actually meaningful to emit while target are
>> > running
>> 
>> A less trivial example is "info break" (to see 
>> what breakpoints were already hit during execution up to now, in case
>> your "commands" for the breakpoints continue the target).
>
>Technically speaking, you don't need async for that -- you can interrupt
>the target, provide output, and then go on. Making this async will maybe
>cut some fraction of section from the run time, why do we care?

I'm working on embedded targets and a multithreaded gdb would help
for many cases.

- A lot of times the hardware is controlling a machine or some system
  that is highly optimized for speed. Any interruption could disturb the
  process or even throw the whole thing out (Imagine a motor that is
  running and not stopped because the end position was not detected).

- The connection to the target can be Ethernet but also a slow SIO.
  So any communication can take quite some time (for CPUs, not
  for humans). So it may not be just a fraction of a second.

- Even while the target is running it's useful to watch some values.
  These aren't necessarily process variables that can be read by some
  other means as a visualisation might do. gdb with its debug info is
  the only way to get there then. And this is only possible if gdb is
  responding even while the target is running.

- If gdb ever comes to multiprocess debugging it would need to be
  multithreaded as well. One process can be running and the other
  is stopped. Or you need to issue a gdb command to stop a process.

There may be other cases I can't remember now. But I'd surely
welcome a multithreaded gdb. gdb is needed for many cases, not
just a local program on a linux box.

bye  Fabi

Re: MI: "^running" issues

[Mark Kettenis]

> Date: Thu, 06 Sep 2007 09:10:15 +0200
> From: Fabian Cenedese <Cenedese@indel.ch>
> 
> >> > What commands are actually meaningful to emit while target are
> >> > running
> >> 
> >> A less trivial example is "info break" (to see 
> >> what breakpoints were already hit during execution up to now, in case
> >> your "commands" for the breakpoints continue the target).
> >
> >Technically speaking, you don't need async for that -- you can interrupt
> >the target, provide output, and then go on. Making this async will maybe
> >cut some fraction of section from the run time, why do we care?
> 
> I'm working on embedded targets and a multithreaded gdb would help
> for many cases.
> 
> - A lot of times the hardware is controlling a machine or some system
>   that is highly optimized for speed. Any interruption could disturb the
>   process or even throw the whole thing out (Imagine a motor that is
>   running and not stopped because the end position was not detected).
> 
> - The connection to the target can be Ethernet but also a slow SIO.
>   So any communication can take quite some time (for CPUs, not
>   for humans). So it may not be just a fraction of a second.
> 
> - Even while the target is running it's useful to watch some values.
>   These aren't necessarily process variables that can be read by some
>   other means as a visualisation might do. gdb with its debug info is
>   the only way to get there then. And this is only possible if gdb is
>   responding even while the target is running.
> 
> - If gdb ever comes to multiprocess debugging it would need to be
>   multithreaded as well. One process can be running and the other
>   is stopped. Or you need to issue a gdb command to stop a process.
> 
> There may be other cases I can't remember now. But I'd surely
> welcome a multithreaded gdb. gdb is needed for many cases, not
> just a local program on a linux box.

Nothing that you can't solve with non-blocking IO.

Debugging multi-threaded code is bad enough in itself.  You don't need
to make matters worse by making gdb itself less deterministic.

Mark

Re: MI: "^running" issues

[Nick Roberts]

 > > There may be other cases I can't remember now. But I'd surely
 > > welcome a multithreaded gdb. gdb is needed for many cases, not
 > > just a local program on a linux box.
 > 
 > Nothing that you can't solve with non-blocking IO.
 > 
 > Debugging multi-threaded code is bad enough in itself.  You don't need
 > to make matters worse by making gdb itself less deterministic.

Perhaps Fabian is assuming that asynchronus operation requires a multithreaded
gdb.  The async branch that I created was multi-threaded but the patch that I
currently have (under Daniel's guidance) and which I've previously posted to
the gdb-patches mailing list:

http://sourceware.org/ml/gdb-patches/2006-11/msg00225.html

and briefly described in:

http://sourceware.org/ml/gdb-patches/2007-06/msg00315.html

is not.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: MI: "^running" issues

[Bob Rossi]

On Thu, Sep 06, 2007 at 10:46:21AM +0400, Vladimir Prus wrote:
> On Thursday 06 September 2007 07:19:11 Eli Zaretskii wrote:
> > > From:  Vladimir Prus <ghost@cs.msu.su>
> > > Date:  Wed, 05 Sep 2007 22:41:38 +0400
> > A less trivial example is "info break" (to see 
> > what breakpoints were already hit during execution up to now, in case
> > your "commands" for the breakpoints continue the target).
> 
> Technically speaking, you don't need async for that -- you can interrupt
> the target, provide output, and then go on. Making this async will maybe
> cut some fraction of section from the run time, why do we care?

This isn't necessarily true. It is not guarenteed that you will be able
to interupt the target. I'm pretty sure this won't always work with cygwin.

> > Note that I'm not actually saying these commands will work
> > asynchronously in the current GDB, as the implementation of async
> > execution was never finished, AFAIK.
> > 
> > > and are those commands of big enough value to user to warrant
> > > extensive coding?

One example of a command that would be really useful would be 
-exec-interupt. I've discussed this extensively in the past. Sending a
^c to the inferior is not the perfect way to stop the inferior from 
running.  That's because typically a front end will put the inferior on
a tty to separate the output of the inferior from the output of gdb/mi.

As soon as this is done, it becomes unclear if the ^c should be sent to
the gdb/mi tty or the inferior stdin tty. Again, this is especially true
on windows.

> > > Can interested parties document those commands at GDB Wiki, or even
> > > in email?
> > 
> > The list depends on what we decide to code, so I don't think anyone
> > can publish it now.
> 
> I think at least some design should be present before coding, so it would
> be reasonable that anybody who submits some async patches be ready to
> explain the final goal -- what commands will be async, will it be just async,
> or multithreaded, and what will be essential benefits for CLI user and for
> frontends.
> 
> The way it stands now is more like "let's develop async mode, and then see
> what commands we can run in async mode, and then see what benefit that
> will have".

Yeah, in general, I think this is how the MI stuff is developed though.
There doesn't seem to be anyone with a clear vision dedicated to gdb/mi. 
It's unfortunate. I mean, each front end developer of gdb/mi eventually 
becomes a gdb/mi developer. It's pretty sad.

Bob Rossi

Re: MI: "^running" issues

[Jim Blandy]

Vladimir Prus <ghost@cs.msu.su> writes:
> Unfortunately, I never saw more concrete details. What commands are actually
> meaningful to emit while target are running and are those commands of big
> enough value to user to warrant extensive coding? Can interested parties document
> those commands at GDB Wiki, or even in email?

It would be nice if GDB could stop one thread and examine its state
while allowing the other threads to continue running.  GDB's
asynchronous mode is a prerequisite for that.

Re: MI: "^running" issues

[Vladimir Prus]

Bob Rossi wrote:

> On Thu, Sep 06, 2007 at 10:46:21AM +0400, Vladimir Prus wrote:
>> On Thursday 06 September 2007 07:19:11 Eli Zaretskii wrote:
>> > > From:  Vladimir Prus <ghost@cs.msu.su>
>> > > Date:  Wed, 05 Sep 2007 22:41:38 +0400
>> > A less trivial example is "info break" (to see
>> > what breakpoints were already hit during execution up to now, in case
>> > your "commands" for the breakpoints continue the target).
>> 
>> Technically speaking, you don't need async for that -- you can interrupt
>> the target, provide output, and then go on. Making this async will maybe
>> cut some fraction of section from the run time, why do we care?
> 
> This isn't necessarily true. It is not guarenteed that you will be able
> to interupt the target. I'm pretty sure this won't always work with
> cygwin.

Given that Visual Studio can interrupt running process, what you describe
above appears to be cygwin problem, not inherent problem with windows. 
Therefore, it probably should be fixed in cygwin. There's no need to
change gdb.

>> > Note that I'm not actually saying these commands will work
>> > asynchronously in the current GDB, as the implementation of async
>> > execution was never finished, AFAIK.
>> > 
>> > > and are those commands of big enough value to user to warrant
>> > > extensive coding?
> 
> One example of a command that would be really useful would be
> -exec-interupt. I've discussed this extensively in the past. Sending a
> ^c to the inferior is not the perfect way to stop the inferior from
> running.  That's because typically a front end will put the inferior on
> a tty to separate the output of the inferior from the output of gdb/mi.
> 
> As soon as this is done, it becomes unclear if the ^c should be sent to
> the gdb/mi tty or the inferior stdin tty. Again, this is especially true
> on windows.

Sendind ^C to gdb/mi tty appears to work fine (with program on separate tty).
What's the problem?
 
>> The way it stands now is more like "let's develop async mode, and then
>> see what commands we can run in async mode, and then see what benefit
>> that will have".
> 
> Yeah, in general, I think this is how the MI stuff is developed though.
> There doesn't seem to be anyone with a clear vision dedicated to gdb/mi.
> It's unfortunate. I mean, each front end developer of gdb/mi eventually
> becomes a gdb/mi developer. It's pretty sad.

It is. But it does not mean future MI development should be in similar vein.

- Volodya

Re: MI: "^running" issues

[Jim Ingham]

We got the asynchronous mode working for the Mac OS X target in the  
Apple gdb sources.  It definitely took a bunch of work beyond what is  
in the current FSF sources to get all the details working (things like  
breakpoint commands that restart the target and command files and some  
other bits like that needed attending to...)  It wasn't deadly hard to  
do, though.

We use it mostly so that Xcode can query the target's state at any  
point - basically a backstop when it looks like maybe the UI and gdb  
have gotten out of sync.  That way you can always unambiguously get  
the tri-state answer - the target's stopped; the target's running;  
gdb's gone south.  This is actually pretty useful to have around,  
though in a perfect world would not be necessary...

We also use -exec-interrupt.  It's certainly true that you can achieve  
the same thing by sending ^C to the target, but it's much more regular  
to do everything you're doing with the target through the same control  
channel.  And of course, this moves to gdb the knowledge of any  
funniness with interrupting a running program - which is where it  
belongs.

Another stronger reason why async was originally done was the  
experience of merging gdb with Tcl/Tk for the Insight debugger.  One  
of the big reasons for the instability of that project is that when  
the target is running, gdb is blocked, so if you want to service other  
events - like window system events in the case of Insight - you've got  
no good way to do that.  We ended up having to run a timer and try to  
service events in the timer interrupt - a dubious practice at best.

It's possible to run the interpreter in a separate thread, and let it  
and gdb communicate through some side channel.  But most interpreters  
have some mechanism for handling events, and it is much cleaner to  
have gdb be just another event source.

I'm not sure that the current gdb event mechanism is clean enough for  
easy incorporation with <Insert your favorite command language>'s  
event loop now.  But that just means there's more design work to do.   
I still think this is a pretty worthwhile goal.

Jim



On Sep 5, 2007, at 11:41 AM, Vladimir Prus wrote:

> Eli Zaretskii wrote:
>
>>> From: Vladimir Prus <ghost@cs.msu.su>
>>> Date: Wed, 5 Sep 2007 09:38:45 +0400
>>> Cc: gdb@sources.redhat.com
>>>
>>>>>   - Killing the idea of async commands. It does not seem to be  
>>>>> very
>>>>>   useful
>>>>> at this point, so we might be better off ripping it and  
>>>>> implementing
>>>>> afresh later, when it's understood what they should be.
>>>>
>>>> I'm working in the opposite direction: trying to get GDB to work
>>>> asynchronously. I don't see the point of ripping out what is  
>>>> already
>>>> there.
>>>
>>> Then, can you please provide a specification of what  
>>> "asynchronously"
>>> means, and why it's good for users and frontends?
>>
>> Asynchronous execution means that some GDB commands can be run while
>> GDB waits for the target to stop.  It is good for users because you
>> can do something useful while you wait for the target to stop.
>
> Unfortunately, I never saw more concrete details. What commands are  
> actually
> meaningful to emit while target are running and are those commands  
> of big
> enough value to user to warrant extensive coding? Can interested  
> parties document
> those commands at GDB Wiki, or even in email?
>
> - Volodya
>
>

Re: MI: "^running" issues

[Vladimir Prus]

Jim Ingham wrote:

> We got the asynchronous mode working for the Mac OS X target in the
> Apple gdb sources.  It definitely took a bunch of work beyond what is
> in the current FSF sources to get all the details working (things like
> breakpoint commands that restart the target and command files and some
> other bits like that needed attending to...)  It wasn't deadly hard to
> do, though.
> 
> We use it mostly so that Xcode can query the target's state at any
> point - basically a backstop when it looks like maybe the UI and gdb
> have gotten out of sync.  That way you can always unambiguously get
> the tri-state answer - the target's stopped; the target's running;
> gdb's gone south.  This is actually pretty useful to have around,
> though in a perfect world would not be necessary...

I see. So, the problem is that in real world those "^running" and
"*stopped" are not necessary always output when needed, so you can
get out of sync?

> We also use -exec-interrupt.  It's certainly true that you can achieve
> the same thing by sending ^C to the target, but it's much more regular
> to do everything you're doing with the target through the same control
> channel.  And of course, this moves to gdb the knowledge of any
> funniness with interrupting a running program - which is where it
> belongs.

Hmm, if you send ^C to gdb, then gdb can handle interrupting the program
just fine.
 
> Another stronger reason why async was originally done was the
> experience of merging gdb with Tcl/Tk for the Insight debugger.  One
> of the big reasons for the instability of that project is that when
> the target is running, gdb is blocked, so if you want to service other
> events - like window system events in the case of Insight - you've got
> no good way to do that.  We ended up having to run a timer and try to
> service events in the timer interrupt - a dubious practice at best.
> 
> It's possible to run the interpreter in a separate thread, and let it
> and gdb communicate through some side channel.  But most interpreters
> have some mechanism for handling events, and it is much cleaner to
> have gdb be just another event source.

I presume this is only relevant when gdb is combined with some other
code in the same process? Last time I've asked about this, I was
told that it not supported, because gdb does fancy things with signals,
or something like that.

- Volodya

Re: MI: "^running" issues

[Jim Ingham]

On Sep 6, 2007, at 11:07 AM, Vladimir Prus wrote:

> Jim Ingham wrote:
>
>> We got the asynchronous mode working for the Mac OS X target in the
>> Apple gdb sources.  It definitely took a bunch of work beyond what is
>> in the current FSF sources to get all the details working (things  
>> like
>> breakpoint commands that restart the target and command files and  
>> some
>> other bits like that needed attending to...)  It wasn't deadly hard  
>> to
>> do, though.
>>
>> We use it mostly so that Xcode can query the target's state at any
>> point - basically a backstop when it looks like maybe the UI and gdb
>> have gotten out of sync.  That way you can always unambiguously get
>> the tri-state answer - the target's stopped; the target's running;
>> gdb's gone south.  This is actually pretty useful to have around,
>> though in a perfect world would not be necessary...
>
> I see. So, the problem is that in real world those "^running" and
> "*stopped" are not necessary always output when needed, so you can
> get out of sync?

Those are always bugs, and we fix them when we find them, but yes that  
did happen - hasn't happened in a while though.  What happens more  
often now for us is that we rely a lot on calling functions to inspect  
opaque data types when the target stops.  This often goes bad - people  
right data inspectors that can deadlock for instance...  So it's  
useful to have some "resync with gdb" action  that check's gdb's state  
and does the right thing based on what it is.

>
>
>> We also use -exec-interrupt.  It's certainly true that you can  
>> achieve
>> the same thing by sending ^C to the target, but it's much more  
>> regular
>> to do everything you're doing with the target through the same  
>> control
>> channel.  And of course, this moves to gdb the knowledge of any
>> funniness with interrupting a running program - which is where it
>> belongs.
>
> Hmm, if you send ^C to gdb, then gdb can handle interrupting the  
> program
> just fine.

Sure, but you're still using two ways to talk to gdb.

>
>
>> Another stronger reason why async was originally done was the
>> experience of merging gdb with Tcl/Tk for the Insight debugger.  One
>> of the big reasons for the instability of that project is that when
>> the target is running, gdb is blocked, so if you want to service  
>> other
>> events - like window system events in the case of Insight - you've  
>> got
>> no good way to do that.  We ended up having to run a timer and try to
>> service events in the timer interrupt - a dubious practice at best.
>>
>> It's possible to run the interpreter in a separate thread, and let it
>> and gdb communicate through some side channel.  But most interpreters
>> have some mechanism for handling events, and it is much cleaner to
>> have gdb be just another event source.
>
> I presume this is only relevant when gdb is combined with some other
> code in the same process? Last time I've asked about this, I was
> told that it not supported, because gdb does fancy things with  
> signals,
> or something like that.

Yes, this would be to support an in-process interpreter.  But I don't  
know about problems with signals.  That part didn't cause Tcl any  
difficulty, but that was a long time ago too.

Jim

Re: MI: "^running" issues

[Vladimir Prus]

On Thursday 06 September 2007 22:34:08 Jim Ingham wrote:
> 
> On Sep 6, 2007, at 11:07 AM, Vladimir Prus wrote:
> 
> > Jim Ingham wrote:
> >
> >> We got the asynchronous mode working for the Mac OS X target in the
> >> Apple gdb sources.  It definitely took a bunch of work beyond what is
> >> in the current FSF sources to get all the details working (things  
> >> like
> >> breakpoint commands that restart the target and command files and  
> >> some
> >> other bits like that needed attending to...)  It wasn't deadly hard  
> >> to
> >> do, though.
> >>
> >> We use it mostly so that Xcode can query the target's state at any
> >> point - basically a backstop when it looks like maybe the UI and gdb
> >> have gotten out of sync.  That way you can always unambiguously get
> >> the tri-state answer - the target's stopped; the target's running;
> >> gdb's gone south.  This is actually pretty useful to have around,
> >> though in a perfect world would not be necessary...
> >
> > I see. So, the problem is that in real world those "^running" and
> > "*stopped" are not necessary always output when needed, so you can
> > get out of sync?
> 
> Those are always bugs, and we fix them when we find them, but yes that  
> did happen - hasn't happened in a while though.  What happens more  
> often now for us is that we rely a lot on calling functions to inspect  
> opaque data types when the target stops.  This often goes bad - people  
> right data inspectors that can deadlock for instance...  So it's  
> useful to have some "resync with gdb" action  that check's gdb's state  
> and does the right thing based on what it is.

You mean, calling functions in the program? Then, presumably if they
hang, you get to interrupt their execution and somehow restore program
context? Why do you need to know if inferior is running, in this case?
As soon as you call a function in program, program is running ;-)

> >> We also use -exec-interrupt.  It's certainly true that you can  
> >> achieve
> >> the same thing by sending ^C to the target, but it's much more  
> >> regular
> >> to do everything you're doing with the target through the same  
> >> control
> >> channel.  And of course, this moves to gdb the knowledge of any
> >> funniness with interrupting a running program - which is where it
> >> belongs.
> >
> > Hmm, if you send ^C to gdb, then gdb can handle interrupting the  
> > program
> > just fine.
> 
> Sure, but you're still using two ways to talk to gdb.

Yes, MI and less nice ^C, over same pipe. I see how that's not
ideal, but IMO is a minor issue.

> > I presume this is only relevant when gdb is combined with some other
> > code in the same process? Last time I've asked about this, I was
> > told that it not supported, because gdb does fancy things with  
> > signals,
> > or something like that.
> 
> Yes, this would be to support an in-process interpreter.  But I don't  
> know about problems with signals.  That part didn't cause Tcl any  
> difficulty, but that was a long time ago too.

I don't know either, it's vague recollection.

- Volodya

Re: MI: "^running" issues

[Eli Zaretskii]

> From: Vladimir Prus <ghost@cs.msu.su>
> Date: Thu, 6 Sep 2007 10:46:21 +0400
> Cc: gdb@sources.redhat.com
> 
> > > What commands are actually meaningful to emit while target are
> > > running
> > 
> > Anything that does not need the target itself, or modifies its state.
> > For example, "help".  
> 
> I hope you'll agree that "help" is not important enough to warrant
> extensive gdb work, so let's drop this example immediately.

Actually, I disagree that it's entirely not important.  I frequently
want to remind myself something about the next command I want to
issue, and doing that while the inferior is running is a good use of
my otherwise idle time.

> > A less trivial example is "info break" (to see 
> > what breakpoints were already hit during execution up to now, in case
> > your "commands" for the breakpoints continue the target).
> 
> Technically speaking, you don't need async for that -- you can interrupt
> the target, provide output, and then go on.

If the target is timing-sensitive, interrupting it is not a good idea,
as it can disrupt the timing and cause all kinds of side effects that
will ruin your debug session.

> > So you are in effect questioning the value of 
> > multithreading.
> 
> Multithreading is not a silver bullet. It should be used when appropriate.
> Are you sure gdb code is thread-safe in any degree? 

GDB code is obviously not thread-safe (we have quite a few global
variables).  But then I didn't suggest to use threads, only that
questioning its limited emulation by the async code is akin to
questioning the value of multithreading.  (I'm sure you know that any
program that uses `select' or `poll' is multithreading in disguise.)

Look, it's quite clear at this point that you've made up your mind
about this issue: you are against the async option.  It's not
necessary to underscore that by attacking every piece of information I
give you.

> > As another data point, the people who wrote the infrastructure for the
> > async execution were two long-time and experienced GDB users and
> > developers, and they obviously thought it was worth coding.
> 
> This is argument by authority, I don't accept it.

You are free to reject it, but then I'm free to object to removing the
async code just because you think it's useless: your authority is no
more important than that of those who suggested and developed the
async code.

> I think at least some design should be present before coding

``Some design'' _was_ presented years ago, when this code was
developed.  You may wish to look in the archives.

> The way it stands now is more like "let's develop async mode, and then see
> what commands we can run in async mode, and then see what benefit that
> will have".

Thank you so much for mocking my attempt to help you understand the
issue, at your own request.

Re: MI: "^running" issues

[Vladimir Prus]

On Thursday 06 September 2007 22:48:43 Eli Zaretskii wrote:
> > From: Vladimir Prus <ghost@cs.msu.su>
> > Date: Thu, 6 Sep 2007 10:46:21 +0400
> > Cc: gdb@sources.redhat.com
> > 
> > > > What commands are actually meaningful to emit while target are
> > > > running
> > > 
> > > Anything that does not need the target itself, or modifies its state.
> > > For example, "help".  
> > 
> > I hope you'll agree that "help" is not important enough to warrant
> > extensive gdb work, so let's drop this example immediately.
> 
> Actually, I disagree that it's entirely not important.  I frequently
> want to remind myself something about the next command I want to
> issue, and doing that while the inferior is running is a good use of
> my otherwise idle time.

OK.

> > > A less trivial example is "info break" (to see 
> > > what breakpoints were already hit during execution up to now, in case
> > > your "commands" for the breakpoints continue the target).
> > 
> > Technically speaking, you don't need async for that -- you can interrupt
> > the target, provide output, and then go on.
> 
> If the target is timing-sensitive, interrupting it is not a good idea,
> as it can disrupt the timing and cause all kinds of side effects that
> will ruin your debug session.

It appears to me that the time to interrupt the target, print output
and resume is roughly equal to the time it takes to a hit of breakpoint
with commands. You'd need a minimal degree of asynchronous operation to
support that.

> > > So you are in effect questioning the value of 
> > > multithreading.
> > 
> > Multithreading is not a silver bullet. It should be used when appropriate.
> > Are you sure gdb code is thread-safe in any degree? 
> 
> GDB code is obviously not thread-safe (we have quite a few global
> variables).  But then I didn't suggest to use threads, only that
> questioning its limited emulation by the async code is akin to
> questioning the value of multithreading.  (I'm sure you know that any
> program that uses `select' or `poll' is multithreading in disguise.)

I actually thought you mean multi-threading, literally. Without MT,
you can basically run one command on gdb side, together with one
command running inferior. With MT, it would be possible to run several
long running commands on gdb side, which I thought you were talking about.

> > > As another data point, the people who wrote the infrastructure for the
> > > async execution were two long-time and experienced GDB users and
> > > developers, and they obviously thought it was worth coding.
> > 
> > This is argument by authority, I don't accept it.
> 
> You are free to reject it, but then I'm free to object to removing the
> async code just because you think it's useless: your authority is no
> more important than that of those who suggested and developed the
> async code.

Does it work now? Are there tests for it? And basically, I don't
care much about existing code. What I'm concerned is that I have
a real bug, where "^running" is not output where it should be.
The code related to that bug has something to do with async mode,
but it's pretty unclear why it should be, and how would I test
that my fixes don't break that async mode.
 
> > I think at least some design should be present before coding
> 
> ``Some design'' _was_ presented years ago, when this code was
> developed.  You may wish to look in the archives.
> 
> > The way it stands now is more like "let's develop async mode, and then see
> > what commands we can run in async mode, and then see what benefit that
> > will have".
> 
> Thank you so much for mocking my attempt to help you understand the
> issue, at your own request.

I'm sorry that you feel that way. Unfortunately, I continue to believe
that in order to talk about async mode, we need some design, readily 
accessible to all interested parties. I failed to find such design
in the archives; and from this discussion it appears that the value
of async mode lies in different things for different people.

- Volodya

Re: MI: "^running" issues

[Vladimir Prus]

Fabian Cenedese wrote:

> 
>>> > What commands are actually meaningful to emit while target are
>>> > running
>>> 
>>> A less trivial example is "info break" (to see
>>> what breakpoints were already hit during execution up to now, in case
>>> your "commands" for the breakpoints continue the target).
>>
>>Technically speaking, you don't need async for that -- you can interrupt
>>the target, provide output, and then go on. Making this async will maybe
>>cut some fraction of section from the run time, why do we care?
> 
> I'm working on embedded targets and a multithreaded gdb would help
> for many cases.
> 
> - A lot of times the hardware is controlling a machine or some system
>   that is highly optimized for speed. Any interruption could disturb the
>   process or even throw the whole thing out (Imagine a motor that is
>   running and not stopped because the end position was not detected).
> 
> - The connection to the target can be Ethernet but also a slow SIO.
>   So any communication can take quite some time (for CPUs, not
>   for humans). So it may not be just a fraction of a second.

Do you have a communication channel that can talk with the device
while the program is running? 

> - Even while the target is running it's useful to watch some values.
>   These aren't necessarily process variables that can be read by some
>   other means as a visualisation might do. gdb with its debug info is
>   the only way to get there then. And this is only possible if gdb is
>   responding even while the target is running.

This seems a valid usecase; I can find essentially the same request
in the archive.

- Volodya

Re: MI: "^running" issues

[André Pönitz]

On Thursday 06 September 2007 20:41:01 Vladimir Prus wrote:
> > Those are always bugs, and we fix them when we find them, but yes that  
> > did happen - hasn't happened in a while though.  What happens more  
> > often now for us is that we rely a lot on calling functions to inspect  
> > opaque data types when the target stops.  This often goes bad - people  
> > right data inspectors that can deadlock for instance...  So it's  
> > useful to have some "resync with gdb" action  that check's gdb's state  
> > and does the right thing based on what it is.
> 
> You mean, calling functions in the program? Then, presumably if they
> hang, you get to interrupt their execution and somehow restore program
> context? Why do you need to know if inferior is running, in this case?
> As soon as you call a function in program, program is running ;-)

For a while. Then it may stop. Most times you get a notice. Sometimes
you don't. Sometimes (most notably gdb/cygwin) the notice gets mangled
with whatever output the called "data dumper function" produces and
it is no more recognizable by the "result parser". Sometimes "old" output
arrives after calling into the inferior.

Some of the scenarios might point to bugs in the Gui, some might not.
In any case, there are lots of reasons why one might want a method to 
get absolute state information out of Gdb at "random" times.

> > > I presume this is only relevant when gdb is combined with some other
> > > code in the same process? Last time I've asked about this, I was
> > > told that it not supported, because gdb does fancy things with  
> > > signals, or something like that.

It works well enough for simple cases. Most troubles I have had so far
is with memory allocation and result passing on Windows. So if one 
manages to do the work without dynamic allocations and finds a way
that does not mangle "data dumper output" with ordinary gdb output,
the method is fairly robust. Most notably it also works when the data
dumper segfaults.

Of course, having a "real" interpreter within Gdb would be really nice...

Andre'

Re: MI: "^running" issues

[Nick Roberts]

 > > As another data point, the people who wrote the infrastructure for the
 > > async execution were two long-time and experienced GDB users and
 > > developers, and they obviously thought it was worth coding.
 > 
 > This is argument by authority, I don't accept it.

This style of argument is quite antagonistic especially when givien to those
who are trying to help you.  I'm not sure that you've demonstrated the need
to remove the asynchronous code and the onus is on you to try to understand
it, not on others to explain it to you.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: MI: "^running" issues

[Eli Zaretskii]

> From: Vladimir Prus <ghost@cs.msu.su>
> Date: Thu, 6 Sep 2007 23:34:41 +0400
> Cc: gdb@sources.redhat.com
> 
> > > > A less trivial example is "info break" (to see 
> > > > what breakpoints were already hit during execution up to now, in case
> > > > your "commands" for the breakpoints continue the target).
> > > 
> > > Technically speaking, you don't need async for that -- you can interrupt
> > > the target, provide output, and then go on.
> > 
> > If the target is timing-sensitive, interrupting it is not a good idea,
> > as it can disrupt the timing and cause all kinds of side effects that
> > will ruin your debug session.
> 
> It appears to me that the time to interrupt the target, print output
> and resume is roughly equal to the time it takes to a hit of breakpoint
> with commands.

That's probably true, but consider this: when I put a breakpoint in
some place, I presumably know what I'm doing, and can plan my
breakpoints and the associated commands so that they don't disrupt the
program's operation.  By contrast, an interrupt at some random time
cannot be planned like that.

> And basically, I don't care much about existing code.

Well, I do, especially given the people who wrote it.

> What I'm concerned is that I have
> a real bug, where "^running" is not output where it should be.

If there is a bug, it should be fixed, there's no doubt about that.

> The code related to that bug has something to do with async mode,
> but it's pretty unclear why it should be, and how would I test
> that my fixes don't break that async mode.

Well, Nick is trying to improve on the async code, perhaps he could
help understand the offending code and find the right fix for the
problem you found.

Re: MI: "^running" issues

[Nick Roberts]

 > > The code related to that bug has something to do with async mode,
 > > but it's pretty unclear why it should be, and how would I test
 > > that my fixes don't break that async mode.
 > 
 > Well, Nick is trying to improve on the async code, perhaps he could
 > help understand the offending code and find the right fix for the
 > problem you found.

I've run Vladimir's example under GDB with my async patch and it also printed a
^running record and no *stopped, so perhaps ^running should indeed be printed
later.  However, the best way to get the right asynchronous MI output is
probably to develop this code.

I could create another branch for it but I suspect that it wouldn't get looked
at.  I would like to commit it to mainline after the release but Daniel doesn't
like this idea as some changes are copied verbatim from Apple.  However, I've
tried to confine these changes to the "--async" option so that usual
(synchronous) operation is unchanged.

-- 
Nick http://www.inet.net.nz/~nickrob

Re: MI: "^running" issues

[Vladimir Prus]

On Friday 07 September 2007 13:03:57 Nick Roberts wrote:
>  > > The code related to that bug has something to do with async mode,
>  > > but it's pretty unclear why it should be, and how would I test
>  > > that my fixes don't break that async mode.
>  > 
>  > Well, Nick is trying to improve on the async code, perhaps he could
>  > help understand the offending code and find the right fix for the
>  > problem you found.
> 
> I've run Vladimir's example under GDB with my async patch and it also printed a
> ^running record and no *stopped, so perhaps ^running should indeed be printed
> later.  However, the best way to get the right asynchronous MI output is
> probably to develop this code.

Why? It seems to me that outputting "^running" only when the target is
running is completely different matter from being able to enter more
commands when the target is running. I don't see why we can fix "^running" now,
so that folks who are either not interested in asynchronous mode, or don't
like to wait for it, can get right behaviour now?

> I could create another branch for it but I suspect that it wouldn't get looked
> at.  I would like to commit it to mainline after the release but Daniel doesn't
> like this idea as some changes are copied verbatim from Apple.  However, I've
> tried to confine these changes to the "--async" option so that usual
> (synchronous) operation is unchanged.
 
Assuming that 

	http://sourceware.org/ml/gdb-patches/2006-11/msg00225.html
	http://thread.gmane.org/gmane.comp.gdb.patches/31081

are the most recent discussions about your patch, it does not seems like
"copied verbatim from Apple" is the problem. The problem is that is a big patch,
and:

	- I can't find high-level overview of what the patch is trying
	to do, and how it changes gdb behaviour. While a doc patch
	might be premature, some text file would be great.
	- There are no tests to come with the patch, which makes it
	even harder to understand what are you aiming at.

That's pretty much what I was complaining recently -- without a design
doc for async mode it's not only impossible to understand the existing
code, but it's also impossible to understand the code that you're proposing.

- Volodya

Re: MI: "^running" issues

[Eli Zaretskii]

> From: Vladimir Prus <ghost@cs.msu.su>
> Date: Fri, 7 Sep 2007 14:04:13 +0400
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  gdb@sources.redhat.com
> 
> That's pretty much what I was complaining recently -- without a design
> doc for async mode it's not only impossible to understand the existing
> code, but it's also impossible to understand the code that you're proposing.

We don't require a design doc for any other change, and I don't see
why the async mode should be any different.

Re: MI: "^running" issues

[Daniel Jacobowitz]

On Fri, Sep 07, 2007 at 08:56:06PM +0300, Eli Zaretskii wrote:
> > From: Vladimir Prus <ghost@cs.msu.su>
> > Date: Fri, 7 Sep 2007 14:04:13 +0400
> > Cc: Eli Zaretskii <eliz@gnu.org>,
> >  gdb@sources.redhat.com
> > 
> > That's pretty much what I was complaining recently -- without a design
> > doc for async mode it's not only impossible to understand the existing
> > code, but it's also impossible to understand the code that you're proposing.
> 
> We don't require a design doc for any other change, and I don't see
> why the async mode should be any different.

We ask for more information than we've got in this case, though.  And
you try to get internals chapters written for things like this :-)

We're in a tricky situation.  The developers of async mode were
respected contributors - but they're not here any more to explain
their work.  It doesn't work as-is, so we can't test it.  And Nick's
patches to bring it into shape require someone who really understands
what they're doing to review them.  That's not going to be me, since I
couldn't figure the patches out when I last tried.

-- 
Daniel Jacobowitz
CodeSourcery

Re: MI: "^running" issues

[Eli Zaretskii]

> Date: Fri, 7 Sep 2007 14:05:48 -0400
> From: Daniel Jacobowitz <drow@false.org>
> Cc: Vladimir Prus <ghost@cs.msu.su>, nickrob@snap.net.nz,
> 	gdb@sources.redhat.com
> 
> We ask for more information than we've got in this case, though.

I don't think we do, not in all cases, anyway.

> And you try to get internals chapters written for things like this
> :-)

Yes.  But I can hardly say I succeed, given the current shape of
gdbint.texinfo.  And unfortunately I cannot say I get too much
community support for this Quixotic battle, either.  So my internal
sense of justice does not let me demand Nick (or anyone else who knows
something about the async mode) to produce more design info than we
ask from others, as a prerequisite for accepting patches.

> We're in a tricky situation.

Yes.  But I'm not yet ready to give up and agree to ripping the async
mode out of GDB.

Re: MI: "^running" issues

[Daniel Jacobowitz]

On Fri, Sep 07, 2007 at 09:18:34PM +0300, Eli Zaretskii wrote:
> > We're in a tricky situation.
> 
> Yes.  But I'm not yet ready to give up and agree to ripping the async
> mode out of GDB.

I'm not suggesting that, not yet anyway.  But unless someone
understands (and is able to explain) how it works, it's going to go
from broken to completely broken.

-- 
Daniel Jacobowitz
CodeSourcery

Re: MI: "^running" issues

[Nick Roberts]

 > We're in a tricky situation.  The developers of async mode were
 > respected contributors - but they're not here any more to explain
 > their work.  It doesn't work as-is, so we can't test it.  And Nick's
 > patches to bring it into shape require someone who really understands
 > what they're doing to review them.  That's not going to be me, since I
 > couldn't figure the patches out when I last tried.

I don't really see much difference between GDB having code that we don't fully
understand, and installing code that we don't fully understand to use it: in
both cases we rely on the testsuite.  The difference being that, if my changes
are installed, asynchronous operation will get tested and over time the code
will improve.

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: MI: "^running" issues

[Daniel Jacobowitz]

On Sat, Sep 08, 2007 at 12:30:17PM +1200, Nick Roberts wrote:
>  > We're in a tricky situation.  The developers of async mode were
>  > respected contributors - but they're not here any more to explain
>  > their work.  It doesn't work as-is, so we can't test it.  And Nick's
>  > patches to bring it into shape require someone who really understands
>  > what they're doing to review them.  That's not going to be me, since I
>  > couldn't figure the patches out when I last tried.
> 
> I don't really see much difference between GDB having code that we don't fully
> understand, and installing code that we don't fully understand to use it: in
> both cases we rely on the testsuite.  The difference being that, if my changes
> are installed, asynchronous operation will get tested and over time the code
> will improve.

When the code was originally installed, by its authors, they
understood it.  By committing it, they declared that it was right
(at the time, in the current context, et cetera).  Code that we
don't understand doesn't have that assurance.

I think it's a pretty big difference, but as I said, I am not likely
to review such patches myself since I don't understand them.  And I'm
trying to cut back on the time I spent on review over the past few
years.  If another maintainer reviews them and has a different
opinion, I'm not going to get in the way.

-- 
Daniel Jacobowitz
CodeSourcery

Re: MI: "^running" issues

[Nick Roberts]

 > > I've run Vladimir's example under GDB with my async patch and it also
 > > printed a ^running record and no *stopped, so perhaps ^running should
 > > indeed be printed later.  However, the best way to get the right
 > > asynchronous MI output is probably to develop this code.
 > 
 > Why? It seems to me that outputting "^running" only when the target is
 > running is completely different matter from being able to enter more
 > commands when the target is running. 

I mean more generally than when "^running" is printed, e.g. "next" returning
"^done" instead of "*stopped".

 >                                      I don't see why we can fix "^running"
 > now, so that folks who are either not interested in asynchronous mode, or
 > don't like to wait for it, can get right behaviour now?

If there is a simple fix then it can only make sense.  However, I dont think
this should require removing existing async code.

 >...
 > Assuming that 
 > 
 > 	http://sourceware.org/ml/gdb-patches/2006-11/msg00225.html
 > 	http://thread.gmane.org/gmane.comp.gdb.patches/31081
 > 
 > are the most recent discussions about your patch, it does not seems like
 > "copied verbatim from Apple" is the problem. The problem is that is a big
 > patch,

diffstat (without ChangeLog) gives something like:

 20 files changed, 388 insertions, 25 deletions, 221 modifications

 > and:
 > 
 > 	- I can't find high-level overview of what the patch is trying
 > 	to do, and how it changes gdb behaviour. While a doc patch
 > 	might be premature, some text file would be great.

I've already spent a lot of time on this patch and don't plan to spend much
more when there is no maintainer willing/able to review it.  I'll just keep it
in sync with the repository and maybe look at it more closely again if interest
increases.

 > 	- There are no tests to come with the patch, which makes it
 > 	even harder to understand what are you aiming at.

There's a test attached to msg00225.html listed above.  Also, as I think I've
said before, to run testsuite with the asynchronous event loop just put the
line:

set GDBFLAGS "--async"

at the bottom of site.exp.

at the end of site.exp for the tests.  Asynchronous output from MI requires
new tests but I haven't included these in the diff.

 > That's pretty much what I was complaining recently -- without a design
 > doc for async mode it's not only impossible to understand the existing
 > code, but it's also impossible to understand the code that you're proposing.

I can't write a design doc for code that others have already written.  It
doesn't make it impossible to understand, just more difficult.

-- 
Nick                                           http://www.inet.net.nz/~nickrob