non-blocking reads/writes and event loops

non-blocking reads/writes and event loops

[Andrew Cagney]

Hello,

[this is a cross post, reply-to has been set to gdb@sourceware]

Two architectural changes introduced in 5.0 were the the event-loop and
the targets ``remote async'' and ``remote extended-async''.  J.T.'s
recent posting about memory read/write raises a second significant issue
related to the event-loops, GUI's and async but first some background.

The event-loop provides a mechanism that allows GDB to block on more
than one input (or output) source.  For instance, GDB might be blocked
waiting for both the remote target and the user input.  As events
arrive, the event-loop dispatches them.

The async/extended-async targets exploit the event-loop so that, while
the target is running, the user can still enter various commands.  For
instance:

	(gdb) target extended-async ....
	(gdb) run&
	(gdb)
	(gdb) help ...
	....
	(gdb) 
	Breakpoint #1 reached, ...
	(gdb)

In getting this working, numerous problems were identified and several
compromizes were made (look for FIXMEs, especially in remote.c).

The main compromize is to do with when the target has halted.  While the
target is running the async target code returns control to the main
event-loop.  When the target starts ending it stopped packet, or when
the target is assumed to be halted, the code revers to polling.  Memory
reads/writes and the like are blocking operations - they do not return
control to the event loop.  For instance:

	(gdb) print foo

	gdb calls target-vector to read memory (&foo, sizeof(foo))
		target-vector ``calls'' remote to read memory from &foo
		target-vector blocks
		target-vector receives data from remote
	gdb receives data from target-vector
	
	$1 = "foo"		
	(gdb)

The rationale behind the decision to block in this case was very
pragmatic - it wasn't going to be feasible to invert (as was done to
serial.c) all of GDB (so that everything was event based) in a single
hit.

The consequence is that, during these blockages, GDB is slower than
dring normal operation - for the CLI the problem isn't too noticable. 
For the GUI, however, it is very noticable.  As a result, the built in
GUI (ex insight) is forced to use other means to avoid this slugishness
- during those blocked reads a timer is set up and that is used to
process the GUI events (see ui_loop_hook() and
gdb/gdbtk/generic/gdbtk-hooks.c:x_event()).

The thing that needs to be decided is how far GDB should be pushed to
address this problem.  Should GDB continue to be pushed to the point
where everything is event based or should, the current compromise remain
where a GUI is unable to exploit GDBs event-loop.

	Andrew

Re: non-blocking reads/writes and event loops

[Todd Whitesel]

> The thing that needs to be decided is how far GDB should be pushed to
> address this problem.  Should GDB continue to be pushed to the point
> where everything is event based or should, the current compromise remain
> where a GUI is unable to exploit GDBs event-loop.

I am a advocate of long-time eventification. We should not create a mess
in an attempt to do it quickly, but it should remain a long-term goal.
The flexibility that we gain in doing so will pay us back later.

In particular, I think it is extremely inappropriate for GDB itself to
require threads. That would, in principle, be about as bad as allowing
parts of GCC to require a working C++ compiler...

-- 
Todd Whitesel
toddpw @ windriver.com

Re: non-blocking reads/writes and event loops

[Jim Ingham]

on 6/12/00 6:47 PM, Todd Whitesel at toddpw@windriver.com wrote:

>> The thing that needs to be decided is how far GDB should be pushed to
>> address this problem.  Should GDB continue to be pushed to the point
>> where everything is event based or should, the current compromise remain
>> where a GUI is unable to exploit GDBs event-loop.
> 
> I am a advocate of long-time eventification. We should not create a mess
> in an attempt to do it quickly, but it should remain a long-term goal.
> The flexibility that we gain in doing so will pay us back later.

I heartily agree!

> 
> In particular, I think it is extremely inappropriate for GDB itself to
> require threads. That would, in principle, be about as bad as allowing
> parts of GCC to require a working C++ compiler...

That was also the conclusion we came to when Andrew, Elena et al were coming
up with the *-async targets.  I don't think that we should solve the problem
of making gdb non-blocking by putting everything in separate threads.
However, threads may need to be used in the case where you only have a
blocking API to control the inferior, but in that case all you are doing is
using threads to fix this local deficiency, not pushing it into the gdb
architecture.  I think this is managable.

Jim

Re: non-blocking reads/writes and event loops

[Todd Whitesel]

> However, threads may need to be used in the case where you only have a
> blocking API to control the inferior, but in that case all you are doing is
> using threads to fix this local deficiency, not pushing it into the gdb
> architecture.  I think this is managable.

Agreed. If the API designers are dumb enough to trust threads absolutely,
and don't give us an option, then well, that's life.

-- 
Todd Whitesel
toddpw @ windriver.com

Re: non-blocking reads/writes and event loops

[Andrew Cagney]

Todd Whitesel wrote:
> 
> > However, threads may need to be used in the case where you only have a
> > blocking API to control the inferior, but in that case all you are doing is
> > using threads to fix this local deficiency, not pushing it into the gdb
> > architecture.  I think this is managable.
> 
> Agreed. If the API designers are dumb enough to trust threads absolutely,
> and don't give us an option, then well, that's life.

For what it is worth, here is the ``It is hard'' reason number two:

In addition to the expression evaluator, every physical target (that is
the bit that knows how to read/write memory and registers) needs to be
inverted.  The basic programming model used when developing them
requiring a full rewrite.

At present a memory/register IO looks like (vaguely):

	event-loop
	   tty event: (gdb) print ...
		-> expression-evaluator
		    -> target->read()
		        -> os/remote->read()
		        <- data
		    <- data
		<- evaluate
	    print

at the target level, since we are inverting things, it would need to be
broken down into several transactions:

	event-loop
	    tty-event: (gdb) print ...
		-> expression-evaluator
		    -> target->request_read ()
		        -> os/remote->request_read ()

	event-loop
	    os/remote-event: some data ready
		-> os/remote->supply_some_of_data()

	event-loop
	    os/remote-event: some data ready
		-> os/remote->supply_rest_of_data() (all data ready)
		    -> target->supply_data()
			->expression_evaluator->supply_data()

	event-loop
	     ->expression-evaluator->display()

(please don't nit-pick on the detail :-).  The point being that you are
introducing a fundamentally different programming model into _all_
levels of GDB.  You are no longer using procedures but instead passing
around messages and writing state machines or similar.  All of which are
not mechanical.

Just making certain that your eyes are open :-)

	Andrew

Re: non-blocking reads/writes and event loops

[Todd Whitesel]

> > Agreed. If the API designers are dumb enough to trust threads absolutely,
> > and don't give us an option, then well, that's life.
> 
> For what it is worth, here is the ``It is hard'' reason number two:
...
> Just making certain that your eyes are open :-)

No doubts about that. That's why I said "long term goal"...

-- 
Todd Whitesel
toddpw @ windriver.com

Re: non-blocking reads/writes and event loops

[Andrew Cagney]

Todd Whitesel wrote:
> 
> > > Agreed. If the API designers are dumb enough to trust threads absolutely,
> > > and don't give us an option, then well, that's life.
> >
> > For what it is worth, here is the ``It is hard'' reason number two:
> ...
> > Just making certain that your eyes are open :-)
> 
> No doubts about that. That's why I said "long term goal"...

Unfortunatly, unless feasible intermediate steps are identifed that show
how we can get from the eixsting GDB to this ``long term goal'' it will
just remain a long term goal :-(

To that end, I can see several strategies.  Interestingly at least one
discards the assumption that the event loop should _not_ be re-entrant.

o	invert the targets first
		
	This would be implemented
	using something like:

	(gdb) ->do-command
	  -> target_xfer_memory()
	    -> target->request_data()
	    while need more data, run inner event loop
	       <- target-supply-data()
	    <- return data

	That is, the GDB core would continue
	to assume that things are blocking
	but the targets would internally
	be implemented as non-blocking.

	Care would be needed that the internal
	event loop didn't re-call the CLI et.al.


o	invert the expression evaluator first

	In this case, legacy targets would
	be wrapped so that:

	(gdb) ->do-command
	   -> target_memory_request()
	     -> target_xfer_memory ()
	     schedule event to supply
	     returned data to expression evaluator
	-> supply_data to expression evaluator


(wonder if this makes any sense).  Of course there is option three, both
of the above.

For what its worth, for some reason I have preference for the first
option - not sure why, perhaphs it is simply that I'm more familar with
targets and back-ends.

As I noted above, one of the original design decisions for the event
loop that it not be re-entrant.  The above questions that decision. 
Another decision was that GDB's core assume no threads.  Should that too
be questioned?

enjoy,
	Andrew

Re: non-blocking reads/writes and event loops

[Daniel Berlin]

> Todd Whitesel wrote:
> > 
> > > > Agreed. If the API designers are dumb enough to trust threads absolutely,
> > > > and don't give us an option, then well, that's life.
> > >
> > > For what it is worth, here is the ``It is hard'' reason number two:
> > ...
> > > Just making certain that your eyes are open :-)
> > 
> > No doubts about that. That's why I said "long term goal"...
> 
> Unfortunatly, unless feasible intermediate steps are identifed that show
> how we can get from the eixsting GDB to this ``long term goal'' it will
> just remain a long term goal :-(

Almost as if planning something large was an integral part of getting it
done.
Strange.

> 
> To that end, I can see several strategies.  Interestingly at least one
> discards the assumption that the event loop should _not_ be re-entrant.
> 
> o	invert the targets first
> 		
> 	This would be implemented
> 	using something like:
> 
> 	(gdb) ->do-command
> 	  -> target_xfer_memory()
> 	    -> target->request_data()
> 	    while need more data, run inner event loop
> 	       <- target-supply-data()
> 	    <- return data
> 
> 	That is, the GDB core would continue
> 	to assume that things are blocking
> 	but the targets would internally
> 	be implemented as non-blocking.
> 
> 	Care would be needed that the internal
> 	event loop didn't re-call the CLI et.al.
> 
> 
> o	invert the expression evaluator first
> 
> 	In this case, legacy targets would
> 	be wrapped so that:
> 
> 	(gdb) ->do-command
> 	   -> target_memory_request()
> 	     -> target_xfer_memory ()
> 	     schedule event to supply
> 	     returned data to expression evaluator
> 	-> supply_data to expression evaluator
> 
> 
> (wonder if this makes any sense).  Of course there is option three, both
> of the above.
> 
> For what its worth, for some reason I have preference for the first
> option - not sure why, perhaphs it is simply that I'm more familar with
> targets and back-ends.
> 

> As I noted above, one of the original design decisions for the event
> loop that it not be re-entrant.  The above questions that decision. 
And rightfully questions it, IMHO.

> Another decision was that GDB's core assume no threads.  Should that too
> be questioned?

I have no problem with using threads, i do it on a daily basis. The only
issue i would have with GDB's core using threads would be that we take
care not to assume that everyone has pthreads. If we go with threads, i
would ask we add a simple abstraction, like most portable things using
threads (Just from personal experience, Python's source comes to mind as 
something that works with threads, using "easy to make work on a given
platform supporting threads", and provides all the thread functionality
people ask for. Works on BeOS, QNX, and every other python 
supported platform that has threads).

The question about whether we *should* use threads is a different one
altogether.
The real question splits into "Are there parts of gdb where we could be
doing
processing that isn't dependent on other processing, and we therefore are
possibly wasting time on MP systems by not having each done in a thread"
and "Are there parts of GDB where we could simplify code flow by using
threads".

 >
> enjoy, > Andrew > 

Re: non-blocking reads/writes and event loops

[Todd Whitesel]

> For what its worth, for some reason I have preference for the first
> option - not sure why, perhaphs it is simply that I'm more familar with
> targets and back-ends.

IMHO the simplest way is always to invert from the top down, so things are
just merged into the main event loop as you go.

> As I noted above, one of the original design decisions for the event
> loop that it not be re-entrant.  The above questions that decision. 

Well that depends on how much of the event loop machinery is used by the
low-level inversion. If it calls things that assume a single global event
loop, then we have a problem.

If you have to invert a low-level algorithm early on, there are a couple
ways to do it:

  1. make the event loop machinery instantiable and use a new instance of it.
	THIS IS ARGUABLY NECESSARY FOR MULTIPLE TARGET CONNECTIONS ANYWAY.

  2. do not attempt to provide full non-blocking facilities yet, just show
	that the low-level code works correctly in inverted form -- its
	sub-event loop just calls it repeatedly, so it still blocks but at
	least uses the new code structure.

IMHO either is fine; which one you use depends on whether you have gotten
the instantiable event loop machinery working yet.

> Another decision was that GDB's core assume no threads.  Should that too
> be questioned?

I don't mind specific native target support assuming threads.

However using threads as a general method of avoiding the inversion of
GDB core code is a COP OUT.

-- 
Todd Whitesel
toddpw @ windriver.com

Re: non-blocking reads/writes and event loops

[Elena Zannoni]

Todd Whitesel writes:
 > > For what its worth, for some reason I have preference for the first
 > > option - not sure why, perhaphs it is simply that I'm more familar with
 > > targets and back-ends.
 > 
 > IMHO the simplest way is always to invert from the top down, so things are
 > just merged into the main event loop as you go.
 > 
 > > As I noted above, one of the original design decisions for the event
 > > loop that it not be re-entrant.  The above questions that decision. 
 > 
 > Well that depends on how much of the event loop machinery is used by the
 > low-level inversion. If it calls things that assume a single global event
 > loop, then we have a problem.

Yes, that wasn't one of the things I worried about when I wrote the
event loop.

 > 
 > If you have to invert a low-level algorithm early on, there are a couple
 > ways to do it:
 > 
 >   1. make the event loop machinery instantiable and use a new instance of it.
 > 	THIS IS ARGUABLY NECESSARY FOR MULTIPLE TARGET CONNECTIONS ANYWAY.
 > 
 >   2. do not attempt to provide full non-blocking facilities yet, just show
 > 	that the low-level code works correctly in inverted form -- its
 > 	sub-event loop just calls it repeatedly, so it still blocks but at
 > 	least uses the new code structure.
 > 

Yes, we must do things in smaller steps. Otherwise the task is so
daunting, it will never get done. 

There were actually a few things that we were considering as 'next
steps' in the event-loopization process. One is to hook up the async
version of the remote target to the Insight gui. Another is to make a
native target asynchronous. The choice here would be Linux, I
think. Also there are interface/CLI issues (style decisions) still to
be resolved with the asyncrhonous remote target.

As Andrew pointed out, the remote target is the hairy case. We
encountered several problems when writing the asynchronous version. We
got as far as we could, w/o having to revise major assumptions, and
found that we got maybe 70/80 % of the stuff working fine. The
remaining bits were the hard ones (the ones where GDB gets stuck
and you don't know what happened), for which we really need to do
major surgery to remote.c and the layers below that.

Has anybody tried to use 'target async' and 'target extended-async' in
place of 'target remote' and 'target extended-remote'? I have heard no
feedback on them at all. 

Elena

 > IMHO either is fine; which one you use depends on whether you have gotten
 > the instantiable event loop machinery working yet.
 > 
 > > Another decision was that GDB's core assume no threads.  Should that too
 > > be questioned?
 > 
 > I don't mind specific native target support assuming threads.
 > 
 > However using threads as a general method of avoiding the inversion of
 > GDB core code is a COP OUT.
 > 
 > -- 
 > Todd Whitesel
 > toddpw @ windriver.com

Re: non-blocking reads/writes and event loops

[Jim Blandy]

> The question about whether we *should* use threads is a different
> one altogether.  The real question splits into "Are there parts of
> gdb where we could be doing processing that isn't dependent on other
> processing, and we therefore are possibly wasting time on MP systems
> by not having each done in a thread" and "Are there parts of GDB
> where we could simplify code flow by using threads".

Srikanth at HP has mentioned the idea of doing symbol table reads in
the background.  I thought that sounded pretty cool.