[RFC] plugin/extension interface

[RFC] plugin/extension interface

[Andrew STUBBS]

Hi all,

I would like to test the waters and find out what your opinion would be 
on the subject of a GDB plugin/extension interface.

We, here at ST, have a custom target backend which is implemented as a 
DLL connected to some custom code in GDB not totally unlike remote.c. We 
did this because the remote protocol was too slow, too limited and 
required some way to launch other processes which could then easily get 
left over by mistake.

Now we have reason to rewrite the interface - it is currently very 
SH-centric and we wish to share it with another ST architecture - so I 
am considering the various possibilities.

One possibility (of which I am in favour) is to create a generic GDB 
plugin interface which can become part of the official GDB. This course 
of action is not yet certain, but the outcome of any discussion we have 
here will probably influence the final decision (along with some other 
local considerations).

Desirable features (from our point of view):
- Target independent [1].
- Architecture independent [1].
- Flexible/Powerful enough not to limit capabilities.
- Debugger independent [2].
- Extensible.
- Easy to maintain.

It is largely this last requirement which prompts me to ask your opinion 
- it will be easier to maintain if it is accepted into the official 
source base, because we will not have to continually forward port it as 
a local change. Obviously I am more likely to get something accepted if 
you guys have agreed the interface in principle in advance.

It is also my goal that one day we will be able to use a GDB with no 
local patches, or at least only those relating to bugs that will be 
fixed in the next version. Obviously my recent contributions will go a 
long way toward this and getting the backend sorted would get me much 
closer still.

[1] The target/architecture dependent parts should exist entirely in the 
DLL while the plugin interface remains neutral.

[2] The DLL should also work with a simple loader front-end and, 
potentially, other debuggers, should the need arise (obviously a similar 
interface would have to be implemented in that debugger and the DLL 
would have to have understand debugger 'personalities'). Basically this 
means not just exporting the target_ops table. This is another reason 
why the remote protocol is undesirable in this case.



I shall now outline one possible implementation of what I envisage. This 
is sufficiently similar to what ST has now not to cause me any undue 
work. It is also sufficiently generic that it should work for anybody 
else with similar requirements.

User Interface Outline
----------------------

CLI commands:
  plugin load PLUGIN ARGS
  plugin unload PLUGIN
  plugin list                  (or info plugins ???)

Any additional parameters passed to the load command will be passed 
through during initialisation.

Upon load the DLL will register zero or more target interfaces and/or 
zero or more CLI commands. This includes set/show commands and nested 
commands - the appropriate built-in ones (info etc.) as well as groups 
it registers itself.

If two plugins register the same target then that is an error.

If two plugins register the same CLI command (say they both use the same 
set/show options) then each plugin will be notified when the command is 
issued. If the plugin requires exclusive use of a command then it must 
make its name unique in some way.

Backend API Outline
-------------------

The majority of the GDB specific parts of a (process stratum) target 
will be coded in GDB, much like the remote target. The DLL interface 
would then consist of low or medium level target manipulation calls. The 
DLL itself will provide all the knowledge of how to connect (given the 
parameters passed to the target command), but ought not need to know 
about the object files and such unless it wants to.

The DLL must have symbols of the right name defined so that they can be 
properly hooked up using dlsym() (or GetProcAddress() on Windows). These 
will consist of functions for GDB to call and function pointers into 
which GDB may poke values for any reverse function calls. Some symbols 
will be compulsory and the load will fail if they do not exist and 
others will be optional, much as in the target_ops table currently.

Compulsory functions (not complete, not final names):
   initialize
   deinitialize

Compulsory when registering a target:
   attach
   detach
   read_mem
   write_mem
   read_reg
   write_reg
   resume
   step
   interrupt
   ...whatever other target_ops seems appropriate

Optional functions (not complete, not final names):
   load                } can be implemented with read/write
   compare_sections    } but DLL may want to for efficiency
   set_breakpoint      } and/or target specific correctness
   remove_breakpoint   }
   set_hardware_breakpoint
   remove_hardware_breakpoint
   set_watchpoint
   remove_hardware_watchpoint
   disconnect
   ...thread stuff
   ...whatever other target_ops seems appropriate

Optional function pointers (not complete, not final names):
   register_target
   register_gdb_command
   error
   print_filtered
   print_unfiltered
   call_gdb_cli
   ...access to other GDB internals as required (safely wrapped up)

Obviously I haven't attempted to finalize the initial function set. The 
above gives a general picture of what I intend. I have an existing 
interface which approximately defines the minimum set, but after some 
parts have been migrated from GDB to the DLL (and perhaps vice-versa), 
and all the deprecated methods have been reimplemented, then I shall 
have a better understanding of exactly what is needed.

The extensibility requirement can be met simply by changing the names of 
functions that change. e.g., "dlsym(init2); if (!init2) dlsym (init);". 
If the interface is not powerful/flexible enough for a plugin then extra 
calls may be added as required. That particular plugin will only work 
(fully) on that version of GDB (or newer), and all other plugins will be 
unaffected (they should not even need recompilation).

Of course, the sky (and the complexity of the code) is the limit. It 
would, for example, be possible to add enough hooks to implement Insight 
as a plugin (perhaps not exactly as it is now), although I'm not 
recommending anybody tries!

The debugger independence is provided by not using GDB specific 
names/arguments and by implementing as much of the GDB specific target 
interface as possible on the GDB side. The DLL will need to be aware of 
what it is connected to if it registers or calls CLI commands, but 
beyond that it should not have to worry too much. It may be that some 
optional features are not supported by all hosts, but they are optional.



If you have read this far then thanks!

I shall be away from next Thursday so I don't expect to get this decided 
this year and certainly not start any work until the new year, but all 
comments, ideas and general gut feelings would be appreciated.

I would prefer to reach an agreement in principle before attempting to 
thrash out too many details.

TIA

Andrew Stubbs

Re: [RFC] plugin/extension interface

[Eli Zaretskii]

> Date: Fri, 02 Dec 2005 18:22:54 +0000
> From: Andrew STUBBS <andrew.stubbs@st.com>
> 
> I would like to test the waters and find out what your opinion would be 
> on the subject of a GDB plugin/extension interface.

Such suggestions (not only for GDB, for other GNU projects as well)
are usually rejected by Richard Stallman and the FSF, because they
make it possible to add to GDB significant new features with
proprietary (i.e. non-free software) shared libraries, thus avoiding
the need to release any parts of GDB under GPL.

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Fri, Dec 02, 2005 at 08:48:58PM +0200, Eli Zaretskii wrote:
> > Date: Fri, 02 Dec 2005 18:22:54 +0000
> > From: Andrew STUBBS <andrew.stubbs@st.com>
> > 
> > I would like to test the waters and find out what your opinion would be 
> > on the subject of a GDB plugin/extension interface.
> 
> Such suggestions (not only for GDB, for other GNU projects as well)
> are usually rejected by Richard Stallman and the FSF, because they
> make it possible to add to GDB significant new features with
> proprietary (i.e. non-free software) shared libraries, thus avoiding
> the need to release any parts of GDB under GPL.

I think we could discuss this with the FSF, if we had sufficiently
compelling reasons.  But, I don't think that we do, right now.

Andrew, I can't support an interface like the one you've described.
GDB is a hugely complicated program, with many independent areas; it's
not clear to me which of those areas you're trying to make pluggable. 
It looks to me like it is specifically the target interface - just a
very small piece of the pie.  Also, new CLI commands.

If that's right, the traditional solution to the target interface is a
third party program which acts as a conversion tool between the GDB
remote protocol and your proprietary target protocol, with whatever
glue logic you would otherwise put into the plugin.  The remote
protocol serves as the boundary interface in the same way that the DLL
boundary would serve as an interface.

If what you want can't be done that way because of shortcomings in
the protocol, we can fix them.  This keeps the already-existing,
documented, and supported remote protocol as the extent of GDB's
obligation to remote targets.  All the other bits you wanted to export
as methods have less clear semantics, and some of them are guaranteed
to change over time.  GDB is burdened already with support for a huge
number of targets using the _one_ interface; I don't want to deal with
backwards compatibility.

For new CLI commands, the right answer is probably to implement them
in-core in some scripting language, and optionally pass things back to
your remote target via "monitor" commands.  The existing scripting
language is inadequate.  You can find preliminary (but usable) perl
bindings in the mailing list archive.  I've hacked Guile into GDB in
the past, and when I can find some time, I intend to revisit the
general problem (probably using Python, this time, since I'm most
familiar with it at the moment).  This requires some MI interface
surgery and is not a small task.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Mark Kettenis]

> Date: Fri, 02 Dec 2005 18:22:54 +0000
> From: Andrew STUBBS <andrew.stubbs@st.com>
> 
> Hi all,
> 
> I would like to test the waters and find out what your opinion would be 
> on the subject of a GDB plugin/extension interface.

My initial reaction (without reading the rest of your mail) would be
"Not over my dead body!".  In my experience plugin/extension
interfaces add a lot of bloat, are difficult to maintain, and
difficult to support.  Look at the problems we've had with the
libthread-db.so stuff on Linux and (to a lesser extent) Solaris.
There are also GPL issues.

> We, here at ST, have a custom target backend which is implemented as a 
> DLL connected to some custom code in GDB not totally unlike remote.c. We 
> did this because the remote protocol was too slow, too limited and 
> required some way to launch other processes which could then easily get 
> left over by mistake.
> 
> Now we have reason to rewrite the interface - it is currently very 
> SH-centric and we wish to share it with another ST architecture - so I 
> am considering the various possibilities.
> 
> One possibility (of which I am in favour) is to create a generic GDB 
> plugin interface which can become part of the official GDB. This course 
> of action is not yet certain, but the outcome of any discussion we have 
> here will probably influence the final decision (along with some other 
> local considerations).
> 
> Desirable features (from our point of view):
> - Target independent [1].
> - Architecture independent [1].
> - Flexible/Powerful enough not to limit capabilities.
> - Debugger independent [2].
> - Extensible.
> - Easy to maintain.
> 
> It is largely this last requirement which prompts me to ask your opinion 
> - it will be easier to maintain if it is accepted into the official 
> source base, because we will not have to continually forward port it as 
> a local change. Obviously I am more likely to get something accepted if 
> you guys have agreed the interface in principle in advance.

Why can't you just compile your remote protocol support code into GDB.
That's the way it's been always done.  It's simple, and works, and I
think that'll actually be the easiest to maintain.  Plugin/extension
interfaces only make sense for third parties that want to distribute
binary stuff.  That doesn't make sense for an open source debugger.

Mark

Re: [RFC] plugin/extension interface

[Jim Blandy]

I'm not so hostile to plug-in interfaces.  It's true that the API
needs to be designed carefully, but allowing people to maintain other
features (targets; architectures; commands) separately from GDB would
take a big load off our backs.

It's true that plugin interfaces weaken the incentives the GPL tries
to create, but while I think that was very important even five years
ago, I don't think that's such a big deal any more.  Maybe I don't get
out enough, but I think there's already a consensus building that Open
Source is simply the preferable way to develop programmers' tools, and
most companies are just doing it freely, not because they have to use
GPL'd ode.

libthread_db has been a debacle, but I'd argue that's because it was
designed for Solaris, and we lacked the option of adjusting the
interface to better suit other systems.  For example, the API doesn't
abstract the process of stopping or continuing threads, meaning that
it doesn't have the information it needs to cache things accurately in
some cases (leading to bad performance) and that GDB pays the price in
complexity of supporting the plugin interface, but ends up knowing a
lot about the thread implementation anyway, so it doesn't get much
benefit.

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Fri, Dec 02, 2005 at 02:12:46PM -0800, Jim Blandy wrote:
> I'm not so hostile to plug-in interfaces.  It's true that the API
> needs to be designed carefully, but allowing people to maintain other
> features (targets; architectures; commands) separately from GDB would
> take a big load off our backs.

I don't think the benefit for targets or architectures is a
particularly large one.  Folks should expect to have to deal with core
GDB for these, or use the remote protocol boundary.  I do think we will
want plugins eventually, but I haven't thought much about whether real
dlopen'd plugins are worthwhile, or e.g. python.

One classic example of this: domain-specific "shared library" objects
for custom module loaders, like the Linux kernel's or X11's.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Mark Kettenis]

> Date: Fri, 2 Dec 2005 14:12:46 -0800
> From: Jim Blandy <jimb@red-bean.com>

> I'm not so hostile to plug-in interfaces.  It's true that the API
> needs to be designed carefully, but allowing people to maintain other
> features (targets; architectures; commands) separately from GDB would
> take a big load off our backs.

I really doubt that.  Short term it'd only increase the load since we
have to build the plug-in interface.  Long term, we still have to
maintain that interface.  We'll also get lots of support questions
where the problems are really with the third party plugins, but where
the bug reporter conveniently forgets to mention that he's using a
third party plugin.  Let's keep people out of our address space if
they don't want to commit themselves to providing sources and manpower
to maintain those sources.

> libthread_db has been a debacle, but I'd argue that's because it was
> designed for Solaris, and we lacked the option of adjusting the
> interface to better suit other systems.  For example, the API doesn't
> abstract the process of stopping or continuing threads, meaning that
> it doesn't have the information it needs to cache things accurately in
> some cases (leading to bad performance) and that GDB pays the price in
> complexity of supporting the plugin interface, but ends up knowing a
> lot about the thread implementation anyway, so it doesn't get much
> benefit.

Never really realised it, but actually all that we use libthread_db
for is mapping process IDs into use-level thread IDs.  Suddenly HP's
ttrace(2) interface makes sense to me; they have a field to store the
user-level thread ID in the kernel.  That way you really don't need
any knwoledge about the threads implementation at all, except that
there's a 1:1 mapping between LWPs and user-level threads.

What I meant by the libthread_db debacle is the fact that gdb picks up
the wrong version of it sometimes and that this makes things fail in
nonobvious ways.  Not the fact that trying to support several
different kernels and threads libraries, most of which have serious
bugs, is really erh... difficult.

Mark

Re: [RFC] plugin/extension interface

[Jim Blandy]

On 12/2/05, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> I really doubt that.  Short term it'd only increase the load since we
> have to build the plug-in interface.  Long term, we still have to
> maintain that interface.  We'll also get lots of support questions
> where the problems are really with the third party plugins, but where
> the bug reporter conveniently forgets to mention that he's using a
> third party plugin.  Let's keep people out of our address space if
> they don't want to commit themselves to providing sources and manpower
> to maintain those sources.

I don't think you're seeing the potential here.  There are a *lot* of
things people would like to do with debuggers: it's one of the best
fields for productive creeping featurism you can imagine.  Writ broad,
a debugger is a program for watching your program run.  That's open to
so much application-specific interpretation it's actually hard to
imagine how generic code (like GDB) would even be useful, if you
haven't seen it in action.

In other words, if the plug-ins had enough hooks to customize GDB's
behavior (printing values, or decoding the stack, say), I think you'd
be really surprised what people would do with it.  They'd do tons of
stuff that we'd never want to include in the core distribution, but
which would add enormous value to GDB for their problems.  And of
course, there would be a few that we'd want to incorporate.

To take another tack: plug-ins were vital to Apache's success.  Why
wouldn't they have the same effect on GDB?

The only qualification I'd make here is that GDB's current internal
organization isn't very friendly to this kind of stuff.  A plug-in
interface that could engender the kind of stuff I'm describing would
have to be pretty clean, high-level, and documented.  I could
certainly see the argument that it's too much work to bring things to
the point where they could bring about the benefits.

... Okay, another qualification is that a well-integrated scripting
language might do everything I'm saying here, and better, because the
extension/core interface is more well-defined.

Re: [RFC] plugin/extension interface

[Bob Rossi]

On Fri, Dec 02, 2005 at 03:07:29PM -0800, Jim Blandy wrote:
> On 12/2/05, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> > I really doubt that.  Short term it'd only increase the load since we
> > have to build the plug-in interface.  Long term, we still have to
> > maintain that interface.  We'll also get lots of support questions
> > where the problems are really with the third party plugins, but where
> > the bug reporter conveniently forgets to mention that he's using a
> > third party plugin.  Let's keep people out of our address space if
> > they don't want to commit themselves to providing sources and manpower
> > to maintain those sources.
> 
> The only qualification I'd make here is that GDB's current internal
> organization isn't very friendly to this kind of stuff.  A plug-in
> interface that could engender the kind of stuff I'm describing would
> have to be pretty clean, high-level, and documented.  I could
> certainly see the argument that it's too much work to bring things to
> the point where they could bring about the benefits.

I'm already in the long process of creating a level on top of GDB using
MI that allows another process to do anything it wants with GDB. IMO,
that is already the clean, high-level, documented interface that other
applications should use to communicate with GDB.

If the interface needs to change, that's where the effort should go. I
don't think providing any other plugin interface besides what's
described above even makes sense since it would be duplicating the
effort.

Bob Rossi

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Fri, Dec 02, 2005 at 03:07:29PM -0800, Jim Blandy wrote:
> I don't think you're seeing the potential here.  There are a *lot* of
> things people would like to do with debuggers:

And every one of these things you've described doing with debuggers,
would require a _DIFFERENT_ plugin interface.  It would be a nightmare
to add this to today's GDB!

See Bob's comment for the right way to handle this problem, in my
opinion also.  I consider all this part of the "front end".  Most of
these things shouldn't be added to GDB at such a level that it gets
exposed through MI; you should add it to your front end in a front end
specific fashion.

Often this means you need to add it to the GDB CLI also.  That's a
different sort of problem.  Scripting languages, rather than plugin
interfaces, are more viable for this.

All this, of course, is moot until someone begins to implement it.
But I have yet to see a binary plugin interface that doesn't have all
the problems Mark described.

> ... Okay, another qualification is that a well-integrated scripting
> language might do everything I'm saying here, and better, because the
> extension/core interface is more well-defined.

Yes.

I believe this will be the thing that pushes us up to version 7.  I
hope it will be soon, time and volunteers permitting.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Jim Blandy]

On 12/2/05, Daniel Jacobowitz <drow@false.org> wrote:
> And every one of these things you've described doing with debuggers,
> would require a _DIFFERENT_ plugin interface.  It would be a nightmare
> to add this to today's GDB!

Oh, I figured we'd just let the plugin code #include GDB's headers
directly, so they could get at whatever they wanted.

... okay, that does sound like a nightmare.

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Fri, Dec 02, 2005 at 04:57:01PM -0800, Jim Blandy wrote:
> On 12/2/05, Daniel Jacobowitz <drow@false.org> wrote:
> > And every one of these things you've described doing with debuggers,
> > would require a _DIFFERENT_ plugin interface.  It would be a nightmare
> > to add this to today's GDB!
> 
> Oh, I figured we'd just let the plugin code #include GDB's headers
> directly, so they could get at whatever they wanted.
> 
> ... okay, that does sound like a nightmare.

Now we're communicating :-)

For the record, here's my overall point in this thread.  We have some
very good abstraction layers already: in particular, I'm talking about
the remote protocol, and the MI/interpreters mechanism.  I want GDB to
be more extensible, but I believe that those protocols are the best way
to do it.  They both have limitations for this kind of use, because
they aren't used that way yet; but what we need to do is commit to
using them, then bite the bullet and begin improving them to meet our
needs.

That way we can keep the interfaces coherent, and hopefully,
documented at or above today's levels.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Russell Shaw]

Daniel Jacobowitz wrote:
> On Fri, Dec 02, 2005 at 04:57:01PM -0800, Jim Blandy wrote:
> 
>>On 12/2/05, Daniel Jacobowitz <drow@false.org> wrote:
>>
>>>And every one of these things you've described doing with debuggers,
>>>would require a _DIFFERENT_ plugin interface.  It would be a nightmare
>>>to add this to today's GDB!
>>
>>Oh, I figured we'd just let the plugin code #include GDB's headers
>>directly, so they could get at whatever they wanted.
>>
>>... okay, that does sound like a nightmare.
> 
> Now we're communicating :-)
> 
> For the record, here's my overall point in this thread.  We have some
> very good abstraction layers already: in particular, I'm talking about
> the remote protocol, and the MI/interpreters mechanism.  I want GDB to
> be more extensible, but I believe that those protocols are the best way
> to do it.  They both have limitations for this kind of use, because
> they aren't used that way yet; but what we need to do is commit to
> using them, then bite the bullet and begin improving them to meet our
> needs.
> 
> That way we can keep the interfaces coherent, and hopefully,
> documented at or above today's levels.

Months ago, i added a remote protocol for a hardware in-circuit emulator
and was going to write a chapter documenting how to add a target protocol
to gdb. My questions on how best to add this to gdb in cvs got no response
whatsoever. Now i've lost all interest.

Re: [RFC] plugin/extension interface

[Bob Rossi]

On Fri, Dec 02, 2005 at 09:31:54PM -0500, Daniel Jacobowitz wrote:
> On Fri, Dec 02, 2005 at 04:57:01PM -0800, Jim Blandy wrote:
> > On 12/2/05, Daniel Jacobowitz <drow@false.org> wrote:
> > > And every one of these things you've described doing with debuggers,
> > > would require a _DIFFERENT_ plugin interface.  It would be a nightmare
> > > to add this to today's GDB!
> > 
> > Oh, I figured we'd just let the plugin code #include GDB's headers
> > directly, so they could get at whatever they wanted.
> > 
> > ... okay, that does sound like a nightmare.
> 
> Now we're communicating :-)
> 
> For the record, here's my overall point in this thread.  We have some
> very good abstraction layers already: in particular, I'm talking about
> the remote protocol, and the MI/interpreters mechanism.  I want GDB to
> be more extensible, but I believe that those protocols are the best way
> to do it.  They both have limitations for this kind of use, because
> they aren't used that way yet; but what we need to do is commit to
> using them, then bite the bullet and begin improving them to meet our
> needs.
> 
> That way we can keep the interfaces coherent, and hopefully,
> documented at or above today's levels.

I know my role in GDB isn't as big as most others in this thread, but
for what it's worth, I fully agree with this position.

Bob Rossi

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Sat, Dec 03, 2005 at 01:41:35PM +1100, Russell Shaw wrote:
> Months ago, i added a remote protocol for a hardware in-circuit emulator
> and was going to write a chapter documenting how to add a target protocol
> to gdb. My questions on how best to add this to gdb in cvs got no response
> whatsoever. Now i've lost all interest.

[Off-topic here, but]

I can't even remember seeing messages from you on this topic; a certain
amount of persistance is always called for in GDB development.

However, in general, I've come to believe that GDB shouldn't continue
to add support for new remote protocols.  What we need is a nice,
open-source framework for gdb remote <-> other remote conversion.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Russell Shaw]

Daniel Jacobowitz wrote:
> On Sat, Dec 03, 2005 at 01:41:35PM +1100, Russell Shaw wrote:
> 
>>Months ago, i added a remote protocol for a hardware in-circuit emulator
>>and was going to write a chapter documenting how to add a target protocol
>>to gdb. My questions on how best to add this to gdb in cvs got no response
>>whatsoever. Now i've lost all interest.
> 
> [Off-topic here, but]
> 
> I can't even remember seeing messages from you on this topic; a certain
> amount of persistance is always called for in GDB development.
> 
> However, in general, I've come to believe that GDB shouldn't continue
> to add support for new remote protocols.  What we need is a nice,
> open-source framework for gdb remote <-> other remote conversion.

The way gdb works, when i type: target avrjtagice -d /dev/ttyS2,
gdb uses (or registers) using: void _initialize_remote_jtag (void).

It really looks like gdb is "loading" the module.

If the functions (or framework) used in these target interfaces are
documented, the problems would be solved with little other than a bit
of documentation work.

I couldn't use the default target protocol, because i wasn't interfacing
to an end target. I was interfacing to an ICE debugger (that already had
its own protocol) for the target.

I did it to replace the current hack of having an extra program on the
pc that converts between the default gdb target protocol, and this ICE-
specific protocol.

The conversion hack was extremely slow, clumsy, fault-intolerant, and error-prone.

With the specific interface in gdb, the performance was wonderful, and new gdb
commands can be easily registered for this ICE, that are only accessible after
the command "target avrjtagice -d /dev/ttyS2" is performed (like an emacs minor
mode or whatever).

Currently, the files for these specific interfaces are just chucked into gdb-6.3/gdb/
(such as remote-e7000.c).

What's really needed is a separate sub-directory for each of these remote-interface
protocol files, and a well documented api for what functions to use within them.
The api already exists, just look in any db-6.3/gdb/remote*.c. One of the subdirectories
can be devoted to the generic target protocol where the comms is to the end embedded
system, instead of an intermediate debugger box.

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Sat, Dec 03, 2005 at 02:13:11PM +1100, Russell Shaw wrote:
> The way gdb works, when i type: target avrjtagice -d /dev/ttyS2,
> gdb uses (or registers) using: void _initialize_remote_jtag (void).

Incorrect; that happens at GDB startup.

> If the functions (or framework) used in these target interfaces are
> documented, the problems would be solved with little other than a bit
> of documentation work.

Not unless we declared them a public interface.  That means supporting
them, preserving compatibility, et cetera.  We are not prepared to do
that.

> I couldn't use the default target protocol, because i wasn't interfacing
> to an end target. I was interfacing to an ICE debugger (that already had
> its own protocol) for the target.
> 
> I did it to replace the current hack of having an extra program on the
> pc that converts between the default gdb target protocol, and this ICE-
> specific protocol.
> 
> The conversion hack was extremely slow, clumsy, fault-intolerant, and 
> error-prone.

I would need to know a lot more about these problems, but it is very
likely that they are problems with the conversion program - not with
the concept.  If they were problems with the remote protocol, we would
be interested in fixing them.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Russell Shaw]

Daniel Jacobowitz wrote:
> On Sat, Dec 03, 2005 at 02:13:11PM +1100, Russell Shaw wrote:
> 
>>The way gdb works, when i type: target avrjtagice -d /dev/ttyS2,
>>gdb uses (or registers) using: void _initialize_remote_jtag (void).
> 
> Incorrect; that happens at GDB startup.
> 
>>If the functions (or framework) used in these target interfaces are
>>documented, the problems would be solved with little other than a bit
>>of documentation work.
> 
> Not unless we declared them a public interface.  That means supporting
> them, preserving compatibility, et cetera.  We are not prepared to do
> that.

Well that's just too bad. It's the best way to go. Atleast i can support
my own private gdb and cut the cruft that's been holding it back for years.

>>I couldn't use the default target protocol, because i wasn't interfacing
>>to an end target. I was interfacing to an ICE debugger (that already had
>>its own protocol) for the target.
>>
>>I did it to replace the current hack of having an extra program on the
>>pc that converts between the default gdb target protocol, and this ICE-
>>specific protocol.
>>
>>The conversion hack was extremely slow, clumsy, fault-intolerant, and 
>>error-prone.
> 
> I would need to know a lot more about these problems, but it is very
> likely that they are problems with the conversion program - not with
> the concept.  If they were problems with the remote protocol, we would
> be interested in fixing them.

The very concept of making a protocol conversion program act like an
end target is totally flawed.

First there's the tediousness of starting and initializing the converter
program and comms from it to the debugger box. Then the gdb<->debugger
comms must be established. Then the debugger<->embedded-system comms and
setup needs to be established using gdb.

Any hang-ups in the converter program or debugger means redoing the whole process.

The whole chain is extra slow because round-trip comms thru
gdb<->converter<->debugger<->target-system
is limited by the context switching of the pc between the gdb and converter processes.

There is also the extra tediousness of debugging the converter program and
trying to feed it with comms data using gdb, at the same time as having yet
another gdb to debug the actual converter.

The second problem is that ICE hardware and other debuggers have various specific
commands such as for setting ICE hardware parameters, selecting memory spaces,
setting breakpoint paramteters, and a ton of other stuff that a generic protocol
has no hope of addressing.

I found that by registering my own commands with add_com(), i can do all kinds
of excellent things such as spit out on the gdb output window a tabulated list
of the fuse settings in a microcontroller, or display and set the current ICE
hardware settings.

With a defined interface for vendors to control specific debugger hardware, gdb
would get a lot more interest and move lightyears ahead. All i've seen is stagnation
ever since i've had an interest in gdb 5 years ago.

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Sat, Dec 03, 2005 at 03:05:25PM +1100, Russell Shaw wrote:
> Well that's just too bad. It's the best way to go. Atleast i can support
> my own private gdb and cut the cruft that's been holding it back for years.

You can do that anyway.  Maintain your own source tree based on a
current release; it's not that hard, lots of people do this already.

> The very concept of making a protocol conversion program act like an
> end target is totally flawed.

Obviously I disagree; I don't find any of the limitations you describe
to be a serious problem in practice.

And yes, we've written and used several of these shim layers here.

> The second problem is that ICE hardware and other debuggers have
> various specific commands such as for setting ICE hardware
> parameters, selecting memory spaces, setting breakpoint paramteters,
> and a ton of other stuff that a generic protocol has no hope of
> addressing.

Please see the "monitor" command, which lets you pass whatever you wish
to the convertor.  This and a couple of user-defined commands are most
of what you need...

> I found that by registering my own commands with add_com(), i can do
> all kinds of excellent things such as spit out on the gdb output
> window a tabulated list of the fuse settings in a microcontroller, or
> display and set the current ICE hardware settings.

... and they can provide arbitrary output, also.  If it's inadequate
for what you want to do with it, please provide specific examples, and
we can fix it.

> With a defined interface for vendors to control specific debugger
> hardware, gdb would get a lot more interest and move lightyears
> ahead. All i've seen is stagnation ever since i've had an interest in
> gdb 5 years ago.

You might be surprised to know that there's a thriving market for this
sort of conversion layer already, e.g. OCDRemote.

As for the state of GDB, that's a separate thread, and making progress
already.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Russell Shaw]

Daniel Jacobowitz wrote:
> On Sat, Dec 03, 2005 at 03:05:25PM +1100, Russell Shaw wrote:
> 
>>Well that's just too bad. It's the best way to go. Atleast i can support
>>my own private gdb and cut the cruft that's been holding it back for years.
> 
>
> You can do that anyway.  Maintain your own source tree based on a
> current release; it's not that hard, lots of people do this already.
> 
> 
>>The very concept of making a protocol conversion program act like an
>>end target is totally flawed.
> 
> 
> Obviously I disagree; I don't find any of the limitations you describe
> to be a serious problem in practice.
> 
> And yes, we've written and used several of these shim layers here.
> 
> 
>>The second problem is that ICE hardware and other debuggers have
>>various specific commands such as for setting ICE hardware
>>parameters, selecting memory spaces, setting breakpoint paramteters,
>>and a ton of other stuff that a generic protocol has no hope of
>>addressing.
> 
> 
> Please see the "monitor" command, which lets you pass whatever you wish
> to the convertor.  This and a couple of user-defined commands are most
> of what you need...

Looking at gdb docs, i couldn't find any "monitor" command or in the index.

(gdb) help monitor

   Send a command to the remote monitor (remote targets only).

Not very helpful.

Anyway, preceding every command with "monitor" is hardly a nice or intuitive
user interface.

>>I found that by registering my own commands with add_com(), i can do
>>all kinds of excellent things such as spit out on the gdb output
>>window a tabulated list of the fuse settings in a microcontroller, or
>>display and set the current ICE hardware settings.
> 
> 
> ... and they can provide arbitrary output, also.  If it's inadequate
> for what you want to do with it, please provide specific examples, and
> we can fix it.
> 
> 
>>With a defined interface for vendors to control specific debugger
>>hardware, gdb would get a lot more interest and move lightyears
>>ahead. All i've seen is stagnation ever since i've had an interest in
>>gdb 5 years ago.
> 
> 
> You might be surprised to know that there's a thriving market for this
> sort of conversion layer already, e.g. OCDRemote.

http://www.macraigor.com/full_gnu.htm

All this daemon stuff to convert a generic protocol to a hardware-specific
protocol is slow and ineligant IMO.

> As for the state of GDB, that's a separate thread, and making progress
> already.

If there was an api for hardware-specific protocols, not only would performance
of these improve, but there would be new developers seeing gdb code and having
an interest in maintaining and improving it (i was really the first example).
What's more, all kinds of unsupported debugger hardware could be added by
users. Because each hardware goes into its own subdirectory, it doesn't
complicate any other area of gdb and can be easily removed. That's how
the backend of gcc works.

Changes in gdb that break these interfaces simply mean the maintainers
of them can adapt to the change, so not hindering gdb development.

A problem with the current method of using shims is that if you want
to develop one, you have to go out and find one on the net to find a
place to start.

With all the interfaces in gdb subdirectories, all the example code that
anyone could ever want for adding support for new hardware, is already
there to look at (i looked at gdb-6.3/gdb/remote*.c to start my one).

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Sat, Dec 03, 2005 at 03:44:31PM +1100, Russell Shaw wrote:
> >Please see the "monitor" command, which lets you pass whatever you wish
> >to the convertor.  This and a couple of user-defined commands are most
> >of what you need...
> 
> Looking at gdb docs, i couldn't find any "monitor" command or in the index.
> 
> (gdb) help monitor
> 
>   Send a command to the remote monitor (remote targets only).

Found `monitor' in Index. (`,' tries to find next.)

`monitor CMD'
     This command allows you to send commands directly to the remote
     monitor.

> Not very helpful.

That's because the bits that go after the word "monitor" depend on your
target :-)

> Anyway, preceding every command with "monitor" is hardly a nice or intuitive
> user interface.

Of course.  Provide a GDB script file that uses "define" to give things
pretty names, for example.

> >As for the state of GDB, that's a separate thread, and making progress
> >already.
> 
> If there was an api for hardware-specific protocols, not only would
> performance of these improve, but there would be new developers
> seeing gdb code and having an interest in maintaining and improving
> it (i was really the first example). What's more, all kinds of
> unsupported debugger hardware could be added by users. Because each
> hardware goes into its own subdirectory, it doesn't complicate any
> other area of gdb and can be easily removed. That's how the backend
> of gcc works.

I'm not convinced that performance is an issue here.  I'd need numbers. 
The total data transfer over the remote protocol is, usually, very
small - even if you're doing application downloads, it's just a couple
of extra buffer copies on the local system.  If you expect to be
transfering huge data sessions, use a local pipe or fifo to connect to
the daemon instead of TCP.

> A problem with the current method of using shims is that if you want
> to develop one, you have to go out and find one on the net to find a
> place to start.
> 
> With all the interfaces in gdb subdirectories, all the example code that
> anyone could ever want for adding support for new hardware, is already
> there to look at (i looked at gdb-6.3/gdb/remote*.c to start my one).

If you wanted to add a skeleton protocol conversion daemon to the GDB
source tree, especially under the GPL, I'm sure we'd be glad to include
it!

The problem with "remote*.c" is that most of them haven't been used in
years.  Many of them are bad examples, or simply broken.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Russell Shaw]

Russell Shaw wrote:
> Daniel Jacobowitz wrote:
> 
>> On Sat, Dec 03, 2005 at 03:05:25PM +1100, Russell Shaw wrote:
>>
...
> With all the interfaces in gdb subdirectories, all the example code that
> anyone could ever want for adding support for new hardware, is already
> there to look at (i looked at gdb-6.3/gdb/remote*.c to start my one).

Maybe these subdirectories should actually be for each cpu type,
and any hardware debugger protocols should be in a subdirectory
of that.

This would simplify the source tree of gdb *alot*.

Currently:

   gdb-6.3/gdb/remote-hms.c
   gdb-6.3/gdb/h8300-tdep.c

New:

   gdb-6.3/gdb/h8300/remote/hms.c
   gdb-6.3/gdb/h8300/tdep.c

Re: [RFC] plugin/extension interface

[Daniel Jacobowitz]

On Sat, Dec 03, 2005 at 04:05:59PM +1100, Russell Shaw wrote:
> Russell Shaw wrote:
> >Daniel Jacobowitz wrote:
> >
> >>On Sat, Dec 03, 2005 at 03:05:25PM +1100, Russell Shaw wrote:
> >>
> ...
> >With all the interfaces in gdb subdirectories, all the example code that
> >anyone could ever want for adding support for new hardware, is already
> >there to look at (i looked at gdb-6.3/gdb/remote*.c to start my one).
> 
> Maybe these subdirectories should actually be for each cpu type,
> and any hardware debugger protocols should be in a subdirectory
> of that.
> 
> This would simplify the source tree of gdb *alot*.
> 
> Currently:
> 
>   gdb-6.3/gdb/remote-hms.c
>   gdb-6.3/gdb/h8300-tdep.c
> 
> New:
> 
>   gdb-6.3/gdb/h8300/remote/hms.c
>   gdb-6.3/gdb/h8300/tdep.c

We've talked about doing this before, but some people like it and
others don't, so we've left alone - also, CVS notoriously doesn't like
renaming files.  It doesn't make a whole lot of difference.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Russell Shaw]

Daniel Jacobowitz wrote:
> On Sat, Dec 03, 2005 at 03:44:31PM +1100, Russell Shaw wrote:
> 
>>>Please see the "monitor" command, which lets you pass whatever you wish
>>>to the convertor.  This and a couple of user-defined commands are most
>>>of what you need...
>>
>>Looking at gdb docs, i couldn't find any "monitor" command or in the index.
>>
>>(gdb) help monitor
>>
>>  Send a command to the remote monitor (remote targets only).
> 
> Found `monitor' in Index. (`,' tries to find next.)
> 
> `monitor CMD'
>      This command allows you to send commands directly to the remote
>      monitor.
> 
>>Not very helpful.
> 
> That's because the bits that go after the word "monitor" depend on your
> target :-)
 >
>>Anyway, preceding every command with "monitor" is hardly a nice or intuitive
>>user interface.
> 
> Of course.  Provide a GDB script file that uses "define" to give things
> pretty names, for example.

It would be useful if the manual actually said that this is the way to
add new commands for an external monitor. It is bad when i can only learn
new things about gdb by complaining.

>>>As for the state of GDB, that's a separate thread, and making progress
>>>already.
>>
>>If there was an api for hardware-specific protocols, not only would
>>performance of these improve, but there would be new developers
>>seeing gdb code and having an interest in maintaining and improving
>>it (i was really the first example). What's more, all kinds of
>>unsupported debugger hardware could be added by users. Because each
>>hardware goes into its own subdirectory, it doesn't complicate any
>>other area of gdb and can be easily removed. That's how the backend
>>of gcc works.
> 
> I'm not convinced that performance is an issue here.  I'd need numbers. 
> The total data transfer over the remote protocol is, usually, very
> small - even if you're doing application downloads, it's just a couple
> of extra buffer copies on the local system.  If you expect to be
> transfering huge data sessions, use a local pipe or fifo to connect to
> the daemon instead of TCP.
> 
>>A problem with the current method of using shims is that if you want
>>to develop one, you have to go out and find one on the net to find a
>>place to start.
>>
>>With all the interfaces in gdb subdirectories, all the example code that
>>anyone could ever want for adding support for new hardware, is already
>>there to look at (i looked at gdb-6.3/gdb/remote*.c to start my one).
> 
> If you wanted to add a skeleton protocol conversion daemon to the GDB
> source tree, especially under the GPL, I'm sure we'd be glad to include
> it!
> 
> The problem with "remote*.c" is that most of them haven't been used in
> years.  Many of them are bad examples, or simply broken.

I suppose i could start the shim daemon from a .gdbinit file using the
gdb "shell" command.

An explanation about shims and how to implement them in a way that is
non-tedious to use, should be in the manual.

Last time i read the remote protocol spec in the gdb manual, most of it
was too terse to be useable.

Re: [RFC] plugin/extension interface

[Eli Zaretskii]

> Date: Sat, 03 Dec 2005 15:44:31 +1100
> From: Russell Shaw <rjshaw@netspace.net.au>
> Cc: gdb@sourceware.org
> 
> Looking at gdb docs, i couldn't find any "monitor" command or in the index.

In what version of GDB did you find that manual?  The current CVS
version does have this command listed and indexed.

Re: [RFC] plugin/extension interface

[Russell Shaw]

Eli Zaretskii wrote:
>>Date: Sat, 03 Dec 2005 15:44:31 +1100
>>From: Russell Shaw <rjshaw@netspace.net.au>
>>Cc: gdb@sourceware.org
>>
>>Looking at gdb docs, i couldn't find any "monitor" command or in the index.
> 
> In what version of GDB did you find that manual?  The current CVS
> version does have this command listed and indexed.

Ninth Edition, for gdb version 6.2.5020041014

It was a year ago when i read it.

Re: [RFC] plugin/extension interface

[Eli Zaretskii]

> Date: Sat, 03 Dec 2005 20:50:21 +1100
> From: Russell Shaw <rjshaw@netspace.net.au>
> Cc: gdb@sourceware.org
> 
> > In what version of GDB did you find that manual?  The current CVS
> > version does have this command listed and indexed.
> 
> Ninth Edition, for gdb version 6.2.5020041014
> 
> It was a year ago when i read it.

Probably before I went through the entire manual and documented every
command I found in the sources.

Thanks.

[commit] Clarify "monitor" command (was: [RFC] plugin/extension interface)

[Eli Zaretskii]

> Date: Sat, 03 Dec 2005 16:25:41 +1100
> From: Russell Shaw <rjshaw@netspace.net.au>
> Cc: gdb@sourceware.org
> 
> It would be useful if the manual actually said that this is the way to
> add new commands for an external monitor. It is bad when i can only learn
> new things about gdb by complaining.

Thank you for pointing this out.  I've just committed the following
change to the GDB manual:

2005-12-03  Eli Zaretskii  <eliz@gnu.org>

	* gdb.texinfo (Connecting): Explain that `monitor' is a way to
	extend GDB with commands for external monitor.

Index: gdb/doc/gdb.texinfo
===================================================================
RCS file: /cvs/src/src/gdb/doc/gdb.texinfo,v
retrieving revision 1.298
diff -u -r1.298 gdb.texinfo
--- gdb/doc/gdb.texinfo	2 Dec 2005 11:44:19 -0000	1.298
+++ gdb/doc/gdb.texinfo	3 Dec 2005 12:42:11 -0000
@@ -11966,10 +11966,15 @@
 another target.
 
 @cindex send command to remote monitor
+@cindex extend @value{GDBN} for remote targets
+@cindex add new commands for external monitor
 @kindex monitor
 @item monitor @var{cmd}
-This command allows you to send commands directly to the remote
-monitor.
+This command allows you to send arbitrary commands directly to the
+remote monitor.  Since @value{GDBN} doesn't care about the commands it
+sends like this, this command is the way to extend @value{GDBN}---you
+can add new commands that only the external monitor will understand
+and implement.
 @end table
 
 @node Server

Re: [commit] Clarify "monitor" command

[Russell Shaw]

Eli Zaretskii wrote:
>>Date: Sat, 03 Dec 2005 16:25:41 +1100
>>From: Russell Shaw <rjshaw@netspace.net.au>
>>Cc: gdb@sourceware.org
>>
>>It would be useful if the manual actually said that this is the way to
>>add new commands for an external monitor. It is bad when i can only learn
>>new things about gdb by complaining.
> 
> Thank you for pointing this out.  I've just committed the following
> change to the GDB manual:
> 
> 2005-12-03  Eli Zaretskii  <eliz@gnu.org>
> 
> 	* gdb.texinfo (Connecting): Explain that `monitor' is a way to
> 	extend GDB with commands for external monitor.

Thanks. A decent manual makes all the difference;)

Re: [commit] Clarify "monitor" command (was: [RFC] plugin/extension interface)

[Daniel Jacobowitz]

On Sat, Dec 03, 2005 at 02:48:47PM +0200, Eli Zaretskii wrote:
> > Date: Sat, 03 Dec 2005 16:25:41 +1100
> > From: Russell Shaw <rjshaw@netspace.net.au>
> > Cc: gdb@sourceware.org
> > 
> > It would be useful if the manual actually said that this is the way to
> > add new commands for an external monitor. It is bad when i can only learn
> > new things about gdb by complaining.

You can also learn new things by asking questions :-)

> Thank you for pointing this out.  I've just committed the following
> change to the GDB manual:

Thanks, Eli!  I was going to do this today, but you beat me to it as
usual :-)


-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: [RFC] plugin/extension interface

[Andrew STUBBS]

Thank you for your opinions. I did not expect quite so much controversy. 
Rather than reply to each part of the conversation separately I shall 
attempt to follow up all in one message. I would have responded sooner 
but for the time difference.

 > Such suggestions (not only for GDB, for other GNU projects as well)
 > are usually rejected by Richard Stallman and the FSF, because they
 > make it possible to add to GDB significant new features with
 > proprietary (i.e. non-free software) shared libraries, thus avoiding
 > the need to release any parts of GDB under GPL.

The GPL issues surprise me. I was under the impression that the GPL was 
harder to avoid than this. Is there anything you can point me at (an FSF 
judgement or whatever) that will prove that a dynamic library is not 
forced to use the GPL if it is linked to a GPL program (other than your 
assurances that this is the case)? The GPL itself mentions that system 
libraries are exempt, but I thought all others had to be GPL or 
compatible. This is an issue I would really like to understand.

 > It's true that plugin interfaces weaken the incentives the GPL tries
 > to create, but while I think that was very important even five years
 > ago, I don't think that's such a big deal any more.

I don't really want to comment on the politics, but I think it is worth 
pointing out that the alternative - implementing a GDB server - does not 
force use of the GPL either.

 > GDB is a hugely complicated program, with many independent areas; it's
 > not clear to me which of those areas you're trying to make pluggable.
 > It looks to me like it is specifically the target interface - just a
 > very small piece of the pie.  Also, new CLI commands.

As you say, GDB has many interfaces. However, I cannot see any practical 
use for exporting many of them - GDB already supports most of what the 
rest of the GNU toolchain can throw at it. Perhaps some people would 
like it to support other toolchains, but they probably have their own 
debuggers. The user interface may be extended through MI, and indeed 
many projects do this including DDD and Eclipse, so there's not real 
need to do anything there (although it might be more efficient). In most 
respects GDB has anticipated everything it can anticipate. Of course, 
there are always exceptions.

As I see it the thing that needs to be extendible is the target 
interface. Targets are one of the few areas that GDB cannot (and should 
not attempt to) anticipate.

 > If that's right, the traditional solution to the target interface is a
 > third party program which acts as a conversion tool between the GDB
 > remote protocol and your proprietary target protocol, with whatever
 > glue logic you would otherwise put into the plugin.  The remote
 > protocol serves as the boundary interface in the same way that the DLL
 > boundary would serve as an interface.

The remote protocol is very good at connecting to already-running 
programs, such as kgdb, uboot/redboot/etc, and gdb server. As you say, 
it permits custom features by means of a monitor command, but these are 
uni-directional and only work once connected to a target.

What I propose is more of a 'local' (but not 'exec') target interface. 
It does not require a gdb server running already and it allows the 
monitor commands to be present before connecting to the target, thus 
allowing configuration in .gdbinit and such like.

I suppose one way to look at it is this: an implementation of the remote 
protocol with direct function call instead of serial/ethernet and, 
because there is no need to arbitrarily impose the limitations of the 
remote protocol, a slightly friendlier monitor interface and, perhaps, 
access to a bit more detail.

The problems I have with the remote protocol as it stands are these:
   a) it is inefficient when the target interface software is not 
actually remote;
   b) it is built around the idea of a ROM monitor which is not suitable 
for many target types - many do not exist (conceptually or actually) 
until the debugger causes a process to start somehow;
   c) the monitor commands cannot feed data or commands back into the 
debugger;
   d) the remote target cannot adjust the debugger to its configuration;
   e) the remote gdb server cannot access the binary, symbol table or 
other bits and pieces that an advanced target interface supporting 
profiling/trace/etc. requires.

Of these a) can be fixed by having another comms flavour (such as direct 
function call or RPC); b) could be fixed by adding the ability to launch 
a process and hook up a pipe or something; and c) and d) can be fixed by 
extending the protocol. The other is more tricky.

 > My initial reaction (without reading the rest of your mail) would be
 > "Not over my dead body!".  In my experience plugin/extension
 > interfaces add a lot of bloat, are difficult to maintain, and
 > difficult to support.

I had planned to add one extra file to GDB (I'm not sure that 
constitutes bloat). This would contain all the code to load the DLL and 
the implementation of the target_ops functions. Looking at it again we 
could unify the latter part with the remote.c code by abstracting away 
the actual communications somehow. I'm not convinced that nailing them 
together would not create more trouble than it is worth though - 
divergent feature sets might introduce a lot of conditional code.

 > All the other bits you wanted to export
 > as methods have less clear semantics, and some of them are guaranteed
 > to change over time.

I certainly do not suggest that we expose any internal GDB structures. 
Not only would doing so make it hard to maintain plugins, they could not 
be compatible with multiple GDB versions.

I propose that a function is added for each piece of data or action that 
a plugin is interested in, and no more. Each of these would be 
independent of the internal GDB implementation. We might have 
'find_symbolname_at_address()' on the GDB side and 
'objfile_has_changed()' on the plugin side. As plugin developers need 
access to other details they can add new methods along similar lines and 
their plugin (or a specific feature of it) will only work with the new 
version of GDB or later. If GDB no longer supports one of these (or they 
are thought ugly or broken) then they can be deprecated (without 
changing the name though) and removed as normal - plugins relying on 
this one feature will need fixing, but not for every new release. New 
ways of doing old things may be added in parallel without trouble. 
Obviously, all new such interfaces would go through the review process, 
just like any other patch.

 > GDB is burdened already with support for a huge
 > number of targets using the _one_ interface; I don't want to deal with
 > backwards compatibility.

I'm not sure what you mean by this. I am not suggesting removing or 
changing the remote target in any way. Of course, to keep the number of 
internal interfaces down there's no reason why it couldn't be 
implemented as a plugin :) I'm not seriously suggesting this.

 > For new CLI commands, the right answer is probably to implement them
 > in-core in some scripting language, and optionally pass things back to
 > your remote target via "monitor" commands.

A new scripting language would, perhaps, make this somewhat less of an 
issue. It is important that there is some way to configure, control and 
query targets with more advanced features. For example, ours supports 
various performance measuring, tracing and direct JTAG wiggling among 
other things. I would be in favour of a more integrated approach. It 
isn't like adding and removing CLI commands is at all difficult.

 > Why can't you just compile your remote protocol support code into GDB.
 > That's the way it's been always done.  It's simple, and works, and I
 > think that'll actually be the easiest to maintain.  Plugin/extension
 > interfaces only make sense for third parties that want to distribute
 > binary stuff.  That doesn't make sense for an open source debugger.

This is indeed the case here. There are certain features of the targets 
which we use under NDA and so those parts of the code must not be open 
sourced. There would be no problem with the rest of the code, beyond 
that we prefer to keep it as a separate project with a separate code base.

 > I do think we will
 > want plugins eventually, but I haven't thought much about whether real
 > dlopen'd plugins are worthwhile, or e.g. python.

Python plugins could certainly provide UIs or user toys, but will they 
ever really be able to connect GDB to a target? Would you want them to? 
I can't see them easily utilising many of GDB's other interfaces either.

 > Short term it'd only increase the load since we
 > have to build the plug-in interface.  Long term, we still have to
 > maintain that interface.

I am intending to build the initial plug-in interface myself, so that 
won't cost you anything beyond review and discussion. It will be one 
self-contained patch which touches very little of the rest of the 
debugger. Most of what it needs is already available for the use of the 
existing target interface and the rest of GDB in general. A few things, 
such as the compare section command, may need some work, but this will 
be quite limited. I do not intend to expose every possible aspect of 
GDB, nor do I think it should. If somebody needs something more later 
then they extend it themselves, also at no great cost to you.

Maintainance is an issue of course, but such is life.

 > We'll also get lots of support questions
 > where the problems are really with the third party plugins, but where
 > the bug reporter conveniently forgets to mention that he's using a
 > third party plugin.  Let's keep people out of our address space if
 > they don't want to commit themselves to providing sources and manpower
 > to maintain those sources.

GDB should report errors that occur within plugins prefixed with the 
plugins name. That should head off most usage related issues. Of course, 
idiots are nothing if not cunning and will always find new ways to waste 
you time so I see your point. Hopefully the command names they quote 
would ring the alarm bells.

 > In other words, if the plug-ins had enough hooks to customize GDB's
 > behavior (printing values, or decoding the stack, say), I think you'd
 > be really surprised what people would do with it.  They'd do tons of
 > stuff that we'd never want to include in the core distribution, but
 > which would add enormous value to GDB for their problems.  And of
 > course, there would be a few that we'd want to incorporate.

I don't have time to go that far in the initial version, but these 
things evolve and I'd be happy to make the start and do targets and 
commands. As I hinted above, there's no point implementing something, in 
an open source project, on the off chance someone will use it. You put 
in the core bits and let people extend it as they need to.

As I think somebody else said, the frontend stuff can be done other 
ways. I'm more interested in backend stuff (which has a frontend presence).

 > I'm already in the long process of creating a level on top of GDB
 > using MI that allows another process to do anything it wants with GDB.

Can it add target interfaces? Efficiently?

 > Oh, I figured we'd just let the plugin code #include GDB's headers
 > directly, so they could get at whatever they wanted.

No, that would be just nasty.

 > ... okay, that does sound like a nightmare.

Agreed, if you tried to take it too far.

 > I did it to replace the current hack of having an extra program on the
 > pc that converts between the default gdb target protocol, and this
 > ICE-specific protocol.

 > The conversion hack was extremely slow, clumsy, fault-intolerant, and 
 > error-prone.

 > With the specific interface in gdb, the performance was wonderful, 
and > new gdb commands can be easily registered for this ICE, that are only
 > accessible after the command "target avrjtagice -d /dev/ttyS2" is
 > performed (like an emacs minor mode or whatever).

So there's at least two (groups) of us have gone through exactly the 
same process. How many others I wonder?

I think the rest of the discussion was mostly about GDB internals and 
documentation. If I have missed any important points that needed an 
answer then I apologise.

Andrew Stubbs

Re: [RFC] plugin/extension interface

[Bob Rossi]

> > I'm already in the long process of creating a level on top of GDB
> > using MI that allows another process to do anything it wants with GDB.
> 
> Can it add target interfaces? Efficiently?

Andrew, I personally think the approach you took to responding is very
confusing. I feel like I'm flipping through chapters of a presedential
debate, and reading random paragraphs. Others may feel different, but if
I were you, I'd respond traditionally.

As far as your question, the answer is no. You would have to extend MI
to be able to add target interfaces efficiently, which would be similar
to extending the CLI. What I am working on, will simply give your
program access to the MI layer, efficently and correctly.

Bob Rossi