New branch created for "available features" support

New branch created for "available features" support

[Daniel Jacobowitz]

Feel free to ignore this; it's mostly a brain dump.  Or feel free to read
down as far as the start of README.AVAIL and stop there.

I've created a new branch:
  gdb-csl-available-20060303-branch

This is for the "available registers as a target property" work that
I described on this list almost a year ago.  I've been working hard
on this, and made a lot of progress over the last few weeks, but
I'm going to have to go out of town on business for most of next week;
and I'm always fearsomely behind on my projects when I get back from
a trip.  So, I may not be able to work on it again for a while.  I
wanted to get my current state into CVS before I left.

The stuff that's there is not, in my opinion, ready.  It needs some
more development, and a few of the items from my TODO list (see below)
are really important to consider before we start using this.  But,
what's on the branch works!  I hacked up a local embedded ARM stub
to supply the sample description below, and the associated registers,
and GDB will successfully display them.  At least it did a few days ago
when I last tried it.  So, it's not done, but it's real.

Here's the DTD and the current README.AVAIL file; I'll probably think
of more TODO items that I forgot to write down, and there's plenty of them
in the code already.  I'm always happy for folks to comment on either
the code or the approach; one of the problems in designing something
extensible is allowing it to extend to things I haven't thought of
yet.

That's why I eventually settled on XML for the descriptions, by the way. 
With the caching items mentioned in the TODO, and support for reading the
descriptions from files instead of from the target, I don't consider the
extra bandwidth to be a problem.  And being able to use e.g. GUI
XML editors to generate the descriptions and standards-compliant XML
validators to verify the syntax has already been really handy.  So I
think this is the right choice.  I've put expat 2.0.0 on the branch,
with a small change to make it build in our environment; this won't
impose any new build requirements.  It's not as trivial to handle
as a full-featured XML library with DOM (Document Object Model)
support, but it's also not very large.  I considered using libxml2
instead so that I could take advantage of the DOM, but now that I've
written the expat parser, I think it's not _too_ horrible.  It's a
somewhat weird style to me, though.








README.AVAIL:


Notes for the branch "gdb-csl-available-20060303-branch"
--------------------------------------------------------

This branch implements a new mechanism which allows GDB to ask a target
"what features do you have?" GDB can then interpret the response and
dynamically present those features to the user.  Some features require
corresponding support in GDB, and must be specially recognized by the target
architecture.  Others do not require additional GDB support, e.g. additional
registers (the only type of feature implemented so far).

The branch does not have a ChangeLog relative to mainline; one will be written
later.  So far, only the ARM target has any support for available features.

The most interesting portion of this document is the TODO list; the rest may
get somewhat out of date.

Control flow in GDB
-------------------

After connecting to the target, we check the new architecture-provided setting
gdbarch_available_features_support.  If it is set, we query the target
for its available features, interpret the response, and switch to a new
gdbarch, derived from the current one, with these features recorded.

In order for the derivation process to work, the architecture's gdbarch_init
must correctly support filling in defaults based on the last used architecture.
If it does not, for example, cache something read from the ELF binary in
gdbarch_tdep, the architecture is likely to get out of sync with the
debuggee.

During debugging, GDB can query information from the current set of
features.  This is currently done in architecture-specific hooks, but may be
done in common code in the future.

Writing a feature description
-----------------------------

Feature descriptions are written in XML.  The current DTD is in
gdb/features/gdb-target.dtd.  There are some limits beyond those
expressed in the DTD - many of these limits are not yet documented
and not yet relevant until additional GDB support has been implemented.
See the TODO.

Here's a simple sample description:

<?xml version="1.0"?>
<!DOCTYPE target SYSTEM "gdb-target.dtd">
<target>
  <feature name="bar">
    <reg name="s0" bitsize="32"/>
    <reg name="s1" bitsize="32" type="float"/>
  </feature>

  <feature-set>
    <feature-ref name="bar" base-regnum="1"/>
  </feature-set>
</target>

This describes a simple target feature set which only contains two registers,
named s0 (32-bit, integer) and s1 (32-bit, floating point).

You can spread a description over multiple files by using the standardized
XInclude mechanism - but only a very simplistic form of XInclude is supported.
The xpointer attribute must be provided, using a bare ID rather than a more
complicated XPointer expression.  The href argument should also be provided,
using a bare basename.  GDB will query the target for the file, if it has
not already seen it.  Presently only <feature> elements may be read using
XInclude.

You can validate the description using any XML validator which supports XInclude.
For instance, with "xmllint" (shipped as part of the GNOME libxml2 package):

	xmllint --xinclude --postvalid my-target.xml

Post validation is usually appropriate when using XInclude; the DTD describes
the document after XInclude processing.




TODO items and unsettled (or unasked) questions
-----------------------------------------------

When finished this documentation needs to move into the user and internals
manual.

The "ro" and "save-restore" tags may not express enough information about
how to treat system registers.  Richard Earnshaw suggested a more detailed
categorization; I need to consider it.

Reading and writing registers using the 'p' and 'P' packets is very inefficient;
a target mechanism and remote protocol packets for multiple register batching
would probably help a lot.

For ARM VFP, there are two views of some registers: s0 / s1 are single precision
registers overlayed in storage with d0, a double precision register.  Many
other targets do the same thing.  Should we express this to GDB, or just
allow it to read them both from the target (somewhat wasteful)?  GDB already
assumes that modifying one register may modify another unpredictably, so
writing is OK.

The DTD allows for description fields, including multi-lingual ones, but
there is no GDB support for descriptions.  It would be good to present
them to the user.

Should we convey the information read from the target (after e.g. XInclude
processing) to MI front ends, or are the changes to the register cache
sufficient?  For instance, Eclipse would probably be happy to display
most of this data (especially descriptions).

The current DTD and GDB support does not allow for nested features.  This
is probably useful.

GDB needs additional error checking for its assumptions of unique names.
For instance, there may be multiple registers with the same name in
GDB's feature database, but they had better not be instantiated in
the same feature set.

Feature sets should probably have names, so that they can be referenced
uniquely and cached, to minimize data the target has to supply.

We need a naming scheme for features (and maybe feature sets).  Some
considerations:
  - Names should generally be globally unique, so that we can efficiently
    cache features and even ship their descriptions with GDB.  Having the
    feature on the target not match GDB's cached value is likely to
    lead to mayhem.  When caching is implemented, perhaps we should
    also have a maint command to check that the cache is correct, for
    targets which can supply their features in detail (it's possible
    that the target can't, and instead relies on GDB loading them
    from files).
  - It should be hierarchical, so that vendors may create their
    own names without risk of interfering with future GDB development
    or other vendors.
  - There should probably be a namespace which will not be cached,
    for convenience during development, or for features which
    dynamically reconfigure on the target.

Should known features be compiled in to GDB, or loaded from the filesystem?
The most essential features should probably be compiled in, so that the
GDB binary is useful standalone.

GDB should support reading features and feature sets from disk instead of
from the target.

GDB should support caching features read from the target in a user-specified
directory.

Should GDB warn about unrecognized features which require additional GDB
support, or silently ignore them?

If the name field of features is hierarchical, and the description is free-form,
there should probably be a "short description" field - a user label without
the uniqueness constraint.

Another suggested type of feature is a memory map - which areas of memory
the debugger may read from / write to.

How should features interact with the standard architecture support?  Basic
options:
  - Target features must not specify the standard registers.
  - Target features may specify the standard registers, and GDB will handle
    any duplication.
  - Target features must specify the standard registers.  GDB can provide
    a standard feature for the architecture, which must be referenced
    (or a set of such standard features, at least one of which must be
    referenced).
I'm somewhat leaning towards #3, but it requires buy-in from target maintainers
who wish to support this feature.  It's nice in that it moves a bit of code
from the tdep files out into a description file; but less nice in that
it might lose flexibility that we need; the target might have to run-time
generate the XML.  Are there any examples where that would be necessary?


The DTD:


<!-- The root element of a GDB target description is <target>.  It
     contains a list of feature definitions, followed by a feature-set.
     This is also the only point at which xi:include is supported;
     it must be used with xpointer to fetch a feature, from a
     document whose root element is either target or feature.  -->

<!ELEMENT target	(feature*, feature-set)>
<!ATTLIST target
	xmlns:xi	CDATA	#FIXED "http://www.w3.org/2001/XInclude">

<!ELEMENT feature-set	(description*, feature-ref+)>

<!-- QUESTION: Is there any reason for feature-ref to have its own
     descriptions?  Or a short name field (descriptive)?  -->
<!ELEMENT feature-ref	EMPTY>
<!ATTLIST feature-ref
	name		IDREF	#REQUIRED
	base-regnum	CDATA	#IMPLIED>

<!-- TODO: Handle arch_data, maybe as unvalidated fields; do we want
    to define a namespace for arch-specific fields?  Issue for feature
    and for reg.  -->

<!-- QUESTION: Should the feature also have a short description to identify
     it?  The format of its "name" field is restricted and probably not
     user-appropriate.  -->
<!ELEMENT feature	(description*, reg*)>
<!ATTLIST feature
	name		ID	#REQUIRED>

<!-- TODO: GDB does not yet support descriptions.  -->

<!-- Registers do not have an explicit register number field; they
     are numbered sequentially from the containing feature's base-regnum
     when the feature is referenced.  -->
<!-- arch_data; see above -->
<!-- Kill save-restore in favor of a more complete scheme -->
<!ELEMENT reg		(description*)>
<!ATTLIST reg
	name		CDATA	#REQUIRED
	bitsize		CDATA	#REQUIRED
	readonly	(yes | no) 'no'
	save-restore	(yes | no) 'yes'
	type		CDATA	'int'
	group		CDATA	#IMPLIED
	>

<!ELEMENT description	(#PCDATA)>
<!ATTLIST description
	xml:lang	CDATA	#IMPLIED>

-- 
Daniel Jacobowitz
CodeSourcery

Re: New branch created for "available features" support

[Eli Zaretskii]

> Date: Fri, 3 Mar 2006 16:35:07 -0500
> From: Daniel Jacobowitz <drow@false.org>
> 
> Feel free to ignore this; it's mostly a brain dump.

A good place to keep this stuff and maintain it as ideas evolve and
design decisions are made is in gdbint.texinfo on that branch.  That
way, when this becomes part of the mainstream, it is automagically
documented ;-)

Thanks.

Re: New branch created for "available features" support

[Daniel Jacobowitz]

On Fri, Mar 03, 2006 at 11:55:35PM +0200, Eli Zaretskii wrote:
> > Date: Fri, 3 Mar 2006 16:35:07 -0500
> > From: Daniel Jacobowitz <drow@false.org>
> > 
> > Feel free to ignore this; it's mostly a brain dump.
> 
> A good place to keep this stuff and maintain it as ideas evolve and
> design decisions are made is in gdbint.texinfo on that branch.  That
> way, when this becomes part of the mainstream, it is automagically
> documented ;-)

See the very top item in the TODO list :-)

I'm definitely not going to forget about it, but I find a text
file easier to evolve when I'm still making big changes, so I'm
going to keep it in README.AVAIL until I'm a little happier with
the overall shape.

-- 
Daniel Jacobowitz
CodeSourcery

Re: New branch created for "available features" support

[Jim Blandy]

So a "feature" would be something like "SSE", or "MMX" with the
additional SSE registers, and then a "feature set" would be something
like "Pentium MMX" or "Pentium 3", referring to the appropriate
individual feature sets?

I understand why you want to supply base register numbers in feature
sets, rather than features; it makes it possible to share features. 
But if we're going to have other sorts of features as of yet
unimagined, they may or may not expect a base register number as a
parameter.  That is, you've tied something specific to register banks
to your <feature> entities.

I'd say, rename <feature> to <register-set>.  Introduce new,
appropriately named kinds of entities for new kinds of features.  That
way, you can look at the tag and know what sorts of parameters it
expects.

Re: New branch created for "available features" support

[Eli Zaretskii]

> Date: Fri, 3 Mar 2006 16:58:00 -0500
> From: Daniel Jacobowitz <drow@false.org>
> Cc: gdb@sourceware.org
> 
> I'm definitely not going to forget about it, but I find a text
> file easier to evolve when I'm still making big changes

Texinfo is just text with a few markups.

Re: New branch created for "available features" support

[Daniel Jacobowitz]

On Fri, Mar 03, 2006 at 02:36:22PM -0800, Jim Blandy wrote:
> So a "feature" would be something like "SSE", or "MMX" with the
> additional SSE registers, and then a "feature set" would be something
> like "Pentium MMX" or "Pentium 3", referring to the appropriate
> individual feature sets?

s/feature sets?/features?/ at the end.  Otherwise, yes.  At the moment
at least (before I get back to the questions of nesting) a feature set
represents the entire target.

> I understand why you want to supply base register numbers in feature
> sets, rather than features; it makes it possible to share features. 
> But if we're going to have other sorts of features as of yet
> unimagined, they may or may not expect a base register number as a
> parameter.  That is, you've tied something specific to register banks
> to your <feature> entities.
> 
> I'd say, rename <feature> to <register-set>.  Introduce new,
> appropriately named kinds of entities for new kinds of features.  That
> way, you can look at the tag and know what sorts of parameters it
> expects.

My mental model was that a single feature might contain registers, or
not, and something else, or not.  That's why base-regnum is optional.
So a feature might contain, say, registers and MMIO ports and have a
base for both; but if it contains no registers, base-regnum might be
omitted.

However, you may be right that I've got the separation at the wrong
level - I'll have to think about that one for a little.  If there's
any advantage to the separation it would be easy to change.

-- 
Daniel Jacobowitz
CodeSourcery