remote nits

remote nits

[Quality Quorum]

Hi, 


I found more remote nits reading manual on the net and 
gdb-19990913/gdb/remote.c . Below is summary.


A. Handling of Depreciated Elements

   It seems to me that debugger should never emit 
   depreciated requests. 'qL' request  is marked 
   as depreciated in the document, however, there is 
   code which will emit it.

B. Three Ways of Specifying a Thread.

   There are THREE ways of specifying thread: signed 32-bit 
   integer (used by 'H' request and 'T' response), 16-bit unsigned
   integer (used by 'T' request  and 'QC' response) and 8-byte thread 
   id (used by 'qL' and 'qP' requests and corresponding responses). 

   BTW, Code implementing 'QC' responses will handle 32-bit signed 
   integers properly while manual tells us that it is using 16-bit
   unsigned ones.

   It seems to me that only one way should survive and others
   should die. My list of preferences is (1) 32-bit integer, 
   (2) 8-byte thread ref, (3) 16-bit unsigned integer. (*) 

C. Requests Violating Protocol Principles

   There are four commands which violate protocol 
   principles. '!' is expecting empty response both 
   in case it is supported or not. If stub does not support
   'D' it will loop forever trying to tell gdb about it.
   'qRcmd' can have a valid request starting from 'E'
   'TXX' request interprets 'ENN' as thread dead indication.

   It seems to me that '!' is not that bad. At least one
   popular implementation I saw (gdbserver) will report last status
   in response to '!'.  It seems that a low pain way to deal 
   with it is to have local variable, say 'remoteforceextended'.
   If this variable is set then any response to '!' is considered
   a success, otherwise "" means no support, "OK" or status string
   mean support, everything else considered an error.

   It seems to me that 'D' could be depreciated: we can use 'c'
   followed by connection close instead - 'c' supposed to have exactly
   same effect and is required to be implemented. Another way is 
   to define new detach request which will require stub to send
   reply before disconnecting.

   It is unclear from document that 'qRcmd' indeed allows responses
   starting from 'E', however there is a code handling this case
   in remote.c

   It seems to me that it is possible to leave TXX request as is.
   Moreover it seems like it is going to be depreciated as a part 
   of thread id normalization process anyway.

D. Bugs

   Three reqests will not work with 64-bit targets 't', 'qCRC' and 
   'qOffsets' due to address range limitations.


I feel ready to write (a) a formal description of the protocol
with (b) proposed changes, once issue (*) of proper representation of
thread ids is resolved one way or another.


Thanks,

Aleksey

Re: remote nits

[J.T. Conklin]

>>>>> "Quality" == Quality Quorum <qqi@world.std.com> writes:
Quality> It seems to me that only one way should survive and others
Quality> should die. My list of preferences is (1) 32-bit integer,
Quality> (2) 8-byte thread ref, (3) 16-bit unsigned integer. (*)

One common way to represent threads is using the address of the thread
control block --- on a machine with 64 bit addressing, a 64 bit thread
identifier would be used.  I'd rather use that address than attempt to
map it into a smaller field.

Perhaps GDB could use an abstract thread representation rather than
requiring targets use any particular thread representation.  Perhaps 
a variable length 'cookie' -- an 8-bit length field followed by that
many bytes of info.

I seem to remember that NFSv2's 32 byte file handle was changed to a
variable length string in NFSv3.

        --jtc

-- 
J.T. Conklin
RedBack Networks

Re: remote nits

[Quality Quorum]

On 13 Dec 1999, J.T. Conklin wrote:

> >>>>> "Quality" == Quality Quorum <qqi@world.std.com> writes:
> Quality> It seems to me that only one way should survive and others
> Quality> should die. My list of preferences is (1) 32-bit integer,
> Quality> (2) 8-byte thread ref, (3) 16-bit unsigned integer. (*)
> 
> One common way to represent threads is using the address of the thread
> control block --- on a machine with 64 bit addressing, a 64 bit thread
> identifier would be used.  I'd rather use that address than attempt to
> map it into a smaller field.
> 
> Perhaps GDB could use an abstract thread representation rather than
> requiring targets use any particular thread representation.  Perhaps 
> a variable length 'cookie' -- an 8-bit length field followed by that
> many bytes of info.
> 
> I seem to remember that NFSv2's 32 byte file handle was changed to a
> variable length string in NFSv3.

1. I was mistaken there are 4 ways to represent thread: 16-bit 
   integer is used by 'qC' and 8-bit integer is used by 'TXX'.

2. Yes, there are at least 5 ways to solve this problem (invent
   something new or use one of 4 exiting ones). The question was
   which one is more appropriate ? 


I would go with following set of rules: (1)thread id is an unsigned 
64-bit integer represented on the wire in %x format (leading zeros
are omitted) in all transactions except 'qP', 'qQ', 'qL' and 'qM'
which use %08x representation; (2) requests 'Hc-xxxxxx' and 'Hg-xxxxx'
should be interpreted as 'all threads' regardless of the length and 
the pattern of 'xxxxxx', (3) 'TXX' is depreciated. It seems reasonable 
to reserve thread_id 0xffffffffffffffff for internal use, so developers
will be able to use the value internally to pass 'all threads' value
between various parts of there stuff.

In may view it will not break any existing code on the stub side.
The only problem will be that newly developed 64-bit stubs 
working with memories above 4GB (which can use big thread_ids) will not
work with old gdbs, but it was going to be the case anyway (there are
quite few non 64-bit transparent places in existing gdb).

>         --jtc

Thanks,

Aleksey

Re: remote nits

[Andrew Cagney]

Quality Quorum wrote:
> 
> Hi,
> 
> I found more remote nits reading manual on the net and
> gdb-19990913/gdb/remote.c . Below is summary.
> 
> A. Handling of Depreciated Elements
> 
>    It seems to me that debugger should never emit
>    depreciated requests. 'qL' request  is marked
>    as depreciated in the document, however, there is
>    code which will emit it.

Its important to apreciate how GDB needs to continue to support existing
targets.  While we don't want anyone to use QL (and wish it was replaced
with something better) that doesn't mean that it is reasonable to simply
pull that code.  People in the field need to be given several years
notice and the chance to migrate to something better. it.

Much of the thread code is in this situtation.

> C. Requests Violating Protocol Principles

>    It seems to me that '!' is not that bad. At least one
>    popular implementation I saw (gdbserver) will report last status
>    in response to '!'.  It seems that a low pain way to deal
>    with it is to have local variable, say 'remoteforceextended'.
>    If this variable is set then any response to '!' is considered
>    a success, otherwise "" means no support, "OK" or status string
>    mean support, everything else considered an error.

You may want to study gdbserver more closely. The thing that it does is
change GDB servers behaviour.  Strictly speaking both of those should be
controlled through ``[Qq]'' packets and ``!'' should be given the boot.

>    It seems to me that 'D' could be depreciated: we can use 'c'
>    followed by connection close instead - 'c' supposed to have exactly
>    same effect and is required to be implemented. Another way is
>    to define new detach request which will require stub to send
>    reply before disconnecting.

There is a good chance that nothing uses it.
However, what mechanism are you proposing that will allow GDB to
disconnect from one target programme and then connect to a second?

>    It is unclear from document that 'qRcmd' indeed allows responses
>    starting from 'E', however there is a code handling this case
>    in remote.c

The normal response to Rcmd is a HEX encoded byte-stream. E is a hex
digit.

	Andrew

Re: remote nits

[Quality Quorum]

On Tue, 14 Dec 1999, Andrew Cagney wrote:

> Quality Quorum wrote:
> > 
> > Hi,
> > 
> > I found more remote nits reading manual on the net and
> > gdb-19990913/gdb/remote.c . Below is summary.
> > 
> > A. Handling of Depreciated Elements
> > 
> >    It seems to me that debugger should never emit
> >    depreciated requests. 'qL' request  is marked
> >    as depreciated in the document, however, there is
> >    code which will emit it.
> 
> Its important to apreciate how GDB needs to continue to support existing
> targets.  While we don't want anyone to use QL (and wish it was replaced
> with something better) that doesn't mean that it is reasonable to simply
> pull that code.  People in the field need to be given several years
> notice and the chance to migrate to something better. it.

I do appreciate and I do understand it, however, there is also a need
in clear documentation. It seems to me that we have to use 
different term (which one ???) to describe things which are going
to be replaced, however, are still in use.

BTW, my general attitude is 'do not change anything, unless it is 
absolutely necessary', however, clean documentation, with clear 
migration part is essential in my view. 
 
> 
> Much of the thread code is in this situtation.
>

There is much more to the thread code than that. As far as I can
see it looks as follows, (1)gdb uses 32-bit signed pid internally
to identify thread, (2) remote machine uses the 64-bit address of the
thread control block to identify the thread,(3)gdb side does 
conversion, (4) this conversion is not fully/cleanly implemented
yet. 

There are three ways to address this issue:

1. Make target to convert its internal thread_ids into pids.
   
   Pro: (a)it provides nice abstract interface, (b) it is not that 
        big a deal for a target to implement, (b) it allows for effective
        implementation of thread list feature, (c) it is fully
        compatible with all operations except 'qL' and 'qP' requests .

   Con: (a) it breaks 'qL' and 'qP' requests. (In my view this is 
        a fatal problem for this approach).

2. Fully implement conversion on gdb side.

   Pro: (a) it provides thread_id which may have meaning for the person
        debugging the stuff, (b) it is trivial to implement on 
        gdb side, (c) it does not break 'qL' and 'qP' requests.

   Con: (a) there are cases where it will break 'qC', 'T' - both cases
        and 'Hx' requests, (b) in many cases target side overhead
        similar to the case (1) will be required to properly implement
        thread list.

3. Fully implement conversion on gdb side with modifications:

      o Thread ids visible on the wire are 64-bit unsigned values

      o Thread id 0xffffffffffffffff means 'all-threads', if no 
        threads are implemented this is the only thread visible 
        to gbd (internally it matches to magic - 'all-thread' pid of 
        4200).

      o There are two formats to represent a thread_id on the wire.

        V-thread - byte encoded into 1-16 characters MSB first,
        with possibly omitted leading 0's, with 'all-threads' id
        encoded as '-xxxxxxxxxxx', where 'xxxxxxx' is any combination of 
        1-16 hex digits, with preferred form being '-1'. E.g.

           if(thread_id == XXX_VAL_THEADID_ALLTHREADS)
             {
	       out_buf[0] = '-';
               out_buf[1] = '1';
               out_buf[2] = 0;
	     }
           else
             {
               sprintf(out_buf, "%Lx", thread_id);
             }
        
        and

        F-thread - byte encoded into 16 characters MSB first,
        e.g. sprintf(out_buf, "%016Lx", thread_id);.

      o Requests 'Hx', 'T' and 'qC' use V-thread format, 'qP' and 'qL'
        use F-thread format.

      Comparing to (2) it has additional advantages (a) of being
      completely compatible with all existing implementations,
      while providing a clean migration path, (b) it will allow to 
      use 'qL' and 'qP' requests without reinventing them.
       
In my view (3) is a clear winner I am going to write formal document
specifying protocol using this encoding during the day today.


> > C. Requests Violating Protocol Principles
> 
> >    It seems to me that '!' is not that bad. At least one
> >    popular implementation I saw (gdbserver) will report last status
> >    in response to '!'.  It seems that a low pain way to deal
> >    with it is to have local variable, say 'remoteforceextended'.
> >    If this variable is set then any response to '!' is considered
> >    a success, otherwise "" means no support, "OK" or status string
> >    mean support, everything else considered an error.
> 
> You may want to study gdbserver more closely. The thing that it does is
> change GDB servers behaviour.  Strictly speaking both of those should be
> controlled through ``[Qq]'' packets and ``!'' should be given the boot.

It seems to me that we have widely used implementation (gdbserver)
with sensible handling of '!', so in my view it is safe to make it 
default behavior with old behavior supported when some variable (e.g.
extendedremoteforce) is set. 


> >    It seems to me that 'D' could be depreciated: we can use 'c'
> >    followed by connection close instead - 'c' supposed to have exactly
> >    same effect and is required to be implemented. Another way is
> >    to define new detach request which will require stub to send
> >    reply before disconnecting.
> 
> There is a good chance that nothing uses it.
> However, what mechanism are you proposing that will allow GDB to
> disconnect from one target programme and then connect to a second?

1. If there is no threads supported on the remote machine, then
   the topic is moot.

2. It there threads then the sequence could be
    'Hc...'  - set new control thread
    'Hg...'  - set new generic thread (whatever does it mean).
    'xxxx'   - set break point in the new control thread context
    'c'      - continue


> >    It is unclear from document that 'qRcmd' indeed allows responses
> >    starting from 'E', however there is a code handling this case
> >    in remote.c
> 
> The normal response to Rcmd is a HEX encoded byte-stream. E is a hex
> digit.

???????? As far as I understand  the document normal response to 
Rcmd is a sequence of zero of more 'O' packets followed, by OK or 'ENN'.


IMHO, the only use of Rcmd is to allow user to type some command 
and see the response, in all other cases query should be defined and
used not a command, we have raw queries just for this purpose.

> 
> 	Andrew
> 

Thanks,

Aleksey

Re: remote nits

[Andrew Cagney]

Quality Quorum wrote:
> 
> 
> ???????? As far as I understand  the document normal response to
> Rcmd is a sequence of zero of more 'O' packets followed, by OK or 'ENN'.

One of the specified return codes is:

    reply OUTPUT 
	A command response with the hex encoded output string OUTPUT. 

and is separate to in intermediate O packet.

> IMHO, the only use of Rcmd is to allow user to type some command
> and see the response, in all other cases query should be defined and
> used not a command, we have raw queries just for this purpose.

Sorry you've lost me here.

	Andrew

Re: remote nits

[Quality Quorum]

On Wed, 15 Dec 1999, Andrew Cagney wrote:

> Quality Quorum wrote:
> > 
> > 
> > ???????? As far as I understand  the document normal response to
> > Rcmd is a sequence of zero of more 'O' packets followed, by OK or 'ENN'.
> 
> One of the specified return codes is:
> 
>     reply OUTPUT 
> 	A command response with the hex encoded output string OUTPUT. 

This one should be outlawed before it is too late, it simply does not
make any sense.

> > and is separate to in intermediate O packet.


> > > IMHO, the only use of Rcmd is to allow user to type some command
> > and see the response, in all other cases query should be defined and
> > used not a command, we have raw queries just for this purpose.
> 
> Sorry you've lost me here.


The question is who is going to issue a command and to interpret a command
response ? 

If it is a user then proper response have to be in user-readable
form ('O' packet). If it is machine, then command query should not be
used (we have raw queries for the purpose).

> 
> 	Andrew
> 

Thanks,

Aleksey

Re: remote nits

[J.T. Conklin]

>>>>> "Quality" == Quality Quorum <qqi@world.std.com> writes:
Quality> 1. I was mistaken there are 4 ways to represent thread:
Quality>    16-bit integer is used by 'qC' and 8-bit integer is used
Quality>    by 'TXX'.
Quality>
Quality> 2. Yes, there are at least 5 ways to solve this problem
Quality>    (invent something new or use one of 4 exiting ones). The
Quality>    question was which one is more appropriate ?

Perhaps the best thing to do is to deprecate the entire set of thread
commands and replace them with a consistantly designed and rigorously
specified set.  I believe that Michael Snyder expressed some interest
in doing some of this (in the context of thread list queries) in the
past.

Quality> I would go with following set of rules: (1)thread id is an
Quality> unsigned 64-bit integer represented on the wire in %x format
Quality> (leading zeros are omitted) in all transactions except 'qP',
Quality> 'qQ', 'qL' and 'qM' which use %08x representation;

Wouldn't that be %016x?

Quality> (2)requests 'Hc-xxxxxx' and 'Hg-xxxxx' should be interpreted
Quality> as 'all threads' regardless of the length and the pattern of
Quality> 'xxxxxx',

The notion of all-threads leads me to belive that perhaps we need some
way to specify both a context and a context id, even if GDB can't take
advantage of this at this time.  For example, possible contexts could
be 'system', 'process', or 'thread'.

The context id would most likely be irrelevant for the system context,
but would be the process id or or thread id for the process or thread
contexts.

One of our goals is a single GDB that can debug multiple processes the
same way (or even better) we can debug multiple threads today.  Since
those processes could have multiple threads, GDB will have to be able
to represent both the processes and the threads.  These contexts may
be a way to accomplish this.  I realize that a lot of work in GDB's
core is necessary, but if we're going to change the protocol at all,
we might as well future proof the investment.

        --jtc

-- 
J.T. Conklin
RedBack Networks

Re: remote nits

[Andrew Cagney]

Quality Quorum wrote:
> 
> On Wed, 15 Dec 1999, Andrew Cagney wrote:
> 
> > Quality Quorum wrote:
> > >
> > >
> > > ???????? As far as I understand  the document normal response to
> > > Rcmd is a sequence of zero of more 'O' packets followed, by OK or 'ENN'.
> >
> > One of the specified return codes is:
> >
> >     reply OUTPUT
> >       A command response with the hex encoded output string OUTPUT.
> 
> This one should be outlawed before it is too late, it simply does not
> make any sense.

Sorry, to late.  Technically it is the ``OOUTPUT'' bit that shouldn't be
there.  Only the qRcmd packet supports it.  However, I do see your
point.

	Andrew

Re: remote nits

[Quality Quorum]

On Wed, 15 Dec 1999, Andrew Cagney wrote:

> Quality Quorum wrote:
> > 
> > On Wed, 15 Dec 1999, Andrew Cagney wrote:
> > 
> > > Quality Quorum wrote:
> > > >
> > > >
> > > > ???????? As far as I understand  the document normal response to
> > > > Rcmd is a sequence of zero of more 'O' packets followed, by OK or 'ENN'.
> > >
> > > One of the specified return codes is:
> > >
> > >     reply OUTPUT
> > >       A command response with the hex encoded output string OUTPUT.
> > 
> > This one should be outlawed before it is too late, it simply does not
> > make any sense.
> 
> Sorry, to late.

Then it has to be depreciated right away - it is broken in too many
places and it is not going to ever work correctly anyway.

And again we can cover it be variable, say - 'brokenqrcmd' 

> Technically it is the ``OOUTPUT'' bit that shouldn't be
> there.  Only the qRcmd packet supports it.  However, I do see your
> point.
> 
> 	Andrew
> 

Thanks,

Aleksey

Re: remote nits

[Quality Quorum]

> >    It is unclear from document that 'qRcmd' indeed allows responses
> >    starting from 'E', however there is a code handling this case
> >    in remote.c
> 
> The normal response to Rcmd is a HEX encoded byte-stream. E is a hex
> digit.

It seems like there is an established practice of using low case
characters for hex encodings sent by stubs - otherwise 'm' request
would never work correctly. So, this is not an issue at all, unless,
you would tell me that existing implementations of qRcmd break
this one too :).

BTW, I got it while working on v0.2 of protocol spec, I was quite 
surprised that the positive impact of this exercise was that immediate.
 
> 
> 	Andrew
> 

Thanks,

Aleksey