to_xfer_partial, qPart, and EOF

to_xfer_partial, qPart, and EOF

[Daniel Jacobowitz]

I started adding a new qPart packet type today and discovered that the
current status quo is fairly confused.  I'll start with the qPart packet. 
Some highlights can be found here, in Roland's posting of the protocol
documentation:

    http://sources.redhat.com/ml/gdb/2004-02/msg00033.html

The original posting used a reply of =DATA to mean "here's something, and
EOF", and +DATA to mean "here's something, but there might be more".  It
did not explicitly mention short responses.

Andrew revised this to something very like the final form; that final form
is:

     Reply:
    `OK'  
          The OFFSET in the request is at the end of the data.  There
          is no more data to be read.

    XX...
          Hex encoded data bytes read.  This may be fewer bytes than
          the LENGTH in the request.

    `E00'
          The request was malformed, or ANNEX was invalid.

    `E'NN
          The offset was invalid, or there was an error encountered
          reading the data.  NN is a hex-encoded `errno' value.

    `""' (empty)
          An empty reply indicates the OBJECT or ANNEX string was not
          recognized by the stub.

So, short responses are allowed, and they do not imply EOF.

A problem with this is that the short reply indicating EOF has gotten lost. 
Roland wrote:

> This looks fine to me, basically.  But I do think it is worth starting out
> with the one more wrinkle of EOF reporting along with data return.  With
> this simpler protocol, there are necessarily two request/reply transactions
> for every time you want to read "all the data", even if there is as little
> as one byte of it to read.  That seems unnecessary and undesireable.
> (Perhaps I've spent too long doing serial debugging when serial ports were
> slower than they usually are these days.  But all the remote protocol
> traffic adds up to slow sessions hacking kernels.)

I'm currently very well aware of the round trip latency in the remote
protocol; it makes a huge difference over TCP, where larger packets are
basically free up to a certain size, but round trips are very slow.  At
the time he wanted to pursue this immediately; Andrew wanted to go without
it for the moment, and it was never revisited.

So that's action item one; I think it's time to add this.  So far so good.

I also looked at the existing code in GDB.  The remote.c support, as
originally committed and still today reads the whole object in
remote_xfer_partial (it will loop and send multiple packets if necessary).
It was approved, of course, but I get the impression Andrew didn't much look
at it - he would have jumped on that normally.

There's several matching bugs in auxv.c: it asserts that there were no short
reads, and it does the second and later reads still with offset 0 but puts
them into consecutive locations in the buffer.  Good thing no one's auxv
vector ever exceeds that first read in practice.

My current proposal is to make these changes:

  - Allow the qPart remote packet to signal EOF after a read.  I think
    the best options are an equals sign at the beginning of the packet
    (as Roland suggested originally), or an equals sign at the end of
    the packet (which might be easier on stubs).

  - Modify the to_xfer_partial interface to signal EOF independently
    from buffer length.  I don't see a way to do this other than by
    adding an argument to target_read_memory and all to_xfer_partial
    implementations.

Any comments on either of these plans?  Otherwise I will probably implement
them in the next couple of days.  I'm doing a lot of work in this area
at present.

-- 
Daniel Jacobowitz
CodeSourcery

Re: to_xfer_partial, qPart, and EOF

[Jim Blandy]

On 1/18/06, Daniel Jacobowitz <drow@false.org> wrote:
> I'm currently very well aware of the round trip latency in the remote
> protocol; it makes a huge difference over TCP, where larger packets are
> basically free up to a certain size, but round trips are very slow.  At
> the time he wanted to pursue this immediately; Andrew wanted to go without
> it for the moment, and it was never revisited.
>
> So that's action item one; I think it's time to add this.  So far so good.

This, and a few other remote protocol wrinkles, are consequences of
the mis-layering of the GDB protocol.  The protocol should simply
specify that the entire block of data gets transmitted, and let lower
layers handle retransmission and fragmentation.

I recognize it's probably not practical to fix this today, and maybe
it never will be.  But I keep running into instances of this when I
work on the remote protocol --- tracepoint definition packets needing
to be broken up into pieces to avoid long packets; breakpoint packets
needing to be idempotent, because they might be retransmitted; and so
on --- so I wanted to mention it.  Perhaps someone will have a flash
of super-coder powers some weekend.

> Any comments on either of these plans?  Otherwise I will probably implement
> them in the next couple of days.  I'm doing a lot of work in this area
> at present.

Sounds great to me.

Re: to_xfer_partial, qPart, and EOF

[Daniel Jacobowitz]

On Wed, Jan 18, 2006 at 05:18:49PM -0800, Jim Blandy wrote:
> This, and a few other remote protocol wrinkles, are consequences of
> the mis-layering of the GDB protocol.  The protocol should simply
> specify that the entire block of data gets transmitted, and let lower
> layers handle retransmission and fragmentation.
> 
> I recognize it's probably not practical to fix this today, and maybe
> it never will be.  But I keep running into instances of this when I
> work on the remote protocol --- tracepoint definition packets needing
> to be broken up into pieces to avoid long packets; breakpoint packets
> needing to be idempotent, because they might be retransmitted; and so
> on --- so I wanted to mention it.  Perhaps someone will have a flash
> of super-coder powers some weekend.

I suspect we will need to redesign the protocol rather than just
layering on top of it, eventually.  Maybe not.  I don't know; I'm not
feeling that ambitious right now.

The two things I most often hear about as fundamental problems with the
existing protocol are:

  - multiple "channels" for output to/from the host, especially
    asynchronous channels (logging without stopping the target)

  - other asynchronous communication, e.g. collecting tracepoints
    or reading memory without stopping the target, where supported

> > Any comments on either of these plans?  Otherwise I will probably implement
> > them in the next couple of days.  I'm doing a lot of work in this area
> > at present.
> 
> Sounds great to me.

Thanks.

-- 
Daniel Jacobowitz
CodeSourcery