GDB/MI Output Syntax

GDB/MI Output Syntax

[Bob Rossi]

Hi,

Along with the newline changes, there are 2 other changes that I propose
to the grammar. They are,

async-record   ==> exec-async-output | status-async-output | notify-async-output
exec-async-output    ==> [ token ] "*" async-output
status-async-output  ==> [ token ] "+" async-output
notify-async-output  ==> [ token ] "=" async-output]

to 

   async-record ==> [token] async-record-kind async-output
   async-record-kind ==> "*" | "+" | "="

however, if you really would like to keep the *-async-output rules, we
could do

   async-record ==> [token] async-record-kind async-output
   async-record-kind ==> exec-async-output | status-async-output | notify-async-output
   exec-async-output    ==> "*"
   status-async-output  ==> "+"
   notify-async-output  ==> "=" 

The second change is identical but refers to the rules of
   stream-record ==> console-stream-output | target-stream-output | log-stream-output
   console-stream-output ==> "~" c-string
   target-stream-output ==> "@" c-string
   log-stream-output ==> "&" c-string

to
   stream-record => stream-record-kind c-string
   stream-record-kind => "~" | "@" | "&"

The reason it would be helpful to modify the grammar in this way is that
it leads to a more elegant form when trying to build an intermediate
representation. At the parse level of 'stream-record' or 'async-record'
you have all of the information necessary in order to populate a
structure with data. Otherwise, the information is a few levels down
stream.

Does this sound reasonable? and again, should I post a new grammar with
the new lines moved to the 'output' production and with the changes I
have above?

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Bob Rossi <bob@brasko.net> wrote:
> Does this sound reasonable? and again, should I post a new grammar with
> the new lines moved to the 'output' production and with the changes I
> have above?

I'd love to see your grammar posted.

Re: GDB/MI Output Syntax

[Bob Rossi]

On Wed, Aug 25, 2004 at 11:56:38AM -0400, Michael Chastain wrote:
> Bob Rossi <bob@brasko.net> wrote:
> > Does this sound reasonable? and again, should I post a new grammar with
> > the new lines moved to the 'output' production and with the changes I
> > have above?
> 
> I'd love to see your grammar posted.

OK, here is what I have so far. It's pretty simple. Besides from
changing the grammar on the documentation to a lower level BNF form I
have made 3 modifications.

   1. move the 'nl' up as high as it can go.
   2. reduce the async_record
   3. reduce the stream_record

so far, it seems to parse everything I throw at it. However, I haven't
tested it to much because I am building an intermediate representation.
This is what I'll use from the front end.

It would probably be helpful to post the grammar the way I have it, or
some slight modification of it instead or on top of what's on the
website now.

output                  -> oob_record_list opt_result_record "(gdb)" nl
oob_record_list         -> oob_record_list oob_record nl | epsilon
opt_result_record       -> result_record nl | epsilon
result_record           -> opt_token "^" result_class result_list_prime
oob_record              -> async_record | stream_record
async_record            -> opt_token async_record_class async_output
async_record_class      -> "*" | "+" | "="
async_output            -> async_class result_list_prime
result_class            -> "done" | "running" | "connected" | "error" | "exit"
async_class             -> "stopped"
result_list_prime       -> result_list | epsilon
result_list             -> result_list "," result | "," result
result                  -> variable "=" value
variable                -> string
value_list_prime        -> value_list | epsilon
value_list              -> value_list "," value | "," value
value                   -> c_string | tuple | list
tuple                   -> "{}" | "{" result result_list_prime "}"
list                    -> "[]" | "[" value value_list_prime "]" | "[" result result_list_prime "]"
stream_record           -> stream_record_class c_string
stream_record_class     -> "~" | "@" | "&"
nl                      -> CR | CR_LF
opt_token               -> token | epsilon
token                   -> any sequence of digits.

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Bob Rossi <bob@brasko.net> wrote:
> so far, it seems to parse everything I throw at it. However, I haven't
> tested it to much because I am building an intermediate representation.
> This is what I'll use from the front end.

How can we hook this up with the gdb test suite?

I've got a corpus of gdb.log files.  Someone could write some Perl
script to pick out pieces and invoke your parser as an external program.
It might help to add a few more rules at the top:

  session                 -> input_output_pair_list
  input_output_pair_list  -> epsilon | input_output_pair_list input output
  input                   -> ...

The sticky part is that dejagnu mixes its own output into this.
Ick.

Getting into the grammar itself:

Comma separators and lists are kludgy.  In these rules:

  result_record      -> opt_token "^" result_class result_list_prime
  result_list_prime  -> result_list | epsilon
  result_list        -> result_list "," result | "," result

The actual gdb output for a result_record could be either:

  105^done
  103^done,BreakPointTable={...}

It looks a little weird to me to parse the first comma as part
of result_list_prime.  How about:

  result_record  -> opt_token "^" result_class
  result_record  -> opt_token "^" result_class "," result_list
  result_list    -> result | result_list "," result

That simplifies tuple and list as well:

  tuple  -> "{}" | "{" result_list "}"
  list   -> "[]" | "[" value_list "]" | "[ result_list ]"

That simplifies the rules also, because they won't need any special code
to construct a list for: "[" result result_list "]" .

This also gets rid of the foo_prime constructions, which can cause
trouble.  The original oob_record_list_prime caused the original
shift/reduce conflict, because the parser had to decide whether to
reduce an epsilon to oob_record_list_prime or keep shifting and reduce
later to the non-epsilon form of the oob_record_list.

Style point: there is a lot of:

  foo_list -> foo_list foo | epsilon
  bar_list -> bar_list bar | bar

I think this is more readable:

  foo_list -> epsilon | foo_list foo
  bar_list -> bar | bar_list bar

Another nit: how is the grammar even working with:

  nl -> CR | CR_LF

Doesn't this have to be:

  nl -> LF | CR | CR LF

Or is the lexer quietly defining CR_LF to include "\n"?

For coding purposes it would be more efficient to make NL
a single token and have the lexer recognize all three forms.

For doco purposes it might be better to explicitly make nl
a non-terminal and show the LF, CR, CR LF terminals.

Either way is okay, but I'd like to have one or the other:
either have the lexer do all the work, or have the lexer be
stupid simple and have the grammar do the work.

Michael

Re: GDB/MI Output Syntax

[Bob Rossi]

On Thu, Aug 26, 2004 at 10:01:39AM -0400, Michael Chastain wrote:
> Bob Rossi <bob@brasko.net> wrote:
> > so far, it seems to parse everything I throw at it. However, I haven't
> > tested it to much because I am building an intermediate representation.
> > This is what I'll use from the front end.
> 
> How can we hook this up with the gdb test suite?
> 
> I've got a corpus of gdb.log files.  Someone could write some Perl
> script to pick out pieces and invoke your parser as an external program.
> It might help to add a few more rules at the top:
> 
>   session                 -> input_output_pair_list
>   input_output_pair_list  -> epsilon | input_output_pair_list input output
>   input                   -> ...
> 
> The sticky part is that dejagnu mixes its own output into this.
> Ick.

Hmmm, this are some ideas, what do you think?
   1. Run gdb through tee and pipe only GDB's stdout to a place where we
      can validate it's output.
   2. Have GDB output it's stdout to 2 places somehow, similar to the
   idea above (except maybe a new GDB logging feature), so that the output 
   can be parsed.
   3. Create a new process, that invokes GDB, validates the output, and
   output's exactly what GDB used to output.
   4. Somehow parse the output of GDB, through tcl like you are
   suggesting?

Here is one consideration, if we write a parser that validates the
GDB/MI Input Syntax in the same manner that we are doing for the Output
Syntax then we could make a combined grammar that is possible of parsing
an entire GDB/MI session. We could also add an adhoc console command
parser, which would parse the console commands if the user typed any.

This would be good for a few reasons. 
   1. GDB could have it's Input parsed with the library. Then, the MI
   could work off of the intermediate language, which would probably be
   nice.
   2. Entire GDB sessions could be validated. 
   3. GDB could yell at clients that are sending invalid commands.

   BTW, I think that currently, each MI command does a lot of the parsing
   for itself, this is probably a bad idea and prone to bugs.
      
I would be reasonably happy to write this Input parser since I am
already writing the Output parser. The in memory representation is very
low level and could easily be used by GDB.

> Getting into the grammar itself:
> 
> Comma separators and lists are kludgy.  In these rules:
> 
>   result_record      -> opt_token "^" result_class result_list_prime
>   result_list_prime  -> result_list | epsilon
>   result_list        -> result_list "," result | "," result
> 
> The actual gdb output for a result_record could be either:
> 
>   105^done
>   103^done,BreakPointTable={...}
> 
> It looks a little weird to me to parse the first comma as part
> of result_list_prime.  How about:
> 
>   result_record  -> opt_token "^" result_class
>   result_record  -> opt_token "^" result_class "," result_list
>   result_list    -> result | result_list "," result

Yes, Yes, this makes much sense.

> That simplifies tuple and list as well:
> 
>   tuple  -> "{}" | "{" result_list "}"
>   list   -> "[]" | "[" value_list "]" | "[ result_list ]"

I like this very much.

> Style point: there is a lot of:
> 
>   foo_list -> foo_list foo | epsilon
>   bar_list -> bar_list bar | bar
> 
> I think this is more readable:
> 
>   foo_list -> epsilon | foo_list foo
>   bar_list -> bar | bar_list bar

This is fine with me either way, so we'll do it your way.

> Another nit: how is the grammar even working with:
> 
>   nl -> CR | CR_LF
> 
> Doesn't this have to be:
> 
>   nl -> LF | CR | CR LF

I don't use this rule in the grammar. I have the lexer return NEWLINE.

> 
> Or is the lexer quietly defining CR_LF to include "\n"?
> 
> For coding purposes it would be more efficient to make NL
> a single token and have the lexer recognize all three forms.
> 
> For doco purposes it might be better to explicitly make nl
> a non-terminal and show the LF, CR, CR LF terminals.
> 
Agreed, this is exactly what I have done.

OK, so here is the new grammar that I have. It parses the few example
that I have, and is reasonable good. It combines many of your ideas with
some new ideas of my own. What do you think?

Any rule that starts with opt_ either goes to epsilon or something else.
Any rule that ends in _list is a list of items.

I find that actually building the syntax tree also helps organize the
commands, and I haven't gotten that far with this version of the
grammar.

opt_output_list         -> epsilon | output_list
output_list             -> output | output_list output
output                  -> opt_oob_record_list opt_result_record "(gdb)" nl
opt_oob_record_list     -> epsilon | opt_oob_record_list oob_record nl
opt_result_record       -> epsilon | result_record nl
result_record           -> opt_token "^" result_class
result_record           -> opt_token "^" result_class "," opt_result_list
oob_record              -> async_record | stream_record
async_record            -> opt_token async_record_class async_output
async_record_class      -> "*" | "+" | "="
async_output            -> async_class "," opt_result_list
result_class            -> "done" | "running" | "connected" | "error" | "exit"
async_class             -> "stopped"
opt_result_list         -> epsilon | result_list
result_list             -> result | result_list "," result
result                  -> variable "=" value
variable                -> string
value_list              -> value | value_list "," value
value                   -> c_string | tuple | list
tuple                   -> "{}" | "{" result_list "}"
list                    -> "[]" | "[" value_list "]" | "[" result_list "]"
stream_record           -> stream_record_class c_string
stream_record_class     -> "~" | "@" | "&"
nl                      -> CR | LF | CR LF
opt_token               -> epsilon | token
token                   -> any sequence of digits.



Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Bob Rossi <bob@brasko.net> wrote:
> Hmmm, this are some ideas, what do you think?
>    1. Run gdb through tee and pipe only GDB's stdout to a place where we
>       can validate it's output.
>    2. Have GDB output it's stdout to 2 places somehow, similar to the
>    idea above (except maybe a new GDB logging feature), so that the output 
>    can be parsed.
>    3. Create a new process, that invokes GDB, validates the output, and
>    output's exactly what GDB used to output.
>    4. Somehow parse the output of GDB, through tcl like you are
>    suggesting?

I think we won't get to a solution today.

It's really a problem at the dejagnu/testsuite level.  We've got this
output stream, but it's mixed in with "PASS: foo" and "Executing on
host: bar" in gdb.log.

At the test suite level, the TCL code is holding the output and
could do something with it.  But it's nontrivial for TCL code to
interface with a C library.  (I'm not about to rebuild the 'expect'
binary).

Also this might relate to "separating gdb output from inferior output".

bob> I don't use this rule in the grammar. I have the lexer return NEWLINE.

That's okay, too.  Either way is fine with me.

I'd like your grammar and the reference grammar to be identical,
so that we know that the reference grammar actually works.

Grammar comments:

===

Can you write the terminal symbols in ALL CAPS;
that's the usual style for bison grammars.

I think the terminals are:

  STRING C_STRING CR LF TOKEN
  "(gdb)" "^" "*" "+" "=" "," "~" "@" "&"
  "done" "running" "connected" "error" "exit"
  "stopped"

The literal strings are okay as is, but I'd really like
symbolic terminals capitalized.

===

Hmmm, in my gdb.log, there is a space in "(gdb) ^M", but in
the gdb.texinfo grammar and the new grammar, there is no space.

===

In the result_record rules:

  result_record -> opt_token "^" result_class
  result_record -> opt_token "^" result_class "," opt_result_list

In the second form, if there is a ",", there must be at least
one result.  So the last symbol should be result_list rather
than opt_result_list.

===

Same deal with the async_output rule:

  async_output -> async_class "," opt_result_list

This should be like result_record.  If opt_result_list is empty
then there is no "," .  Example:

  403*stopped^M
  (gdb) ^M

In fact async_output doesn't add much value to the grammar.
How about just:

  async_record -> opt_token async_record_class async_class
  async_record -> opt_token async_record_class async_class "," result_list

Re: GDB/MI Output Syntax

[Keith Seitz]

On Thu, 2004-08-26 at 13:44, Michael Chastain wrote:
> At the test suite level, the TCL code is holding the output and
> could do something with it.  But it's nontrivial for TCL code to
> interface with a C library.  (I'm not about to rebuild the 'expect'
> binary).

I've got three letters (okay, words) for you: TEA.

Interfacing Tcl and C is TRIVIAL.

Keith

Re: GDB/MI Output Syntax

[Michael Chastain]

Keith Seitz <keiths@redhat.com> wrote:
> I've got three letters (okay, words) for you: TEA.
>
> Interfacing Tcl and C is TRIVIAL.
>
> Keith

I left out a part: it's nontrivial for TCL code that's running in
a TCL interpreter that's already linked into 'expect', a program
that we did not build, to interface with a C library.

lib/gdb.exp can load the TEA sample shared object without error, but the
new commands are not available.

  ERROR: (DejaGnu) proc "sha -string hello" does not exist.

When I run the same 'expect' binary standalone interactively, it can
load the TEA sample library and the new commands are available.  Go
figure.

All the parts are there, but we're not currently using them and they
fail the smoke test.  Multiply by a dozen hosts and that's what I call
nontrivial.

Re: GDB/MI Output Syntax

[Bob Rossi]

On Thu, Aug 26, 2004 at 04:44:06PM -0400, Michael Chastain wrote:
> Grammar comments:
> 
> ===
> 
> Can you write the terminal symbols in ALL CAPS;
> that's the usual style for bison grammars.
> 
> I think the terminals are:
> 
>   STRING C_STRING CR LF TOKEN
>   "(gdb)" "^" "*" "+" "=" "," "~" "@" "&"
>   "done" "running" "connected" "error" "exit"
>   "stopped"
> 
> The literal strings are okay as is, but I'd really like
> symbolic terminals capitalized.

Done. 

I would prefer remove all string literals. Or make rules for them.
gdb -> "(gdb)"
done -> "done"

Any thoughts?
> 
> ===
> 
> Hmmm, in my gdb.log, there is a space in "(gdb) ^M", but in
> the gdb.texinfo grammar and the new grammar, there is no space.

My lexer ignores whitespace,
[ \t\v\f]               {}
is this bad?

we could note in the doco that the grammar doesn't care about
whitespace. Or, we could fix GDB in that case?
> ===
> 
> In the result_record rules:
> 
>   result_record -> opt_token "^" result_class
>   result_record -> opt_token "^" result_class "," opt_result_list
> 
> In the second form, if there is a ",", there must be at least
> one result.  So the last symbol should be result_list rather
> than opt_result_list.
> 

Done, and this is great because it remove the dependency of
opt_result_list. It is no longer needed.
> ===
> 
> Same deal with the async_output rule:
> 
>   async_output -> async_class "," opt_result_list
> 
> This should be like result_record.  If opt_result_list is empty
> then there is no "," .  Example:
> 
>   403*stopped^M
>   (gdb) ^M
> 
> In fact async_output doesn't add much value to the grammar.
> How about just:
> 
>   async_record -> opt_token async_record_class async_class
>   async_record -> opt_token async_record_class async_class "," result_list

Very very good. Done.

Personally, this grammar seems much easier for me to understand,
however, I will look at it more to see if it can be reduced or
simplified anymore.

Here is an update that still seems to work with bison. Andrew, do you
like the idea of starting with a grammar like this, and then showing it
in the style that's on the webpage? Or what do you have in mind?

opt_output_list         -> epsilon | output_list
output_list             -> output | output_list output
output                  -> opt_oob_record_list opt_result_record "(gdb)" nl
opt_oob_record_list     -> epsilon | opt_oob_record_list oob_record nl
opt_result_record       -> epsilon | result_record nl
result_record           -> opt_token "^" result_class
result_record           -> opt_token "^" result_class "," result_list
oob_record              -> async_record | stream_record
async_record            -> opt_token async_record_class async_class
async_record            -> opt_token async_record_class async_class "," result_list
async_record_class      -> "*" | "+" | "="
result_class            -> "done" | "running" | "connected" | "error" | "exit"
async_class             -> "stopped"
result_list             -> result | result_list "," result
result                  -> variable "=" value
variable                -> STRING
value_list              -> value | value_list "," value
value                   -> C_STRING | tuple | list
tuple                   -> "{}" | "{" result_list "}"
list                    -> "[]" | "[" value_list "]" | "[" result_list "]"
stream_record           -> stream_record_class C_STRING
stream_record_class     -> "~" | "@" | "&"
nl                      -> CR | LF | CR LF
opt_token               -> epsilon | TOKEN

BTW, I don't have any string literals in my real grammar. I return
tokens from the parser for each of this. So, I have 

"(gdb)" -> GDB
token   -> INTEGER_LITERAL
CR | LF | CR LF -> NEWLINE

is this pertinant?

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Bob Rossi <bob@brasko.net> wrote:
> I would prefer remove all string literals. Or make rules for them.
> gdb -> "(gdb)"
> done -> "done"

Perhaps:

  GDB  -> "(gdb)"
  DONE -> "done"

and say that the lexer is responsible for recognizing these and
returning those tokens.

> My lexer ignores whitespace,
> [ \t\v\f]               {}
> is this bad?

It surprised me because newlines are at the parse level, so I wasn't
expecting to find white space eaten at the lexer level.  With your grammar,
new line in the wrong place is a syntax error, but tab in the wrong place
is not.

Actually it's probably fine.

If we really want to spec the language, the doco has to list the
terminals that come out of the lexer and describe some of the
lexer's behavior, like eating white space, and what characters are
quoted in a C_STRING ("just like C" ... easy answer).

Speaking of nailing things down ... your grammar doesn't define epsilon!

  epsilon -> 

Re: GDB/MI Output Syntax

[Andrew Cagney]

> Hi,
> 
> Along with the newline changes, there are 2 other changes that I propose
> to the grammar. They are,
> 
> async-record   ==> exec-async-output | status-async-output | notify-async-output
> exec-async-output    ==> [ token ] "*" async-output
> status-async-output  ==> [ token ] "+" async-output
> notify-async-output  ==> [ token ] "=" async-output]

FYI, these were done this way so that the complete structure and intent 
was clear (rather than worry about language issues).  As you note a 
rewrite gives an ll(1) grammar; it also looses that clarity.

> to 
> 
>    async-record ==> [token] async-record-kind async-output
>    async-record-kind ==> "*" | "+" | "="
> 
> however, if you really would like to keep the *-async-output rules, we
> could do
> 
>    async-record ==> [token] async-record-kind async-output
>    async-record-kind ==> exec-async-output | status-async-output | notify-async-output
>    exec-async-output    ==> "*"
>    status-async-output  ==> "+"
>    notify-async-output  ==> "=" 

I can see something like (better names):

	async-record ==> [token] async-output
	async-output ==> exec-async-output | ...
	exec-async-output ==> "*" async-result

	async-result ==> async-class ( "," @var{result} )* @var{nl}

> The second change is identical but refers to the rules of
>    stream-record ==> console-stream-output | target-stream-output | log-stream-output
>    console-stream-output ==> "~" c-string
>    target-stream-output ==> "@" c-string
>    log-stream-output ==> "&" c-string
> 
> to
>    stream-record => stream-record-kind c-string
>    stream-record-kind => "~" | "@" | "&"
> 
> The reason it would be helpful to modify the grammar in this way is that
> it leads to a more elegant form when trying to build an intermediate
> representation. At the parse level of 'stream-record' or 'async-record'
> you have all of the information necessary in order to populate a
> structure with data. Otherwise, the information is a few levels down
> stream.

I don't follow.

Andrew

Re: GDB/MI Output Syntax

[Bob Rossi]

On Thu, Aug 26, 2004 at 05:12:01PM -0400, Andrew Cagney wrote:
> >Hi,
> >
> >Along with the newline changes, there are 2 other changes that I propose
> >to the grammar. They are,
> >
> >async-record   ==> exec-async-output | status-async-output | 
> >notify-async-output
> >exec-async-output    ==> [ token ] "*" async-output
> >status-async-output  ==> [ token ] "+" async-output
> >notify-async-output  ==> [ token ] "=" async-output]
> 
> FYI, these were done this way so that the complete structure and intent 
> was clear (rather than worry about language issues).  As you note a 
> rewrite gives an ll(1) grammar; it also looses that clarity.

Yes, clarity is very important. Honestly, Michael helped me get the
original grammar to be ll(1). The only problem with the original grammar
that I have found so far is that it is missing a 'nl' on one line.

I only propose rearranging the gramar a little, written in the way
that it is in the doco. Also, I propose that the grammar Michael and I
are working also get posted, and say that it is a dirivative of the
original. Both ways of posting the data are helpful to front end
developers.

The only change to the original grammar I am interested in is to match
the names/rules to be similar to the grammar that Michael and I are
generated (loosing productions we have eliminated, adding productions
that we added, ... ). Either way, I think it would be helpful to say, 
here is the grammar, and here is the bison input to the grammar.

In the end, it's up to you guys.

> >The second change is identical but refers to the rules of
> >   stream-record ==> console-stream-output | target-stream-output | 
> >   log-stream-output
> >   console-stream-output ==> "~" c-string
> >   target-stream-output ==> "@" c-string
> >   log-stream-output ==> "&" c-string
> >
> >to
> >   stream-record => stream-record-kind c-string
> >   stream-record-kind => "~" | "@" | "&"
> >
> >The reason it would be helpful to modify the grammar in this way is that
> >it leads to a more elegant form when trying to build an intermediate
> >representation. At the parse level of 'stream-record' or 'async-record'
> >you have all of the information necessary in order to populate a
> >structure with data. Otherwise, the information is a few levels down
> >stream.
> 
> I don't follow.

This may not interest you at all. However, with the rules,
   > >   stream-record ==> console-stream-output | target-stream-output | 
   > >   log-stream-output
   > >   console-stream-output ==> "~" c-string
   > >   target-stream-output ==> "@" c-string
   > >   log-stream-output ==> "&" c-string

when bison get's to stream-record, it doesn't have available to it the 
c-string data. So, I can't do 
   stream_record: stream_record_class CSTRING {
       $$ = malloc ( sizeof ( struct stream_record ) );
       $$->stream_record = $1;
       $$->cstring = strdup ( gdbmi_text );
   };

With the rules,

   > >   stream-record => stream-record-kind c-string
   > >   stream-record-kind => "~" | "@" | "&"

I can do the above code, which is nice.

I understand there is other ways to do the same thing, however, this
seems elegant to me.

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Just a nit: LL(1) and LALR(1) are different things, and bison/yacc
accept LALR(1) grammars.

Bob Rossi <bob@brasko.net> wrote:
> Also, I propose that the grammar Michael and I are working also get
> posted, and say that it is a dirivative of the original. Both ways of
> posting the data are helpful to front end developers.

Again I say, I want the printed grammar in gdb.texinfo to be as close as
possible to something that a front-end writer can stick into bison and
*use*.  Because if we have two grammars, one printed one that does not
compile and one machine-readaable one that does compile, and they are
different, guess what's going to happen.

Re: GDB/MI Output Syntax

[Eli Zaretskii]

> Date: Thu, 26 Aug 2004 18:46:20 -0400
> From: Michael Chastain <mec.gnu@mindspring.com>
> Cc: gdb@sources.redhat.com
> 
> Again I say, I want the printed grammar in gdb.texinfo to be as close as
> possible to something that a front-end writer can stick into bison and
> *use*.

The grammar should be indeed in the manual, abut there's nothing wrong
to have there several grammars (e.g., LALR(1) and LL(1)).  If that
gets too long, we can always make an appendix and put them there.

Note that the current ``grammar'' in the MI documentation, at least as
far as I'm concerned, is meant to be an informal introduction of the
MI syntax, not a rigurous grammar description.

Re: GDB/MI Output Syntax

[Michael Chastain]

Well, the difference betwen:

  # more nonterminals
  async_record        : exec_async_output | status_async_output | notify_async_output ;
  exec_async_output   : opt_token "*" async_output ;
  status_async_output : opt_token "+" async_output ;
  notify_async_output : opt_token "=" async_output ;

And:

  # less nonterminals
  async_record      : opt_token async_record_kind async_output
  async_record_kind : "*" | "+" | "="

... that difference is not going to make-or-break the necessary
LALR(1) property.  The place to watch out for that is with
rules that can be reduced with 0 input tokens.  That gets ambiguous
real fast.

My esthetics say to combine common things, Andrew's esthetics
say to be more concrete names for things.  It's an esthetic tradeoff.
Andrew has more say as the MI maintainer.

exec_async_output is actually a pretty good nonterminal.  It's
async_output that really bugs me.  async_output is not a complete
object, it's a sub-part that doesn't really need to be there.
Just substitute in its definition.

> 	async-result ==> async-class ( "," @var{result} )* @var{nl}

I think the "*" stuff has to go, though.  bison and yacc don't process
it.  Which means that as soon as someone tries to write a bison grammar,
they have to write something different from our doco.  So every front
end will have a slightly different parser and that will be to GDB's
sorrow.  Eventurally GDB will do something that's in the printed
grammar, and some front end will tip over because it's not accepted by
their bison grammar.

Also, some front-end writers will walk into the same sand-trap that
Bob did when they are translating (a b)* into bison language,
and mistakenly write a non-LALR(1) grammar.

The closer our doco is to a real bison grammar that really compiles,
the less grief.

bob> The reason it would be helpful to modify the grammar in this way is that
bob> it leads to a more elegant form when trying to build an intermediate
bob> representation. At the parse level of 'stream-record' or 'async-record'
bob> you have all of the information necessary in order to populate a
bob> structure with data. Otherwise, the information is a few levels down
bob> stream.

ac> I don't follow.

Every time a yacc/bison rule is reduced, the code associated with that
rule has to make another data structure that copies/refers-to all the
data from each of the components.  So a flatter rule tree means
the C code looks like "a->b->data" instead of "a->b->c->d->data".

Re: GDB/MI Output Syntax

[Paul Schlie]

> Bob Rossi wrote:
>> Michael Chastain wrote:
>> ...
>> It would be much better to use TCL data structures to parse MI rather
>> than regular expressions.  I had a great experience getting away from
>> regular expressions with cp_test_ptype_class.
>> 
>> It's still a dozen host arches (actually, a dozen build arches,
>> TCL runs on build machine).  But we're not debugging a target program
>> with shared libraries, we're just using one as a host.
>> ...
>
> Hey, has anything ever evolved out of this?
> 
> Here is my road map for developing an MI parser for CGDB.
>   
>   1. Create a grammar that is easily translated into LR(1)
>   2. Generate the parser with flex and bison
>   3. Have the parser test the output of the GDB MI testsuite
>      (Don't know how to do this)
>   4. Have the parser verify the semantics of GDB's output.
> ...

Or how about a basic scheme (<keyword> <expression> ...) syntax, can't get
much simpler or more flexible than that, not to mention it's fairly straight
forward easy to read/parse/extend and may realativly easily accomplished by
imbedding an open-source basic scheme interpreter, vs re-inventing the
wheel; nearly eliminating the necessity for steps 1, 2; and longer term
could easily eliminate gdb's present less than flexible command interpreter,
as there's truly no good reason for the two to be distinct. (Not a new
notion; but possibly timely and arguably far more productive than developing
yet another yet another syntax/language/intepreter/etc.)


 

Re: GDB/MI Output Syntax

[Kip Macy]

Similar alternatives have been discussed previously and decided against.
Check the archives for details.

			-Kip

> Or how about a basic scheme (<keyword> <expression> ...) syntax, can't get
> much simpler or more flexible than that, not to mention it's fairly straight
> forward easy to read/parse/extend and may realativly easily accomplished by
> imbedding an open-source basic scheme interpreter, vs re-inventing the
> wheel; nearly eliminating the necessity for steps 1, 2; and longer term
> could easily eliminate gdb's present less than flexible command interpreter,
> as there's truly no good reason for the two to be distinct. (Not a new
> notion; but possibly timely and arguably far more productive than developing
> yet another yet another syntax/language/intepreter/etc.)
>
>
>
>
>
>

Re: GDB/MI Output Syntax

[Paul Schlie]

Well went back 2 years on the gdb list and found nothing indicating that it
was ever "decided against", but do acknowledge that it's academic unless
energy is committed to it or any other implementation.

> From: Kip Macy <kmacy@fsmware.com>
> Similar alternatives have been discussed previously and decided against.
> Check the archives for details.
> 
> -Kip
> 
>> Or how about a basic scheme (<keyword> <expression> ...) syntax, can't get
>> much simpler or more flexible than that, not to mention it's fairly straight
>> forward easy to read/parse/extend and may realativly easily accomplished by
>> imbedding an open-source basic scheme interpreter, vs re-inventing the
>> wheel; nearly eliminating the necessity for steps 1, 2; and longer term
>> could easily eliminate gdb's present less than flexible command interpreter,
>> as there's truly no good reason for the two to be distinct. (Not a new
>> notion; but possibly timely and arguably far more productive than developing
>> yet another yet another syntax/language/intepreter/etc.)

Re: GDB/MI Output Syntax

[Bob Rossi]

On Wed, Jan 05, 2005 at 07:27:48PM -0500, Paul Schlie wrote:
> > Bob Rossi wrote:
> >> Michael Chastain wrote:
> >> ...
> >> It would be much better to use TCL data structures to parse MI rather
> >> than regular expressions.  I had a great experience getting away from
> >> regular expressions with cp_test_ptype_class.
> >> 
> >> It's still a dozen host arches (actually, a dozen build arches,
> >> TCL runs on build machine).  But we're not debugging a target program
> >> with shared libraries, we're just using one as a host.
> >> ...
> >
> > Hey, has anything ever evolved out of this?
> > 
> > Here is my road map for developing an MI parser for CGDB.
> >   
> >   1. Create a grammar that is easily translated into LR(1)
> >   2. Generate the parser with flex and bison
> >   3. Have the parser test the output of the GDB MI testsuite
> >      (Don't know how to do this)
> >   4. Have the parser verify the semantics of GDB's output.
> > ...
> 
> Or how about a basic scheme (<keyword> <expression> ...) syntax, can't get
> much simpler or more flexible than that, not to mention it's fairly straight
> forward easy to read/parse/extend and may realativly easily accomplished by
> imbedding an open-source basic scheme interpreter, vs re-inventing the
> wheel; nearly eliminating the necessity for steps 1, 2; and longer term
> could easily eliminate gdb's present less than flexible command interpreter,
> as there's truly no good reason for the two to be distinct. (Not a new
> notion; but possibly timely and arguably far more productive than developing
> yet another yet another syntax/language/intepreter/etc.)

I understand your point here. Here's the state of affair's as far as I
see it

I want to write a curses based front end to GDB. I looked at the
alternatives for communicating with GDB and found that MI was the
documented way to do this. So, with that in mind I started to look for a
parser and grammar and none were found. This frustrated me cause i
didn't want to write one, so I suggested GDB output XML. Everyone
hated this idea. So, I decided to write the parser that everyone else
could use from here on out, so no one would have to reinvent the wheel.

I've already put some effort into the MI and this is the point that I'm
currently at. The goal I'm shooting for is to use this parser to really
validate the output of the MI. Better than is being done know. Make sure
the syntax and the semantics are perfect.

Basically, I'm not looking into writing a whole new frame work for
communicating with GDB, I just want to improve what's already there.

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Paul Schlie]

Understood.
 
> I understand your point here. Here's the state of affair's as far as I
> see it
> 
> I want to write a curses based front end to GDB. I looked at the
> alternatives for communicating with GDB and found that MI was the
> documented way to do this. So, with that in mind I started to look for a
> parser and grammar and none were found. This frustrated me cause i
> didn't want to write one, so I suggested GDB output XML. Everyone
> hated this idea. So, I decided to write the parser that everyone else
> could use from here on out, so no one would have to reinvent the wheel.
> 
> I've already put some effort into the MI and this is the point that I'm
> currently at. The goal I'm shooting for is to use this parser to really
> validate the output of the MI. Better than is being done know. Make sure
> the syntax and the semantics are perfect.
> 
> Basically, I'm not looking into writing a whole new frame work for
> communicating with GDB, I just want to improve what's already there.
> 
> Thanks,
> Bob Rossi

Re: GDB/MI Output Syntax

[Jim Ingham]

Nah, this works just fine.

The problem is that the sample extension actually adds a command called  
"sha1".  In interactive mode, tcl resolves shortened commands to the  
full name if the resolution is unique.  So if you do:

$ tclsh
% info commands sha*
% package require Tclsha1
0.5
% info commands sha*
sha1
% sha -string hello
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d
% sha1 -string hello
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d

But in script mode, Tcl doesn't complete commands for you (to better  
catch errors in scripts).  So that's why you were getting the error  
when you tried to use the "sha" command in DejaGnu.  If you used the  
sha1 command, it would work:

inghji:tot-shlib-experiment/build/gdb > make check RUNTESTFLAGS="--D1"
Nothing to be done for all...
rootme=`pwd`; export rootme; \
srcdir=../../../gdb/src/gdb/testsuite ; export srcdir ; \
EXPECT=`if [ -f ${rootme}/../../expect/expect ] ; then echo  
${rootme}/../../expect/expect ; else echo expect ; fi` ; export EXPECT  
; \
EXEEXT= ; export EXEEXT ; \
          
RPATH_ENVVAR=$rootme/../../expect:$rootme/../../libstdc++: 
$rootme/../../tk/unix:$rootme/../../tcl/unix:$rootme/../../bfd: 
$rootme/../../opcodes:$RPATH_ENVVAR; \
export RPATH_ENVVAR; \
if [ -f ${rootme}/../../expect/expect ] ; then  \
   TCL_LIBRARY=${srcdir}/../../tcl/library ; \
   export TCL_LIBRARY ; fi ; \
/Volumes/ThePlayground/Users/jingham/Work/gdb-sources/tot-shlib- 
experiment/gdb/src/gdb/../dejagnu/runtest --D1
1: set frame_version    1.4.2.x

dbg1.0> package require Tclsha1
0.5
dbg1.1> sha1 -string hello
aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d

I actually think that building Bob's parser into a Tcl C extension &  
using that with the DejaGnu testsuite would be brilliant.  Not only  
would it ensure that we keep the output correctly parseable, but it  
would mean that we could solve the problem the current testsuite has  
that when you add a field to some mi command (which should be a  
backwards compatible operation) you have to go fix ALL the test cases.   
If you used the parser to read the MI output into a Tcl array, then the  
tests could just query the array for the elements they wanted to test,  
and not be bothered by elements that they were not concerned with  
testing.

Jim


On Aug 27, 2004, at 9:04 AM, gdb-digest-help@sources.redhat.com wrote:

> From: Michael Chastain <mec.gnu@mindspring.com>
> Date: August 26, 2004 3:16:43 PM PDT
> To: keiths@redhat.com
> Cc: gdb@sources.redhat.com, bob@brasko.net
> Subject: Re: GDB/MI Output Syntax
>
>
> Keith Seitz <keiths@redhat.com> wrote:
>> I've got three letters (okay, words) for you: TEA.
>>
>> Interfacing Tcl and C is TRIVIAL.
>>
>> Keith
>
> I left out a part: it's nontrivial for TCL code that's running in
> a TCL interpreter that's already linked into 'expect', a program
> that we did not build, to interface with a C library.
>
> lib/gdb.exp can load the TEA sample shared object without error, but  
> the
> new commands are not available.
>
>   ERROR: (DejaGnu) proc "sha -string hello" does not exist.
>
> When I run the same 'expect' binary standalone interactively, it can
> load the TEA sample library and the new commands are available.  Go
> figure.
>
> All the parts are there, but we're not currently using them and they
> fail the smoke test.  Multiply by a dozen hosts and that's what I call
> nontrivial.
>
>

Re: GDB/MI Output Syntax

[Michael Chastain]

Jim Ingham <jingham@apple.com> wrote:
> The problem is that the sample extension actually adds a command called  
> "sha1".  In interactive mode, tcl resolves shortened commands to the  
> full name if the resolution is unique.

*bonk* *bonk* *bonk*

That's the sound of me banging my head against the wall.  Stupid me!

> I actually think that building Bob's parser into a Tcl C extension &  
> using that with the DejaGnu testsuite would be brilliant.

New vistas are opening up to me!

It would be much better to use TCL data structures to parse MI rather
than regular expressions.  I had a great experience getting away from
regular expressions with cp_test_ptype_class.

It's still a dozen host arches (actually, a dozen build arches,
TCL runs on build machine).  But we're not debugging a target program
with shared libraries, we're just using one as a host.

Re: GDB/MI Output Syntax

[Bob Rossi]

On Fri, Aug 27, 2004 at 03:12:36PM -0400, Michael Chastain wrote:
> Jim Ingham <jingham@apple.com> wrote:
> > The problem is that the sample extension actually adds a command called  
> > "sha1".  In interactive mode, tcl resolves shortened commands to the  
> > full name if the resolution is unique.
> 
> *bonk* *bonk* *bonk*
> 
> That's the sound of me banging my head against the wall.  Stupid me!
> 
> > I actually think that building Bob's parser into a Tcl C extension &  
> > using that with the DejaGnu testsuite would be brilliant.
> 
> New vistas are opening up to me!
> 
> It would be much better to use TCL data structures to parse MI rather
> than regular expressions.  I had a great experience getting away from
> regular expressions with cp_test_ptype_class.
> 
> It's still a dozen host arches (actually, a dozen build arches,
> TCL runs on build machine).  But we're not debugging a target program
> with shared libraries, we're just using one as a host.

Hey, has anything ever evolved out of this? 

Here is my road map for developing an MI parser for CGDB.
   
   1. Create a grammar that is easily translated into LR(1)
   2. Generate the parser with flex and bison
   3. Have the parser test the output of the GDB MI testsuite
      (Don't know how to do this)
   4. Have the parser verify the semantics of GDB's output.

I basically believe I'm done with step 1 and 2, besides bugs and other
stuff. After I'm done with step 3, I plan on adding the parser into CGDB. I may
even wait until after step 4.

My questions are, 
   1. will the grammar and bison syntax be good to document so that others can see it?
   2. Can anyone help me put the parser into the testsuite ( It could
   take me forever since I know nothing about TCL, Excpect, ... )
   3. Would a patch like this be acceptable even if it was accomplished?

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Eli Zaretskii]

> Date: Wed, 5 Jan 2005 18:26:57 -0500
> From: Bob Rossi <bob@brasko.net>
> Cc: jingham@apple.com, gdb@sources.redhat.com
> 
>    1. will the grammar and bison syntax be good to document so that others can see it?

I think it's enough to point to the grammar definition file in the MI
docs.  There's no need to do more in the docs; the grammar should
document itself.

>    3. Would a patch like this be acceptable even if it was accomplished?

Why not?

Re: GDB/MI Output Syntax

[Bob Rossi]

On Thu, Jan 06, 2005 at 06:45:33AM +0200, Eli Zaretskii wrote:
> > Date: Wed, 5 Jan 2005 18:26:57 -0500
> > From: Bob Rossi <bob@brasko.net>
> > Cc: jingham@apple.com, gdb@sources.redhat.com
> > 
> >    1. will the grammar and bison syntax be good to document so that others can see it?
> 
> I think it's enough to point to the grammar definition file in the MI
> docs.  There's no need to do more in the docs; the grammar should
> document itself.
> 
> >    3. Would a patch like this be acceptable even if it was accomplished?
> 
> Why not?

It's just that nothing like that is being done now, and I didn't know if
it would be OK.

I can't seem to get a response from Jim or Michael, so I'm trying to
figure this out on my own.

At this point, I don't even know what would invoke this parser.
Would the TCL do it? for instance, would I have to do something like
catch a MI output command in the TCL by grabbing the regular expression
up to the "(gdb)" and then literally pass that string into my parser?

That would en tale modifying all of the .exp files, which would really
stink.

I don't even know were to start. :(
Any help would be appreciated.

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Jim Ingham]

Sorry for not responding, we're really busy here right now.

I don't think there is any way to use your parser that won't involve 
changing the testsuite files.  After all, they are just passing regular 
expressions, and unless you are ALSO willing to write a parser that 
turns the mi testsuite regular expressions into some sensible form (not 
something I would suggest) you are going to have to rewrite these 
tests.

I was more thinking of this as a project that you would use to write 
new tests, and then any time a change in the mi output broke a test 
case, you could rewrite it using the new form - since you are going to 
have to rewrite it anyway...

The way that I was thinking to use the parser, is to write a command 
like: "gdb_mi_parse_output" that would take the mi output, run your 
parser on it, and turn the result into some Tcl data structure. This 
would be a C based command, so we would have to build a little Tcl 
extension to do this, and include it with gdb when we ran the test 
suite.

The mi output is all basically just nested lists, so you could probably 
express it as a nested list pretty easily.  I'd have to think a bit 
about how to design this well.  Anyway, then you could rewrite the test 
suite cases to just pull fields out of the Tcl data structure, and see 
if the values were what was expected.

  Unfortunately, I am definitely not going to have any time to work on 
this in the next little while, however.

Jim

On Jan 6, 2005, at 3:31 PM, Bob Rossi wrote:



> On Thu, Jan 06, 2005 at 06:45:33AM +0200, Eli Zaretskii wrote:
>
>>> Date: Wed, 5 Jan 2005 18:26:57 -0500
>>> From: Bob Rossi <bob@brasko.net>
>>> Cc: jingham@apple.com, gdb@sources.redhat.com
>>>
>>>    1. will the grammar and bison syntax be good to document so that 
>>> others can see it?
>>>
>>
>> I think it's enough to point to the grammar definition file in the MI
>> docs.  There's no need to do more in the docs; the grammar should
>> document itself.
>>
>>
>>>    3. Would a patch like this be acceptable even if it was 
>>> accomplished?
>>>
>>
>> Why not?
>>
>
> It's just that nothing like that is being done now, and I didn't know 
> if
> it would be OK.
>
> I can't seem to get a response from Jim or Michael, so I'm trying to
> figure this out on my own.
>
> At this point, I don't even know what would invoke this parser.
> Would the TCL do it? for instance, would I have to do something like
> catch a MI output command in the TCL by grabbing the regular expression
> up to the "(gdb)" and then literally pass that string into my parser?
>
> That would en tale modifying all of the .exp files, which would really
> stink.
>
> I don't even know were to start. :(
> Any help would be appreciated.
>
> Thanks,
> Bob Rossi
>

Re: GDB/MI Output Syntax

[Bob Rossi]

On Thu, Jan 06, 2005 at 04:36:13PM -0800, Jim Ingham wrote:
> Sorry for not responding, we're really busy here right now.
> 
> I don't think there is any way to use your parser that won't involve 
> changing the testsuite files.  After all, they are just passing regular 
> expressions, and unless you are ALSO willing to write a parser that 
> turns the mi testsuite regular expressions into some sensible form (not 
> something I would suggest) you are going to have to rewrite these 
> tests.

OK, so rewriting the tests is fine. They are not very complicated.

> I was more thinking of this as a project that you would use to write 
> new tests, and then any time a change in the mi output broke a test 
> case, you could rewrite it using the new form - since you are going to 
> have to rewrite it anyway...

That's sounds OK. However, I would like to rewrite a few of them now, so
that I can test that the parser is working.

> The way that I was thinking to use the parser, is to write a command 
> like: "gdb_mi_parse_output" that would take the mi output, run your 
> parser on it, and turn the result into some Tcl data structure. This 
> would be a C based command, so we would have to build a little Tcl 
> extension to do this, and include it with gdb when we ran the test 
> suite.

Here's the deal, I already have the C function that takes an MI output
command, parses it, and returns a tree that represents the command. 

   int gdbmi_parser_parse_string ( 
       gdbmi_parser_ptr parser, /* a gdbmi parser context */
       const char *mi_command,  /* command the parser should parse */
       gdbmi_output_ptr *pt );  /* The parse tree */

If Tcl is like C (struct, enum's, union's) I could easily put the parse
tree declaration in some common file and generate both the C and the Tcl
data structures. This part could be done in just a few days if I could
get some guidance on how Tcl works.

> The mi output is all basically just nested lists, so you could probably 
> express it as a nested list pretty easily.  I'd have to think a bit 
> about how to design this well.  Anyway, then you could rewrite the test 
> suite cases to just pull fields out of the Tcl data structure, and see 
> if the values were what was expected.

This part I could do also. In fact, I need to do essentially this to
make my front end work. It's all the same code. I think taking this step
would do a lot in the sense of standardizing the output of GDB/MI.

>  Unfortunately, I am definitely not going to have any time to work on 
> this in the next little while, however.

If you could help me with some technical details, I'd be happy to do the
rest of the work. Is this possible or are you just to maxed out?

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Russell Shaw]

Bob Rossi wrote:
...
> If Tcl is like C (struct, enum's, union's) I could easily put the parse
> tree declaration in some common file and generate both the C and the Tcl
> data structures. This part could be done in just a few days if I could
> get some guidance on how Tcl works.

TCL really means "command line language".

Everything in TCL is a command line eg: <command> <arg1 <arg2> <arg3> etc.

You can nest commands so that one command can be expanded and executed so
that its output supplies args for its parent, using [] brackets:

<command1> <arg11> [ <command2> <arg21> <arg22> ] <arg12> <arg13>...

TCL is all about string expansion of command lines.

A very comprehensive book: Pactical Programming in Tcl and Tk.
Another book that explains TCL and also Expect (probably of more
immediate use): Exploring Expect. Expect also has a large man page.

C and TCL is like comparing chalk and cheese. TCL is interpreted too.

Re: GDB/MI Output Syntax

[Bob Rossi]

On Fri, Jan 07, 2005 at 02:16:29PM +1100, Russell Shaw wrote:
> Bob Rossi wrote:
> ...
> >If Tcl is like C (struct, enum's, union's) I could easily put the parse
> >tree declaration in some common file and generate both the C and the Tcl
> >data structures. This part could be done in just a few days if I could
> >get some guidance on how Tcl works.
> 
> A very comprehensive book: Pactical Programming in Tcl and Tk.
> Another book that explains TCL and also Expect (probably of more
> immediate use): Exploring Expect. Expect also has a large man page.
> 
> C and TCL is like comparing chalk and cheese. TCL is interpreted too.

Thanks for the info, I looked at "Practical Programming" and it helped me
out a lot!

So, here is my initial design, can I get any feedback?

Basically, I have a parser that I generated using flex/bison. That
parser is capable of parsing an MI output command and putting it into a
syntax tree. This is all done in C.

To get the data into Tcl for the testsuite I had to write a Tcl
extension. The design I chose from the Tcl side is something like this.

Here is part of the parse tree on the C side,
   /**
    * This is the root of a parsed GDB/MI Output command.
    */
   struct gdbmi_output {
       /**
        * Every output command has a list of optional oob_record's.
        * This will be the head of the list, otherwise NULL.
        */
       gdbmi_oob_record_ptr oob_record;

       /**
        * Every output command has an optional result_record.
        * list, otherwise NULL.
        */
       gdbmi_result_record_ptr result_record;

       /** A pointer to the next output */
       gdbmi_output_ptr next;
   };
   
This is the Tcl side which understands how to access this data.

# ------------ 'struct gdbmi_output' documentation

#   This function will parse an MI output command and return a
#   reference to the syntax tree created.
#
# Arguments:
#   mi_output_command is the string representing the MI output
#   command that should be parsed and returned.
#
# Results:
#   The string representing the gdbmi_output record. This in itself
#   is not very useful, but can be operated on by the gdbmi_* commands.
#
proc gdbmi_parse_output_record {mi_output_command}

#   This function takes in a parse tree and get's the gdbmi_oob_record
#   from the parse tree context.
#
# Arguments:
#   gdbmi_output is the gdbmi_output command that came from
#   gdbmi_parse_output_record
#
# Results:
#   A string representing an gdbmi_oob_record record
#
proc gdbmi_get_oob_record {gdbmi_output}

#   This function takes in a parse tree and get's the gdbmi_result_record
#   from the parse tree context.
#
# Arguments:
#   gdbmi_output is the gdbmi_output command that came from
#   gdbmi_parse_output_record
#
# Results:
#   A string representing a gdbmi_result_record record
#
proc gdbmi_get_result_record {gdbmi_output}


From this, I think it would be possible to query the syntax tree in the
Tcl. This would allow the testsuite to verify the syntax and possible the
semantics of the gdb/mi output commands.

This is the approach I'm working towards. Any comments or suggestions?

In the end, it would also be possible to have the parser parse the
output command and then write another tcl command that fills in a data
structure for the specific command under test. This way the scripts do
not have to deal with the syntax tree directly, but an in memory
representation of the MI output command. That would look something like
this,
   set mi_output_command [gdbmi_parse $mi_output_command]
   set mi_break_output [gdbmi_get_breakoutput $mi_output_command]
 
The downside to this approach is creating in memory representations of
the specific MI commands, the upside to this is not making the
testcase's deal with the syntax tree directly. Also, front ends' could
link to the C parser and get the in memory representation.

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Bob Rossi]

Hi,

Any feedback? Either everyone's really busy, no one cares, or everyone
love's this approach. At least we're all agreeing. :)

Since I haven't received any feedback, I'm going to write it this way. I
should have something workable within a week. Although, I don't know how
to load this into expect yet. I'm assuming I'll have to recompile expect
and link in tcl as a shared library, along with my extension.

Thanks,
Bob Rossi

On Tue, Jan 11, 2005 at 02:35:26PM -0500, Bob Rossi wrote:
> On Fri, Jan 07, 2005 at 02:16:29PM +1100, Russell Shaw wrote:
> > Bob Rossi wrote:
> > ...
> > >If Tcl is like C (struct, enum's, union's) I could easily put the parse
> > >tree declaration in some common file and generate both the C and the Tcl
> > >data structures. This part could be done in just a few days if I could
> > >get some guidance on how Tcl works.
> > 
> > A very comprehensive book: Pactical Programming in Tcl and Tk.
> > Another book that explains TCL and also Expect (probably of more
> > immediate use): Exploring Expect. Expect also has a large man page.
> > 
> > C and TCL is like comparing chalk and cheese. TCL is interpreted too.
> 
> Thanks for the info, I looked at "Practical Programming" and it helped me
> out a lot!
> 
> So, here is my initial design, can I get any feedback?
> 
> Basically, I have a parser that I generated using flex/bison. That
> parser is capable of parsing an MI output command and putting it into a
> syntax tree. This is all done in C.
> 
> To get the data into Tcl for the testsuite I had to write a Tcl
> extension. The design I chose from the Tcl side is something like this.
> 
> Here is part of the parse tree on the C side,
>    /**
>     * This is the root of a parsed GDB/MI Output command.
>     */
>    struct gdbmi_output {
>        /**
>         * Every output command has a list of optional oob_record's.
>         * This will be the head of the list, otherwise NULL.
>         */
>        gdbmi_oob_record_ptr oob_record;
> 
>        /**
>         * Every output command has an optional result_record.
>         * list, otherwise NULL.
>         */
>        gdbmi_result_record_ptr result_record;
> 
>        /** A pointer to the next output */
>        gdbmi_output_ptr next;
>    };
>    
> This is the Tcl side which understands how to access this data.
> 
> # ------------ 'struct gdbmi_output' documentation
> 
> #   This function will parse an MI output command and return a
> #   reference to the syntax tree created.
> #
> # Arguments:
> #   mi_output_command is the string representing the MI output
> #   command that should be parsed and returned.
> #
> # Results:
> #   The string representing the gdbmi_output record. This in itself
> #   is not very useful, but can be operated on by the gdbmi_* commands.
> #
> proc gdbmi_parse_output_record {mi_output_command}
> 
> #   This function takes in a parse tree and get's the gdbmi_oob_record
> #   from the parse tree context.
> #
> # Arguments:
> #   gdbmi_output is the gdbmi_output command that came from
> #   gdbmi_parse_output_record
> #
> # Results:
> #   A string representing an gdbmi_oob_record record
> #
> proc gdbmi_get_oob_record {gdbmi_output}
> 
> #   This function takes in a parse tree and get's the gdbmi_result_record
> #   from the parse tree context.
> #
> # Arguments:
> #   gdbmi_output is the gdbmi_output command that came from
> #   gdbmi_parse_output_record
> #
> # Results:
> #   A string representing a gdbmi_result_record record
> #
> proc gdbmi_get_result_record {gdbmi_output}
> 
> 
> >From this, I think it would be possible to query the syntax tree in the
> Tcl. This would allow the testsuite to verify the syntax and possible the
> semantics of the gdb/mi output commands.
> 
> This is the approach I'm working towards. Any comments or suggestions?
> 
> In the end, it would also be possible to have the parser parse the
> output command and then write another tcl command that fills in a data
> structure for the specific command under test. This way the scripts do
> not have to deal with the syntax tree directly, but an in memory
> representation of the MI output command. That would look something like
> this,
>    set mi_output_command [gdbmi_parse $mi_output_command]
>    set mi_break_output [gdbmi_get_breakoutput $mi_output_command]
>  
> The downside to this approach is creating in memory representations of
> the specific MI commands, the upside to this is not making the
> testcase's deal with the syntax tree directly. Also, front ends' could
> link to the C parser and get the in memory representation.
> 
> Thanks,
> Bob Rossi

GDB/MI Output Syntax

[Bob Rossi]

Hi,

I am working with the MI grammer. Thanks to Michael Chastain's lexer
hack I now have a working LL(1) grammer. However, I am a little
disappointed in the fact that I need to do this hack. Shouldn't the MI
grammar be unambiguous?

I understand that everyone in this community hates XML, so I know that
is a closed door, however, is it possible to change the MI grammar so
that it is not ambigious? 

Anyways, now that the parser is written, I fired up GDB and sent 
   ~"GNU gdb 6.1-debian\n"
   ~"Copyright 2004 Free Software Foundation, Inc.\n"
   ~"GDB is free software, covered by the GNU General Public License, and you are\n"
   ~"welcome to change it and/or distribute copies of it under certain conditions.\n"
   ~"Type \"show copying\" to see the conditions.\n"
   ~"There is absolutely no warranty for GDB.  Type \"show warranty\" for details.\n"
   ~"This GDB was configured as \"i386-linux\"."
   ~"\n"
   (gdb) 

to my parser. The problem is, it was a parse error. This simple output
from GDB does not adher to the GDB/MI grammar! I have the very simple
patch to the grammar that makes this example work, however, I am scared
that the more I parse, the less will work. Does anyone know first hand
if this is true? If it is true, I propose that we at least make GDB
adher to the grammer.

Once my parser is finalized, I propose we validate the output of GDB
against it somehow in the testsuite. Maybe by making a gdbmi-validator
program that simply invokes GDB, passes all input to GDB and parse's all
output, along with passing output back to caller. This would make a
trasparent validator of everything GDB/MI says. What does anyone think?

As far as the error's are concerned, can the grammar be updated?
Updating the grammar to match the output of GDB is propably OK, since
most parser's somehow don't care that it doesn't match the grammar as it
is. Why is that? (this is one point where XML shines)

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Bob Rossi <bob@brasko.net> wrote:

  ~"GNU gdb 6.1-debian\n"
  ~"Copyright 2004 Free Software Foundation, Inc.\n"
  ~"GDB is free software, covered by the GNU General Public License, and you are\n"
  ~"welcome to change it and/or distribute copies of it under certain conditions.\n"
  ~"Type \"show copying\" to see the conditions.\n"
  ~"There is absolutely no warranty for GDB.  Type \"show warranty\" for details.\n"
  ~"This GDB was configured as \"i386-linux\"."
  ~"\n"
  (gdb) 

This looks almost okay to me, it's just got a few differences with
newline characters versus the grammar

  output -> (out-of-band-record)* [result-record] "gdb" NL
  (out-of-band-record)* -> stream_record stream_record stream_record stream_record stream_record stream_record stream_record stream_record
  stream_record -> console_stream_output
  console_stream_output -> "~" C_STRING
  [result-record] ->

It is eight stream-records in a row, where each stream-record is a
console-stream-output.  The gotcha is that the grammar as written does
not allow for a NL after a console-stream-output or a stream-record.

bob> Once my parser is finalized, I propose we validate the output of GDB
bob> against it somehow in the testsuite.

That would be awesome.  But how can we hook up a bison grammar to TCL ...
something for me to sleep on.

Although that's not my area of test suite; that is Andrew and Elena's
area.

If we put a machine-compilable grammer into the gdb doco, then front end
authors could file PR's whenever gdb produces output that doesn't meet
the grammar.  Conversely, if the gdb team updates the MI, we would be
forced to update the grammar to keep our test suite passing, and the
updated grammar would tip off the front end writers that new output
forms are coming at them.

bob> Updating the grammar to match the output of GDB is propably OK, since
bob> most parser's somehow don't care that it doesn't match the grammar as it
bob> is. Why is that? (this is one point where XML shines)

My guess -- only a guess -- is that the front ends are just good enough
at parsing to get the job done.

You're a front end writer and I'm not.  You tell me!

Re: GDB/MI Output Syntax

[Bob Rossi]

On Tue, Aug 24, 2004 at 12:15:24AM -0400, Michael Chastain wrote:
> Bob Rossi <bob@brasko.net> wrote:
> 
>   ~"GNU gdb 6.1-debian\n"
>   ~"Copyright 2004 Free Software Foundation, Inc.\n"
>   ~"GDB is free software, covered by the GNU General Public License, and you are\n"
>   ~"welcome to change it and/or distribute copies of it under certain conditions.\n"
>   ~"Type \"show copying\" to see the conditions.\n"
>   ~"There is absolutely no warranty for GDB.  Type \"show warranty\" for details.\n"
>   ~"This GDB was configured as \"i386-linux\"."
>   ~"\n"
>   (gdb) 
> 
> This looks almost okay to me, it's just got a few differences with
> newline characters versus the grammar
> 
>   output -> (out-of-band-record)* [result-record] "gdb" NL
>   (out-of-band-record)* -> stream_record stream_record stream_record stream_record stream_record stream_record stream_record stream_record
>   stream_record -> console_stream_output
>   console_stream_output -> "~" C_STRING
>   [result-record] ->
> 
> It is eight stream-records in a row, where each stream-record is a
> console-stream-output.  The gotcha is that the grammar as written does
> not allow for a NL after a console-stream-output or a stream-record.

Yes, this is the error I saw. I am glad to see that it's reproducable.

> bob> Once my parser is finalized, I propose we validate the output of GDB
> bob> against it somehow in the testsuite.
> 
> That would be awesome.  But how can we hook up a bison grammar to TCL ...
> something for me to sleep on.
> 
> Although that's not my area of test suite; that is Andrew and Elena's
> area.
> 
> If we put a machine-compilable grammer into the gdb doco, then front end
> authors could file PR's whenever gdb produces output that doesn't meet
> the grammar.  Conversely, if the gdb team updates the MI, we would be
> forced to update the grammar to keep our test suite passing, and the
> updated grammar would tip off the front end writers that new output
> forms are coming at them.

Yes, I would very much appreciate this. However, how does this work with
backwards compatibility?

I don't think I would have a problem if GDB changed the grammar as long
as somehow a revision of some sort was put into GDB. So that my front
end could determine what parser to use for this version of GDB.

Thanks,
Bob Rossi

Re: GDB/MI Output Syntax

[Michael Chastain]

Bob Rossi <bob@brasko.net> wrote:
> Yes, I would very much appreciate this. However, how does this work with
> backwards compatibility?

Actually I would suggest:

-- bring the grammar up to a some kind of demo program that can parse
   a gdb session.

-- then step back from the computer, chill out, and think about how to
   use the grammar to create value in the world

Re: GDB/MI Output Syntax

[Andrew Cagney]

> Bob Rossi <bob@brasko.net> wrote:
> 
>   ~"GNU gdb 6.1-debian\n"
>   ~"Copyright 2004 Free Software Foundation, Inc.\n"
>   ~"GDB is free software, covered by the GNU General Public License, and you are\n"
>   ~"welcome to change it and/or distribute copies of it under certain conditions.\n"
>   ~"Type \"show copying\" to see the conditions.\n"
>   ~"There is absolutely no warranty for GDB.  Type \"show warranty\" for details.\n"
>   ~"This GDB was configured as \"i386-linux\"."
>   ~"\n"
>   (gdb) 
> 
> This looks almost okay to me, it's just got a few differences with
> newline characters versus the grammar
> 
>   output -> (out-of-band-record)* [result-record] "gdb" NL
>   (out-of-band-record)* -> stream_record stream_record stream_record stream_record stream_record stream_record stream_record stream_record
>   stream_record -> console_stream_output
>   console_stream_output -> "~" C_STRING
>   [result-record] ->
> 
> It is eight stream-records in a row, where each stream-record is a
> console-stream-output.  The gotcha is that the grammar as written does
> not allow for a NL after a console-stream-output or a stream-record.

That's clearly a bug in the doco.  Any half decent compiler construction 
course will use a new grammer each year (while stopping cheats, also 
means that the students get to flush out a few bugs :-).

-> should all the [missing] @var{nl} be moved to @var{output}, that way 
the're all in the one place.

Rossi writes:
> Yes, this could work. However, was the original grammer meant to be
> LALR(2)?

rewritable into something needing only one level of look-ahead, which I 
see is being done.

Andrew

Re: GDB/MI Output Syntax

[Bob Rossi]

On Tue, Aug 24, 2004 at 02:58:01PM -0400, Andrew Cagney wrote:
> >Bob Rossi <bob@brasko.net> wrote:
> >
> >  ~"GNU gdb 6.1-debian\n"
> >  ~"Copyright 2004 Free Software Foundation, Inc.\n"
> >  ~"GDB is free software, covered by the GNU General Public License, and 
> >  you are\n"
> >  ~"welcome to change it and/or distribute copies of it under certain 
> >  conditions.\n"
> >  ~"Type \"show copying\" to see the conditions.\n"
> >  ~"There is absolutely no warranty for GDB.  Type \"show warranty\" for 
> >  details.\n"
> >  ~"This GDB was configured as \"i386-linux\"."
> >  ~"\n"
> >  (gdb) 
> >
> >This looks almost okay to me, it's just got a few differences with
> >newline characters versus the grammar
> >
> >  output -> (out-of-band-record)* [result-record] "gdb" NL
> >  (out-of-band-record)* -> stream_record stream_record stream_record 
> >  stream_record stream_record stream_record stream_record stream_record
> >  stream_record -> console_stream_output
> >  console_stream_output -> "~" C_STRING
> >  [result-record] ->
> >
> >It is eight stream-records in a row, where each stream-record is a
> >console-stream-output.  The gotcha is that the grammar as written does
> >not allow for a NL after a console-stream-output or a stream-record.
> 
> That's clearly a bug in the doco.  Any half decent compiler construction 
> course will use a new grammer each year (while stopping cheats, also 
> means that the students get to flush out a few bugs :-).

OK, so we can add that to the doco? I'll submit a patch.

> -> should all the [missing] @var{nl} be moved to @var{output}, that way 
> the're all in the one place.
> 
> Rossi writes:
> >Yes, this could work. However, was the original grammer meant to be
> >LALR(2)?
> 
> rewritable into something needing only one level of look-ahead, which I 
> see is being done.

Andrew, Michael has helped me figure out a more efficient way to reduce
the grammar. Now I have a LL(1) parser that has no conflicts, without
playing games in the lexer.

Also, a word of good news, so far I have sent several 'output' commands
of GDB through the parser, and it likes the syntax! This is a good thing
in my opinion, and so far, something to be thankful for :)

I would greatly appreciate it if we could validate GDB's output with the
library I am currently working on. I would love to know that everything
GDB outputs is sanitary. Anyways, the next step for me is to write a
syntax tree, representing the output. From there, I will be able to get
some work done on my front end.

Thanks,
Bob Rossi