confused by yyungetc() (cp/lex.c)

confused by yyungetc() (cp/lex.c)

[Zack Weinberg]

I'm presently trying to wean cp/lex.c off its dependency on text
streams so it can use the integrated preprocessor.  I am severely
confused by yyungetc():

void
yyungetc (ch, rescan)
     int ch;
     int rescan;
{
  /* Unget a character from the input stream.  */
  if (yychar == YYEMPTY || rescan == 0)
    {
      /* If we're putting back a brace, undo the change in indent_level
	 from the first time we saw it.  */
      if (ch == '{')
	indent_level--;
      else if (ch == '}')
	indent_level++;

      put_back (ch);
    }
  else
    {
      yychar = ch;
    }
}

What it does is pretty straightforward: it puts a single-character
token back on the input so it will be reconsidered by yyparse.  (It's
a hack around only having one lookahead token.)

What I don't understand is the choice between pushing the token back
into the lexer's stream and just setting yychar.  In a very few cases
it's obvious, as when we're doing heroic error recovery by pushing
back *two* tokens - that looks like

  yyungetc (':', 0);
  yychar = ')';

-- here we have to put the colon somewhere other than yychar.  In
almost all other cases, however, we are clobbering yychar if it is set
(yychar != YYEMPTY) and not using it if it isn't.  There are a couple
of rules in cp/parse.y that call yyungetc with rescan=0 but don't
immediately set yychar, too.

zw