stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

I'm trying to build gcc-3.3 (2002-08-18 cvs) but get this error


stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I. -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/. -I/opt/gcc+binutils/gcc/gcc/config -I/opt/gcc+binutils/gcc/gcc/../include /opt/gcc+binutils/gcc/gcc/cppfiles.c -o cppfiles.o
stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I. -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/. -I/opt/gcc+binutils/gcc/gcc/config -I/opt/gcc+binutils/gcc/gcc/../include /opt/gcc+binutils/gcc/gcc/cpptrad.c -o cpptrad.o
stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I. -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/. -I/opt/gcc+binutils/gcc/gcc/config -I/opt/gcc+binutils/gcc/gcc/../include /opt/gcc+binutils/gcc/gcc/cpphash.c -o cpphash.o
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1168: error: parse error before ']' token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1380: error: syntax error at '#' token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#' token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#' token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#' token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#' token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1451:21: warning: null character(s) ignored
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1451: confused by earlier errors, bailing out
make[2]: *** [cppfiles.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/opt/gcc+binutils/objdir/gcc'
make[1]: *** [stage2_build] Error 2
make[1]: Leaving directory `/opt/gcc+binutils/objdir/gcc'
make: *** [bootstrap-lean] Error 2

Any idea of what might be going wrong for me?

This was on a Debian 3.0 (Woody) quad ROSS sun4m system with these
packages:

binutils                           2.12.90.0.1-4
dejagnu                            1.4.2-1.1
gcc                                2:2.95.4-14 (Debian prerelease)
gcc-2.95			   1:2.95.4-7 (Debian prerelease)
gnat				   3.14p-3
self compiled 2.4.19-rc3 SMP kernel running during build and test

In-tree joined gcc and binutils cvs trunks.

RE: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2905 bytes --]

Hmm, there are no lines after 1168... Weird.

/ChJ

-----Original Message-----
From: gcc-owner@gcc.gnu.org [ mailto:gcc-owner@gcc.gnu.org ] On Behalf Of
Christian Jönsson
Sent: Sunday, August 18, 2002 4:42 PM
To: gcc
Subject: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168:
error: parse error before ']' token etc.


I'm trying to build gcc-3.3 (2002-08-18 cvs) but get this error


stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W
-Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
-Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
-I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
-I/opt/gcc+binutils/gcc/gcc/config
-I/opt/gcc+binutils/gcc/gcc/../include
/opt/gcc+binutils/gcc/gcc/cppfiles.c -o cppfiles.o
stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W
-Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
-Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
-I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
-I/opt/gcc+binutils/gcc/gcc/config
-I/opt/gcc+binutils/gcc/gcc/../include
/opt/gcc+binutils/gcc/gcc/cpptrad.c -o cpptrad.o
stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W
-Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
-Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
-I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
-I/opt/gcc+binutils/gcc/gcc/config
-I/opt/gcc+binutils/gcc/gcc/../include
/opt/gcc+binutils/gcc/gcc/cpphash.c -o cpphash.o
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1168: error: parse error before ']'
token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1380: error: syntax error at '#'
token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#'
token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#'
token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#'
token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#'
token
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1451:21: warning: null character(s)
ignored
/opt/gcc+binutils/gcc/gcc/cppfiles.c:1451: confused by earlier errors,
bailing out
make[2]: *** [cppfiles.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/opt/gcc+binutils/objdir/gcc'
make[1]: *** [stage2_build] Error 2
make[1]: Leaving directory `/opt/gcc+binutils/objdir/gcc'
make: *** [bootstrap-lean] Error 2

Any idea of what might be going wrong for me?

This was on a Debian 3.0 (Woody) quad ROSS sun4m system with these
packages:

binutils                           2.12.90.0.1-4
dejagnu                            1.4.2-1.1
gcc                                2:2.95.4-14 (Debian prerelease)
gcc-2.95			   1:2.95.4-7 (Debian prerelease)
gnat				   3.14p-3
self compiled 2.4.19-rc3 SMP kernel running during build and test

In-tree joined gcc and binutils cvs trunks.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3359 bytes --]

On Sun, Aug 18, 2002 at 04:54:17PM +0200, Christian Jönsson wrote:
> Hmm, there are no lines after 1168... Weird.
> 
> /ChJ
> 
> -----Original Message-----
> From: gcc-owner@gcc.gnu.org [ mailto:gcc-owner@gcc.gnu.org ] On Behalf Of
> Christian Jönsson
> Sent: Sunday, August 18, 2002 4:42 PM
> To: gcc
> Subject: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168:
> error: parse error before ']' token etc.
> 
> 
> I'm trying to build gcc-3.3 (2002-08-18 cvs) but get this error
> 
> 
> stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W
> -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
> -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
> -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
> -I/opt/gcc+binutils/gcc/gcc/config -I/opt/gcc+binutils/gcc/gcc/../include
> /opt/gcc+binutils/gcc/gcc/cppfiles.c -o cppfiles.o
> stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W
> -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
> -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
> -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
> -I/opt/gcc+binutils/gcc/gcc/config -I/opt/gcc+binutils/gcc/gcc/../include
> /opt/gcc+binutils/gcc/gcc/cpptrad.c -o cpptrad.o
> stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC   -W
> -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
> -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
> -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
> -I/opt/gcc+binutils/gcc/gcc/config -I/opt/gcc+binutils/gcc/gcc/../include
> /opt/gcc+binutils/gcc/gcc/cpphash.c -o cpphash.o
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1168: error: parse error before ']'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1380: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1451:21: warning: null character(s)
> ignored
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1451: confused by earlier errors,
> bailing out


Same today (2002-08-21) on an other, quite similar system...

This was on a Debian 3.0 (Woody) dual SuperSparc-II sun4m system with these
packages:

binutils                           2.12.90.0.1-4
dejagnu                            1.4.2-1.1
gcc                                2:2.95.4-14 (Debian prerelease)
gcc-2.95			   1:2.95.4-7 (Debian prerelease)
gnat				   3.14p-3
self compiled 2.4.19-rc3 SMP kernel running during build and test

In-tree joined gcc and binutils cvs trunks.

What on earth could it be?

/ChJ
 
> Any idea of what might be going wrong for me?
> 
> This was on a Debian 3.0 (Woody) quad ROSS sun4m system with these
> packages:
> 
> binutils                           2.12.90.0.1-4
> dejagnu                            1.4.2-1.1
> gcc                                2:2.95.4-14 (Debian prerelease)
> gcc-2.95			   1:2.95.4-7 (Debian prerelease)
> gnat				   3.14p-3
> self compiled 2.4.19-rc3 SMP kernel running during build and test
> 
> In-tree joined gcc and binutils cvs trunks.
> 

RE: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3872 bytes --]

FWIW, I can bootstrap gcc-3.2, but still the same problem with
gcc-3.3...

The problem:

stage1 compile with installed gcc-2.95.4 (debian prerelase) "works" on
cppfiles.c

Stage2 compile with stage1/gcc bails out on cppflies.c

(I had the wrong subject, it's stage2 of the bootstrap that fails)

Cheers,

/ChJ

-----Original Message-----
From: 
Sent: Wednesday, August 21, 2002 4:03 PM
To: 'gcc'
Subject: Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168:
error: parse error before ']' token etc.


On Sun, Aug 18, 2002 at 04:54:17PM +0200, Christian Jönsson wrote:
> Hmm, there are no lines after 1168... Weird.
> 
> /ChJ
> 
> -----Original Message-----
> From: gcc-owner@gcc.gnu.org [ mailto:gcc-owner@gcc.gnu.org ] On Behalf 
> Of Christian Jönsson
> Sent: Sunday, August 18, 2002 4:42 PM
> To: gcc
> Subject: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168:
> error: parse error before ']' token etc.
> 
> 
> I'm trying to build gcc-3.3 (2002-08-18 cvs) but get this error
> 
> 
> stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC
-W
> -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
> -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
> -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/. 
> -I/opt/gcc+binutils/gcc/gcc/config 
> -I/opt/gcc+binutils/gcc/gcc/../include
> /opt/gcc+binutils/gcc/gcc/cppfiles.c -o cppfiles.o
> stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC
-W
> -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
> -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
> -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
> -I/opt/gcc+binutils/gcc/gcc/config
-I/opt/gcc+binutils/gcc/gcc/../include
> /opt/gcc+binutils/gcc/gcc/cpptrad.c -o cpptrad.o
> stage1/xgcc -Bstage1/ -B/usr/sparc-linux/bin/ -c   -g -O2 -DIN_GCC
-W
> -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes
> -Wtraditional -pedantic -Wno-long-long   -DHAVE_CONFIG_H    -I. -I.
> -I/opt/gcc+binutils/gcc/gcc -I/opt/gcc+binutils/gcc/gcc/.
> -I/opt/gcc+binutils/gcc/gcc/config
-I/opt/gcc+binutils/gcc/gcc/../include
> /opt/gcc+binutils/gcc/gcc/cpphash.c -o cpphash.o
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1168: error: parse error before
']'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1380: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1385: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1388: error: syntax error at '#'
> token
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1451:21: warning: null
character(s)
> ignored
> /opt/gcc+binutils/gcc/gcc/cppfiles.c:1451: confused by earlier errors,
> bailing out


Same today (2002-08-21) on an other, quite similar system...

This was on a Debian 3.0 (Woody) dual SuperSparc-II sun4m system with
these
packages:

binutils                           2.12.90.0.1-4
dejagnu                            1.4.2-1.1
gcc                                2:2.95.4-14 (Debian prerelease)
gcc-2.95			   1:2.95.4-7 (Debian prerelease)
gnat				   3.14p-3
self compiled 2.4.19-rc3 SMP kernel running during build and test

In-tree joined gcc and binutils cvs trunks.

What on earth could it be?

/ChJ
 
> Any idea of what might be going wrong for me?
> 
> This was on a Debian 3.0 (Woody) quad ROSS sun4m system with these
> packages:
> 
> binutils                           2.12.90.0.1-4
> dejagnu                            1.4.2-1.1
> gcc                                2:2.95.4-14 (Debian prerelease)
> gcc-2.95			   1:2.95.4-7 (Debian prerelease)
> gnat				   3.14p-3
> self compiled 2.4.19-rc3 SMP kernel running during build and test
> 
> In-tree joined gcc and binutils cvs trunks.
> 

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Neil Booth]

Christian J?nsson wrote:-

> FWIW, I can bootstrap gcc-3.2, but still the same problem with
> gcc-3.3...
> 
> The problem:
> 
> stage1 compile with installed gcc-2.95.4 (debian prerelase) "works" on
> cppfiles.c
> 
> Stage2 compile with stage1/gcc bails out on cppflies.c
> 
> (I had the wrong subject, it's stage2 of the bootstrap that fails)

You are the only person that has ever reported this.  I'm afraid
you will need to investigate it yourself.

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

On Tue, Aug 27, 2002 at 05:57:32PM +0100, Neil Booth wrote:
> Christian J?nsson wrote:-
> 
> > FWIW, I can bootstrap gcc-3.2, but still the same problem with
> > gcc-3.3...
> > 
> > The problem:
> > 
> > stage1 compile with installed gcc-2.95.4 (debian prerelase) "works" on
> > cppfiles.c
> > 
> > Stage2 compile with stage1/gcc bails out on cppflies.c
> > 
> > (I had the wrong subject, it's stage2 of the bootstrap that fails)
> 
> You are the only person that has ever reported this.  I'm afraid
> you will need to investigate it yourself.

hmm, I just tried the recently release Aurora SPARC Linux build-0.32,
same thing there... I am really lost here. Is there something with my
config options that are, well, bad?

--enable-shared --enable-threads=posix --with-gnu-as --with-gnu-ld
--with-system-zlib --enable-long-long
--enable-languages=c,c++,f77,java,objc --enable-nls --enable-symvers
--without-x --without-included-gettext --disable-checking

or is it that the joining of the gcc and binutils trunk trees should
be done in an other fashion these days? I have these symlinks from the
gcc dir into the src dir of binutils:

bfd@
binutils@
gas@
gprof@
ld@
opcodes@

and these from the gcc/include dir into the binutil's src/include dir

aout@
bfdlink.h@
bin-bugs.h@
bout.h@
callback.h@
coff@
dis-asm.h@
elf@
filenames.h@
fopen-same.h@
ieee.h@
opcode@
progress.h@
remote-sim.h@

am I missing something that has changed here??

Cheers,

/ChJ

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Neil Booth]

Neil Booth wrote:-

> Christian J?nsson wrote:-
> 
> > FWIW, I can bootstrap gcc-3.2, but still the same problem with
> > gcc-3.3...
> > 
> > The problem:
> > 
> > stage1 compile with installed gcc-2.95.4 (debian prerelase) "works" on
> > cppfiles.c
> > 
> > Stage2 compile with stage1/gcc bails out on cppflies.c
> > 
> > (I had the wrong subject, it's stage2 of the bootstrap that fails)
> 
> You are the only person that has ever reported this.  I'm afraid
> you will need to investigate it yourself.

I'm 99% certain what causes this.  What filesystem are you using?
Is it XFS?  Notice the filesize of cppfiles.c?

Neil.

RE: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

Uhm, no, e2fs

fw:~# fdisk -l /dev/sdd

Disk /dev/sdd (Sun disk label): 21 heads, 94 sectors, 2072 cylinders
Units = cylinders of 1974 * 512 bytes

   Device Flag    Start       End    Blocks   Id  System
/dev/sdd1             0      1037   1023519   83  Linux native
/dev/sdd2  u       1037      2072   1021545   83  Linux native
/dev/sdd3             0      2072   2045064    5  Whole disk
fw:~# df -h /dev/sdd
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdd              718M  552M  130M  81%
fw:~# df -h /dev/sdd1
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdd1             984M  322M  612M  35% /share1
fw:~# df -h /dev/sdd2
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdd2             982M  256M  676M  28% /share2
fw:~#



-----Original Message-----
From: Neil Booth [mailto:neil@daikokuya.co.uk] 
Sent: Tuesday, September 03, 2002 9:54 PM
To: Christian J?nsson
Cc: 'gcc'
Subject: Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168:
error: parse error before ']' token etc.


Neil Booth wrote:-

> Christian J?nsson wrote:-
> 
> > FWIW, I can bootstrap gcc-3.2, but still the same problem with 
> > gcc-3.3...
> > 
> > The problem:
> > 
> > stage1 compile with installed gcc-2.95.4 (debian prerelase) "works" 
> > on cppfiles.c
> > 
> > Stage2 compile with stage1/gcc bails out on cppflies.c
> > 
> > (I had the wrong subject, it's stage2 of the bootstrap that fails)
> 
> You are the only person that has ever reported this.  I'm afraid you 
> will need to investigate it yourself.

I'm 99% certain what causes this.  What filesystem are you using? Is it
XFS?  Notice the filesize of cppfiles.c?

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

['Neil Booth']

Christian J?nsson wrote:-

> Uhm, no, e2fs
 
What is happening is that the file is being mmap-ed, but for some reason
the mapped memory is not NUL-padded.  It might be an idea for me to add
a check to cppfiles.c for this case and abort if it's not a NUL.  What do
you think, Zack?

What I don't understand is, if you're using a standard kernel and e2fs
like me, why I have never seen this issue.

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Zack Weinberg]

On Wed, Sep 04, 2002 at 08:44:21PM +0100, 'Neil Booth' wrote:
> Christian J?nsson wrote:-
> 
> > Uhm, no, e2fs
>  
> What is happening is that the file is being mmap-ed, but for some reason
> the mapped memory is not NUL-padded.  It might be an idea for me to add
> a check to cppfiles.c for this case and abort if it's not a NUL.  What do
> you think, Zack?

I'm coming to feel that mmap is more trouble than it's worth.  Would
your contempated line-at-a-time lexer be happy with files being read
in 4k or 8k chunks with plain old read()?

zw

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

['Neil Booth']

Zack Weinberg wrote:-

> > What is happening is that the file is being mmap-ed, but for some reason
> > the mapped memory is not NUL-padded.  It might be an idea for me to add
> > a check to cppfiles.c for this case and abort if it's not a NUL.  What do
> > you think, Zack?
> 
> I'm coming to feel that mmap is more trouble than it's worth.  Would
> your contempated line-at-a-time lexer be happy with files being read
> in 4k or 8k chunks with plain old read()?

It certainly could be made so.  Let's see if is fits nicely with
whatever comes up.

I'm working on tweaking CPP in the BIB to do line-at-a-time tokens
now, but don't know when I'll have something working.  This is separate
to prescanning for stages 1 and 2.

Although doing a preprass on logical lines to do stages 1 and 2 first
would make some things nice, it makes some things harder.  For example,
it's hard to avoid warning about trigraphs in comments as it has no
idea whether it's in a comment.  Issues like this make me wonder whether
a prescan is worth it.

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Zack Weinberg]

On Thu, Sep 05, 2002 at 07:05:19PM +0100, 'Neil Booth' wrote:
> Although doing a preprass on logical lines to do stages 1 and 2 first
> would make some things nice, it makes some things harder.  For example,
> it's hard to avoid warning about trigraphs in comments as it has no
> idea whether it's in a comment.  Issues like this make me wonder whether
> a prescan is worth it.

I was thinking of doing stage 3 (comment strip) in there too.  It's
pretty easy to enumerate the cases to be dealt with:

case 1: Logical line contains no characters in the set /\?, no
characters outside the basic source set, and is entirely inside the
read() buffer.  We scan for the newline, and return the interval to
the phase-4 lexer.

case 2: Logical line contains / not immediately followed by * or /; \
not immediately followed by whitespace and newline; or ? not
immediately followed by the rest of a trigraph.  Catch this out of
line, jump back into the scanner loop.

case 3: Logical line ends with a // comment.  Overwrite the leading
slash with a \n terminator, advance the read pointer past the comment,
and return the narrowed interval to phase 4.  (This saves copying the
line.)

case 4: Logical line contains a trigraph, backslash-newline, block
comment, or character outside the basic source set.  Jump to
particular logic to handle that case.  They'll all probably involve
copying the logical line to a second buffer, with some sort of
annotation on the side so we don't lose track of the original source
position.

We actually get a break from the committee, in that neither \[uU]
escapes nor raw multibyte characters can validly represent characters
in the basic source set, which means we don't need to worry about
e.g. \u002F\u002A starting a comment.

If I get some free time (ha) I'll look at coding this up...

zw

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

['Neil Booth']

Zack Weinberg wrote:-

> I was thinking of doing stage 3 (comment strip) in there too.  It's
> pretty easy to enumerate the cases to be dealt with:
> 
> case 1: Logical line contains no characters in the set /\?, no
> characters outside the basic source set, and is entirely inside the
> read() buffer.  We scan for the newline, and return the interval to
> the phase-4 lexer.
> 
> case 2: Logical line contains / not immediately followed by * or /; \
> not immediately followed by whitespace and newline; or ? not
> immediately followed by the rest of a trigraph.  Catch this out of
> line, jump back into the scanner loop.

OK.  You get the boring cases of / and * separated by escaped newlines
too (or is that case 4?).  What about quotes?
 
> case 3: Logical line ends with a // comment.  Overwrite the leading
> slash with a \n terminator, advance the read pointer past the comment,
> and return the narrowed interval to phase 4.  (This saves copying the
> line.)

OK; this assumes we've given up on mmap, right?  What about quotes?
 
> case 4: Logical line contains a trigraph, backslash-newline, block
> comment, or character outside the basic source set.  Jump to
> particular logic to handle that case.  They'll all probably involve
> copying the logical line to a second buffer, with some sort of
> annotation on the side so we don't lose track of the original source
> position.

Yes.  What would you do about caret diagnostics to get the caret to
point to the right place in the original line?

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Zack Weinberg]

On Wed, Sep 11, 2002 at 07:42:30AM +0100, 'Neil Booth' wrote:
> > case 2: Logical line contains / not immediately followed by * or /; \
> > not immediately followed by whitespace and newline; or ? not
> > immediately followed by the rest of a trigraph.  Catch this out of
> > line, jump back into the scanner loop.
> 
> OK.  You get the boring cases of / and * separated by escaped newlines
> too (or is that case 4?). 

Yes, that's case 4.

> What about quotes?

Ugh, I forgot all about quotes.  I guess they're another case again,
or a state flag.

That might kick the complexity up to the point where it's not worth
it.

> > case 4: Logical line contains a trigraph, backslash-newline, block
> > comment, or character outside the basic source set.  Jump to
> > particular logic to handle that case.  They'll all probably involve
> > copying the logical line to a second buffer, with some sort of
> > annotation on the side so we don't lose track of the original source
> > position.
> 
> Yes.  What would you do about caret diagnostics to get the caret to
> point to the right place in the original line?

That's what the annotations on the side are for.  They're basically a
list of reverse transformations that take you from the logical line to
the set of physical lines it was composed from.

It occurs to me that one good reason to keep the complete text of all
files around forever is to facilitate generating caret diagnostics; we
may want to do that well after we're done with any given logical line.

zw

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

['Neil Booth']

Zack Weinberg wrote:-

> > What about quotes?
> 
> Ugh, I forgot all about quotes.  I guess they're another case again,
> or a state flag.

The hardest quote of all is '<' since you need context.  Lexing ISO C
is really quite painful, the more you think about it 8-)
 
> That might kick the complexity up to the point where it's not worth
> it.

That's the conclusion I came to when I tried it.  What we have now is
not too bad I think; it just needs some tweaks.  However, having a
"sanitized" source line is nice in that you can have each token point
directly to its spelling.  However, I'm not sure the complexity it
entails is any improvement over the current situation.

> That's what the annotations on the side are for.  They're basically a
> list of reverse transformations that take you from the logical line to
> the set of physical lines it was composed from.

Yup.  I tried that once, but I couldn't convince myself that the
overhead was worth it.

I figure that if we made a token's "col" mean "logical character on
a physical line", we could have a fairly simple loop that understands
trigraphs that selects the right point.  If, in the caret line, we replace
every character of the original line with a space (except tabs) we will
get the same location for output (with some handling of multibyte chars
when we support them).

> It occurs to me that one good reason to keep the complete text of all
> files around forever is to facilitate generating caret diagnostics; we
> may want to do that well after we're done with any given logical line.

That one possibility, though I tended to think that we should treat
diagnostics as the slow path, and so taking the hit of re-reading
the file if necessary would be OK.

Ugh.  I really don't think there is a clean solution to this mess,
certainly not one that is simultaneously efficient.

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Zack Weinberg]

On Wed, Sep 11, 2002 at 10:32:04PM +0100, 'Neil Booth' wrote:
> Zack Weinberg wrote:-
> 
> > > What about quotes?
> > 
> > Ugh, I forgot all about quotes.  I guess they're another case again,
> > or a state flag.
> 
> The hardest quote of all is '<' since you need context.  Lexing ISO C
> is really quite painful, the more you think about it 8-)

No kidding.

> I figure that if we made a token's "col" mean "logical character on
> a physical line", we could have a fairly simple loop that understands
> trigraphs that selects the right point.  If, in the caret line, we replace
> every character of the original line with a space (except tabs) we will
> get the same location for output (with some handling of multibyte chars
> when we support them).

I'm not sure I understand the context in which you propose this
solution.  Can you expand a bit?

> > It occurs to me that one good reason to keep the complete text of all
> > files around forever is to facilitate generating caret diagnostics; we
> > may want to do that well after we're done with any given logical line.
> 
> That one possibility, though I tended to think that we should treat
> diagnostics as the slow path, and so taking the hit of re-reading
> the file if necessary would be OK.

Yes, diagnostics are definitely the slow path.  If we're going to be
rereading the file we might want to consider stashing byte offsets for
purposes of doing lseek() to get back there.  Except that may not work
right on VMS.

> Ugh.  I really don't think there is a clean solution to this mess,
> certainly not one that is simultaneously efficient.

It occurs to me that the prime difficulty with reading the file in
chunks, independent of whether we have a prescan pass, is the (common
but not normal) situation of having a line cross a chunk boundary.  We
could deal with that by scanning backward from the end of the chunk
for the last newline.  Put the limit pointer there.  Then we only have
to check the limit pointer at newlines, and when we hit it we can just
copy the trailing text down to the front of the buffer and keep
reading.

If the file has a single line longer than the chunk size we have to
resize the buffer, but we'd have to do that anyway.

zw

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

['Neil Booth']

Zack Weinberg wrote:-

> > I figure that if we made a token's "col" mean "logical character on
> > a physical line", we could have a fairly simple loop that understands
> > trigraphs that selects the right point.  If, in the caret line, we replace
> > every character of the original line with a space (except tabs) we will
> > get the same location for output (with some handling of multibyte chars
> > when we support them).
> 
> I'm not sure I understand the context in which you propose this
> solution.  Can you expand a bit?

Sorry I was unclear.  Any context in which we pre-scan the line removing
trigraphs and escaped newlines.

> It occurs to me that the prime difficulty with reading the file in
> chunks, independent of whether we have a prescan pass, is the (common
> but not normal) situation of having a line cross a chunk boundary.  We
> could deal with that by scanning backward from the end of the chunk
> for the last newline.  Put the limit pointer there.  Then we only have
> to check the limit pointer at newlines, and when we hit it we can just
> copy the trailing text down to the front of the buffer and keep
> reading.

Yes.  You might be able to just terminate the chunk with NUL and lex
like we do now.  You just need to be able to back up over the previous
token when you reach the NUL, since it might have been prematurely
terminated.  This means you have to be careful about issuing diagnostics
too early, in case they were caused by the premature NUL.
 
> If the file has a single line longer than the chunk size we have to
> resize the buffer, but we'd have to do that anyway.

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Zack Weinberg]

On Thu, Sep 12, 2002 at 09:05:45AM +0100, 'Neil Booth' wrote:
> Zack Weinberg wrote:-
> 
> > > I figure that if we made a token's "col" mean "logical character
> > > on a physical line", we could have a fairly simple loop that
> > > understands trigraphs that selects the right point.  If, in the
> > > caret line, we replace every character of the original line with
> > > a space (except tabs) we will get the same location for output
> > > (with some handling of multibyte chars when we support them).
> > 
> > I'm not sure I understand the context in which you propose this
> > solution.  Can you expand a bit?
> 
> Sorry I was unclear.  Any context in which we pre-scan the line removing
> trigraphs and escaped newlines.

Okay, yes, that will work.  It unfortunately doesn't help us avoid
warning about trigraphs inside comments, though.  One possibility is
that we treat trigraphs as alternate spellings except for ??/newline
(and string constants).

Or then again, maybe what we do now is good enough.

The real question is how we make this stuff play nice with character
encoding conversion.  And that also raises the question of characters
for which wcwidth() != 1.

> > It occurs to me that the prime difficulty with reading the file in
> > chunks, independent of whether we have a prescan pass, is the (common
> > but not normal) situation of having a line cross a chunk boundary.  We
> > could deal with that by scanning backward from the end of the chunk
> > for the last newline.  Put the limit pointer there.  Then we only have
> > to check the limit pointer at newlines, and when we hit it we can just
> > copy the trailing text down to the front of the buffer and keep
> > reading.
> 
> Yes.  You might be able to just terminate the chunk with NUL and lex
> like we do now.  You just need to be able to back up over the previous
> token when you reach the NUL, since it might have been prematurely
> terminated.  This means you have to be careful about issuing diagnostics
> too early, in case they were caused by the premature NUL.

The idea is to have \n be the end-of-chunk marker, not NUL, which i
think actually makes life easier.

zw

RE: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Christian Jönsson]

Hmm, for some reason, it bootstraps right now.... Just the last few
days...

Oh well, great I guess..

Will post testsuite results soon... Tomorrow morning I guess...

Cheers,

/ChJ

-----Original Message-----
From: gcc-owner@gcc.gnu.org [mailto:gcc-owner@gcc.gnu.org] On Behalf Of
'Neil Booth'
Sent: Thursday, September 12, 2002 10:06 AM
To: Zack Weinberg
Cc: 'gcc'
Subject: Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168:
error: parse error before ']' token etc.


Zack Weinberg wrote:-

> > I figure that if we made a token's "col" mean "logical character on 
> > a physical line", we could have a fairly simple loop that 
> > understands trigraphs that selects the right point.  If, in the 
> > caret line, we replace every character of the original line with a 
> > space (except tabs) we will get the same location for output (with 
> > some handling of multibyte chars when we support them).
> 
> I'm not sure I understand the context in which you propose this 
> solution.  Can you expand a bit?

Sorry I was unclear.  Any context in which we pre-scan the line removing
trigraphs and escaped newlines.

> It occurs to me that the prime difficulty with reading the file in 
> chunks, independent of whether we have a prescan pass, is the (common 
> but not normal) situation of having a line cross a chunk boundary.  We

> could deal with that by scanning backward from the end of the chunk 
> for the last newline.  Put the limit pointer there.  Then we only have

> to check the limit pointer at newlines, and when we hit it we can just

> copy the trailing text down to the front of the buffer and keep 
> reading.

Yes.  You might be able to just terminate the chunk with NUL and lex
like we do now.  You just need to be able to back up over the previous
token when you reach the NUL, since it might have been prematurely
terminated.  This means you have to be careful about issuing diagnostics
too early, in case they were caused by the premature NUL.
 
> If the file has a single line longer than the chunk size we have to 
> resize the buffer, but we'd have to do that anyway.

Neil.

Re: stage1 bootstrap failuer to build gcc.3.3: cppfiles.c:1168: error: parse error before ']' token etc.

[Neil Booth]

Christian J?nsson wrote:-

> Hmm, for some reason, it bootstraps right now.... Just the last few
> days...

I imagine a patch changed the file size.  You only need to add or
remove a space...

Neil.