Re: Can we remove bison output from cvs?

Re: Can we remove bison output from cvs?

[Bruce Korb]

[-- Attachment #1: Type: text/plain, Size: 726 bytes --]

> The main cost is checking in the generated files whenever the source
> files
> are changed.
> Although it should be possible to do this automatically on the server
> side, but that'll need someone to tinker with the configuration to get
> it running.

Not very hard.  Attached is a script I threw together after dinner last night.
There are actually three files:

1.  rules.sh   the script that does the work.

2.  rules.def  describes the dependencies and commands to transform
               the sources into the generated files (incomplete)

3.  submit.tpl hieroglyphics that direct the transformation of #2 into #1
               don't bother with this unless you are *really* interested.
rules.sh
rules.def
submit.tpl


[-- Attachment #2: rules.def --]
[-- Type: text/plain, Size: 1037 bytes --]


autogen definitions submit;

rule = {
    target = fixincl.x;

    dir = gcc/fixinc;
    depend = inclhack.def;
    depend = fixincl.tpl;
    script = "autogen -L${SRCDIR} -T${SRCDIR}/fixincl.tpl -b fixincl "
                      "${SRCDIR}/inclhack.def";
};

rule = {
    target = fixincl.sh;

    dir = gcc/fixinc;
    depend = inclhack.tpl;
    script = "autogen -L${SRCDIR} -DPROGRAM=1 -b fixincl "
                      "${SRCDIR}/inclhack.def";
};

rule = {
    target = inclhack.sh;

    dir = gcc/fixinc;
    depend = inclhack.def;
    depend = inclhack.tpl;
    depend = hackshell.tpl;
    script = "autogen -L${SRCDIR} ${SRCDIR}/inclhack.def";
};

rule = {
    target = c-parse.c;
    target = c-parse.h;
    target = c-parse.y;

    depend = c-parse.in;
    dir = gcc;
    script =
	"rm -f c-parse.y\n"
	"( echo '/*WARNING: This file is automatically generated!*/'\n"
	"  sed -e'/^ifobjc$/,/^end ifobjc$/d' -e'/^ifc$/d' -e'/^end ifc$/d' "
		"${SRCDIR}/c-parse.in\n"
	") > c-parse.y\n"
	"bison -d c-parse.y -o c-parse.c";
};

[-- Attachment #3: rules.sh --]
[-- Type: text/x-shellscript, Size: 4059 bytes --]

#!/bin/sh
#
#  DO NOT EDIT THIS FILE   (rules.sh)
#  
#  It has been autogen-ed  Friday January 15, 1999 at 07:47:10 AM PST
#  From the definitions    rules.def
#  and the template file   submit.tpl

HOMEDIR=`pwd`
SRCDIR=`cd egcs/gcc/fixinc ; pwd`

test -d egcsbld/gcc/fixinc/CVS || {
    echo "The directory ${HOMEDIR}/egcsbld/gcc/fixinc must be added to CVS"
    exit 1
}
TARGET="fixincl.x"

for f in ${TARGET} ; do
    if [ ! -f egcsbld/gcc/fixinc/$f ] ; then
        touch 0101000080 egcsbld/gcc/fixinc/$f
        cvs add egcsbld/gcc/fixinc/$f
    fi
done > /dev/null 2>&1

if  ([ egcs/gcc/fixinc/inclhack.def -nt egcsbld/gcc/fixinc/fixincl.x \
    -o egcs/gcc/fixinc/fixincl.tpl -nt egcsbld/gcc/fixinc/fixincl.x \
    ]) > /dev/null 2>&1
then
    cd egcsbld/gcc/fixinc
    cvs edit ${TARGET} > /dev/null 2>&1
    cat <<-	'_EOF_'
	Rebuilding fixincl.x
	autogen -L${SRCDIR} -T${SRCDIR}/fixincl.tpl -b fixincl ${SRCDIR}/inclhack.def
	_EOF_

    if ( autogen -L${SRCDIR} -T${SRCDIR}/fixincl.tpl -b fixincl ${SRCDIR}/inclhack.def )
    then
        cvs commit -m"Regenerated source" ${TARGET}
    else
        echo "Could not rebuild ${TARGET}"
        cvs unedit ${TARGET}
    fi
    cd ${HOMEDIR}
fi


TARGET="fixincl.sh"

for f in ${TARGET} ; do
    if [ ! -f egcsbld/gcc/fixinc/$f ] ; then
        touch 0101000080 egcsbld/gcc/fixinc/$f
        cvs add egcsbld/gcc/fixinc/$f
    fi
done > /dev/null 2>&1

if  ([ egcs/gcc/fixinc/inclhack.tpl -nt egcsbld/gcc/fixinc/fixincl.sh \
    ]) > /dev/null 2>&1
then
    cd egcsbld/gcc/fixinc
    cvs edit ${TARGET} > /dev/null 2>&1
    cat <<-	'_EOF_'
	Rebuilding fixincl.sh
	autogen -L${SRCDIR} -DPROGRAM=1 -b fixincl ${SRCDIR}/inclhack.def
	_EOF_

    if ( autogen -L${SRCDIR} -DPROGRAM=1 -b fixincl ${SRCDIR}/inclhack.def )
    then
        cvs commit -m"Regenerated source" ${TARGET}
    else
        echo "Could not rebuild ${TARGET}"
        cvs unedit ${TARGET}
    fi
    cd ${HOMEDIR}
fi


TARGET="inclhack.sh"

for f in ${TARGET} ; do
    if [ ! -f egcsbld/gcc/fixinc/$f ] ; then
        touch 0101000080 egcsbld/gcc/fixinc/$f
        cvs add egcsbld/gcc/fixinc/$f
    fi
done > /dev/null 2>&1

if  ([ egcs/gcc/fixinc/inclhack.def -nt egcsbld/gcc/fixinc/inclhack.sh \
    -o egcs/gcc/fixinc/inclhack.tpl -nt egcsbld/gcc/fixinc/inclhack.sh \
    -o egcs/gcc/fixinc/hackshell.tpl -nt egcsbld/gcc/fixinc/inclhack.sh \
    ]) > /dev/null 2>&1
then
    cd egcsbld/gcc/fixinc
    cvs edit ${TARGET} > /dev/null 2>&1
    cat <<-	'_EOF_'
	Rebuilding inclhack.sh
	autogen -L${SRCDIR} ${SRCDIR}/inclhack.def
	_EOF_

    if ( autogen -L${SRCDIR} ${SRCDIR}/inclhack.def )
    then
        cvs commit -m"Regenerated source" ${TARGET}
    else
        echo "Could not rebuild ${TARGET}"
        cvs unedit ${TARGET}
    fi
    cd ${HOMEDIR}
fi


SRCDIR=`cd egcs/gcc ; pwd`

test -d egcsbld/gcc/CVS || {
    echo "The directory ${HOMEDIR}/egcsbld/gcc must be added to CVS"
    exit 1
}
TARGET="c-parse.c c-parse.h c-parse.y"

for f in ${TARGET} ; do
    if [ ! -f egcsbld/gcc/$f ] ; then
        touch 0101000080 egcsbld/gcc/$f
        cvs add egcsbld/gcc/$f
    fi
done > /dev/null 2>&1

if  ([ egcs/gcc/c-parse.in -nt egcsbld/gcc/c-parse.c \
    -o egcs/gcc/c-parse.in -nt egcsbld/gcc/c-parse.h \
    -o egcs/gcc/c-parse.in -nt egcsbld/gcc/c-parse.y \
    ]) > /dev/null 2>&1
then
    cd egcsbld/gcc
    cvs edit ${TARGET} > /dev/null 2>&1
    cat <<-	'_EOF_'
	Rebuilding c-parse.c, c-parse.h, and c-parse.y
	rm -f c-parse.y
( echo '/*WARNING: This file is automatically generated!*/'
  sed -e'/^ifobjc$/,/^end ifobjc$/d' -e'/^ifc$/d' -e'/^end ifc$/d' ${SRCDIR}/c-parse.in
) > c-parse.y
bison -d c-parse.y -o c-parse.c
	_EOF_

    if ( rm -f c-parse.y
( echo '/*WARNING: This file is automatically generated!*/'
  sed -e'/^ifobjc$/,/^end ifobjc$/d' -e'/^ifc$/d' -e'/^end ifc$/d' ${SRCDIR}/c-parse.in
) > c-parse.y
bison -d c-parse.y -o c-parse.c )
    then
        cvs commit -m"Regenerated source" ${TARGET}
    else
        echo "Could not rebuild ${TARGET}"
        cvs unedit ${TARGET}
    fi
    cd ${HOMEDIR}
fi

[-- Attachment #4: submit.tpl --]
[-- Type: text/plain, Size: 1276 bytes --]

[= autogen template -*- Mode: shell-script -*-
sh
=]
#!/bin/sh
#
[=_eval "##  " _dne =]

HOMEDIR=`pwd`[=
_FOR rule "\n\n" =][=


_IF dir _get "NEWDIR='%s' ; if [ \"$LASTDIR\" = \"$NEWDIR\" ]
then echo 0 ; else LASTDIR=\"$NEWDIR\" ; echo 1 ; fi"
_printf _shell 0 >

 =]
SRCDIR=`cd egcs/[=dir=] ; pwd`

test -d egcsbld/[=dir=]/CVS || {
    echo "The directory ${HOMEDIR}/egcsbld/[=dir=] must be added to CVS"
    exit 1
}[=
_ENDIF
=]
TARGET="[=_FOR target " "=][=target=][=/target=]"

for f in ${TARGET} ; do
    if [ ! -f egcsbld/[=dir=]/$f ] ; then
        touch 0101000080 egcsbld/[=dir=]/$f
        cvs add egcsbld/[=dir=]/$f
    fi
done > /dev/null 2>&1

if  ([ [=
    _FOR target "-o " =][=
        _FOR depend "-o "
             =]egcs/[=dir=]/[=depend=] -nt egcsbld/[=dir=]/[=target=] \
    [=  /depend=][=
    /target =]]) > /dev/null 2>&1
then
    cd egcsbld/[=dir=]
    cvs edit ${TARGET} > /dev/null 2>&1
    cat <<-	'_EOF_'
	Rebuilding[=_FOR target ,=] [=
            _IF _index 0 > _last & =]and [=
            _ENDIF =][=target=][=/target=]
	[=script=]
	_EOF_

    if ( [=script=] )
    then
        cvs commit -m"Regenerated source" ${TARGET}
    else
        echo "Could not rebuild ${TARGET}"
        cvs unedit ${TARGET}
    fi
    cd ${HOMEDIR}
fi[=
/rule=]

RE: Can we remove bison output from cvs?

[Jan Reimers]

> ----------
> From: 	Joern Rennecke[SMTP:amylaar@cygnus.co.uk]
> Sent: 	Thursday, January 14, 1999 2:34 PM
> To: 	ddsinc09@ix.netcom.com
> Cc: 	pfeifer@dbai.tuwien.ac.at; egcs@cygnus.com; bothner@cygnus.com;
> ghazi@caip.rutgers.edu
> Subject: 	Re: Can we remove bison output from cvs?
> 
> > Can the repository be split?  I.e.  the main repository contains the
> > hand crafted text files and a secondary repository contains derived
> text
> > for people without the "proper" tools.  Releases would, of course,
> > have to contain a merge of the trees.  For anyone to build without
> > all the development tools, they would have to cvs-get both trees and
> > do a symlink merge a la "mkmerge" or  a fairly straight forward
> script.
> 
> can handle multiple modules which may overlap in directory trees.  So
> if things are handled that way, you'd just check out a different
> module
> to get the generated files alongside with the source.
> The main cost is checking in the generated files whenever the source
> files
> are changed.
> Although it should be possible to do this automatically on the server
> side, but that'll need someone to tinker with the configuration to get
> it running.
Could this be extended to make chill/java/objc/testsuites optional for
CVS checkout?

JR

Re: Can we remove bison output from cvs?

[Jeffrey A Law]

  In message < 71B30885B657D111809D080009EEBBF34FE6C7@MAILSERV.molienergy.bc.ca >
you write:
  > Could this be extended to make chill/java/objc/testsuites optional for
  > CVS checkout?
You can already do that if someone fixes the cvs modules description.  
Submissions would be welcome.

jeff

Re: Can we remove bison output from cvs?

[Bruce Korb]

Gerald Pfeifer wrote:
> 
> On Wed, 13 Jan 1999, Per Bothner wrote:
> > The cvs archive is primarily for developers, plus people who want to
> > be on the bleeding edgs.  In both cases, I think it is reasonable
> > to require that people have bison+makeinfo installed.
> 
> I believe that requiring bison, and perhaps makeinfo as well, should not
> be that much of a problem.
> 
> However, please do not make it a requirement to install various additional
> packages like auto* for CVS users, both for reasons of disk space on guest
> accounts and time to install these tools on various platforms.

Can the repository be split?  I.e.  the main repository contains the
hand crafted text files and a secondary repository contains derived text
for people without the "proper" tools.  Releases would, of course,
have to contain a merge of the trees.  For anyone to build without
all the development tools, they would have to cvs-get both trees and
do a symlink merge a la "mkmerge" or  a fairly straight forward script.

Re: Can we remove bison output from cvs?

[Per Bothner]

> However, it may still be a bad idea to remove configure from the repo since
> you need configure to build a Makefile.

Yes, let us start with removing the files that can be re-generated with make:
bison output, info files, maybe [cg]perf files.  Expect for the last one, this
matches what we have done in the Cygnus tree, so we know it works well.

Longer-term, we may consider removing configure generated by autoconf
from configure.in, and Makefile.in generated by automake from Makefile.am.
One intermediate step:  For directories using automake, we can modify
configure so that it runs automake if Makefile.in is missing.

(Even more drastic/long-term:  I would like to move away from "configure+make+
make install" model to a single standard GNU script "build" that can take a
source tree, run automake, aclocal, autoconf, configure, make, make install,
depending on options to "build".  It could even work on a URL or local
filename and download+unpack it.  And on systems with a package database
(such as Debian or RedHat Linux) it could optionally update the database.)

> The other issue at hand is where are the parser files built?
>
> Then can be built in the build directory.  That's how Cygnus does things, but
> I don't want to force that convention on anyone.

I do.  That was in my original message.

> The downside here is that
> cvs folks and snapshot/release folks would be using different procedures since
> the snapshots & releases need to continue to include the generated files. 

Not really.  Many GNU packages handle this already.  automake supports "make dist",
which generates a tar file *after* building the needed generated files.
(Automake has a --cygnus option which among other things places the
generated parser files in the build directory, rather than the source directory,
but either way they are included in the tar files.)  Some Cygnus GNU packages
use the "taz" rule in the top-leval Makefile.in in Cygnus releases.  They
include (or at least used to) gdb and binutils.

	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner

Re: Can we remove bison output from cvs?

[Marc Espie]

In article < 199901102154.NAA09424@atrus.synopsys.com > you write:
>
>> Joe Buck wrote:
>> > This would mean that we can't do the same for configure files until
>> > we get a new autoconf release (since we are currently relying
>> > on a snapshot here).
>> 
>> Does the new autoconf not work?

>The autoconf we are currently using is a CVS version.  The latest FSF
>version doesn't work.  That's what I meant.  I don't think it's
>desirable to require everyone to deal with this -- it's easy enough
>to tell folks "you must have bison version x.xx", but with CVS it's
>more likely that people will wind up with different versions of
>the configure files.

I've been changing configure.in and rebuilding configure with  
autoconf 2.13, the new released version, without noticing any ill effects...

While you're at it, how about getting rid of texinfo from the cvs repository ?
I haven't noticed any problems with texinfo 3.12...

RE: Can we remove bison output from cvs?

[Jan Reimers]

> ----------
> From: 	Jeffrey A Law[SMTP:law@hurl.cygnus.com]
> Reply To: 	law@cygnus.com
> Sent: 	Thursday, January 14, 1999 4:49 PM
> To: 	Jan Reimers
> Subject: 	Re: Can we remove bison output from cvs? 
> 
> 
> 
>   In message
> <71B30885B657D111809D080009EEBBF34FE6C9@MAILSERV.molienergy.bc.ca>
> you write:
>   > OK, I'll try and check out the modules file tonight and make some
> mods.
>   > I'll try to test with a mock up repository on my disk, but someone
> with
>   > write access will have to do the real commit/test.
> As far as I know we don't currently use the modules at all, so don't
> worry
> about breaking anything.
> 
> Send the result to egcs-patches@cygnus.com
> 
> Fixing this would be a big win...
> 
> Thanks!  
> 
> jeff
> 
It seems CVS 1.10 does not support what we need to do in the modules
file, namely:

#start CVSROOT/modules
auto_generated_files -a {big list of auto generated files}

egcs_noauto -a !auto_generated_files egcs
#start CVSROOT/modules

the cvs modules parsing code does support the ignore directive, but ONLY
for directories, NOT for aliases (as I show above) or for individual
file names.  I didn't bother looking and non-release versions of cvs
like 1.10.{1->4} since I suspect few if any egcs users will be running
these versions.
	Anybody know what the cvs developers are up to in the area?
Would they accept a patch from an unknown?
	As for checking out things like egcs-core, egcs-g++, egcs-java
etc. I can try and set that up if it is useful.

________________________
                                                  
Jan N. Reimers (Ph.D.)
Manager, Materials Research 
NEC Moli Energy (Canada) Ltd.
________________________ 

Re: Can we remove bison output from cvs?

[Joern Rennecke]

> Can the repository be split?  I.e.  the main repository contains the
> hand crafted text files and a secondary repository contains derived text
> for people without the "proper" tools.  Releases would, of course,
> have to contain a merge of the trees.  For anyone to build without
> all the development tools, they would have to cvs-get both trees and
> do a symlink merge a la "mkmerge" or  a fairly straight forward script.

can handle multiple modules which may overlap in directory trees.  So
if things are handled that way, you'd just check out a different module
to get the generated files alongside with the source.
The main cost is checking in the generated files whenever the source files
are changed.
Although it should be possible to do this automatically on the server
side, but that'll need someone to tinker with the configuration to get
it running.

Re: Can we remove bison output from cvs?

[Marc Lehmann]

On Thu, Jan 14, 1999 at 02:41:20PM +0100, Gerald Pfeifer wrote:
> However, please do not make it a requirement to install various additional
> packages like auto* for CVS users, both for reasons of disk space on guest
> accounts and time to install these tools on various platforms.
> 

Well, I don't think people that don't want to install these tools need cvs
snapshots. The autogenerated files cause grief with no end on my side, they
very often cause rejects (at least for me), when I don't have the exact same
version of some tool as was used to build the file in the repository (given
that the output even depends on the machine where they were build)

The files that most often cause problems are the *info files and bison
output, with configure seldom causing big problems.

Removing these generated files from cvs (ONLY!) would not only solve these
problems, its also the same way as almost every other project I know of (say,
gimp, gnome and many gnu tools with cvs repository) handles this.


      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com      |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |

Can we remove bison output from cvs?

[Per Bothner]

This has been discussed before, and I don't recall any real objections.

I would like to propose that we remove bison-generated .c and .h
files from the cvs repository.  They would still be in the release
and snapshot tar-balls, generated by whatever script makes those
tar fils.  That way people who just want to build and install gcc
can do so without having to first install bison.  However, I see
no reason why we should not expect people building from cvs to
have bison installed.

There are a couple of tricky aspects to keep in mind:

When parse.c is built, it should be left in the build
directory, not the $(srcdir).  After all, it is a
generated file, not a source file, and we want to
allow people to have their $(srcdir) be on a read-only
filesystem.  Ok, that may be unlikely for people
building from a checked-out cvs tree.  Still, that
is how Cygnus has done it for years, it works, it is
clean, it is Right.

When building Net releases, these should (following GNU standards)
include generated bison output files.  In this case, the files
will end up in the $(srcdir).  One way to do that is for the
tar-file-generating script to build parse.c using srcdir="." .

The same Makefile has to work for both those who have
parse.c in the $(srcdir) (because they are building from
release tar files) and those who have parse.c in the
build directory (because they built from a cvs-based
source tree, and so built parse.c themselves).
By leaving the directory of parse.c unspecified, we can
let the make VPATH search mechanism take care of it.

Here is a patch for java/Makefile.in showing the proposed
change.  It may get a little hairier when bison is also
set to generate a parse.h file, but the basic idea is the same.

Any objections or comments?

Index: Makefile.in
===================================================================
RCS file: /egcs/carton/cvsfiles/egcs/gcc/java/Makefile.in,v
retrieving revision 1.22
diff -u -r1.22 Makefile.in
--- Makefile.in	1999/01/06 21:09:59	1.22
+++ Makefile.in	1999/01/10 17:09:05
@@ -223,24 +223,15 @@
         $(srcdir)/../machmode.h $(srcdir)/../machmode.def
 EXPR_H = $(srcdir)/../expr.h ../insn-codes.h
 
-# Separating PARSE_DIR from PARSE_RELDIR lets us easily change the
-# code to support building parse.c in the build directory, at some
-# expense in readability.
-PARSE_DIR = $(srcdir)
-PARSE_RELDIR = .
-PARSE_C = $(PARSE_DIR)/parse.c
-PARSE_SCAN_C = $(PARSE_DIR)/parse-scan.c
-PARSE_H = $(srcdir)/parse.h
-
-$(PARSE_C):  $(srcdir)/parse.y $(srcdir)/lex.c $(PARSE_H) $(srcdir)/lex.h
+parse.c:  $(srcdir)/parse.y $(srcdir)/lex.c $(srcdir)/parse.h $(srcdir)/lex.h
 	$(SET_BISON); \
-	cd $(PARSE_DIR) && $$bison -t -v $(BISONFLAGS) $(JAVABISONFLAGS) \
-	    -o parse.c $(PARSE_RELDIR)/parse.y
-$(PARSE_SCAN_C):  $(srcdir)/parse-scan.y $(srcdir)/lex.c $(PARSE_H) \
+	$$bison -t -v $(BISONFLAGS) $(JAVABISONFLAGS) \
+	    -o parse.c $(srcdir)/parse.y
+parse-scan.c:  $(srcdir)/parse-scan.y $(srcdir)/lex.c $(srcdir)/parse.h \
 	  $(srcdir)/lex.h
 	$(SET_BISON); \
-	cd $(PARSE_DIR) && $$bison -t -v $(BISONFLAGS) -o parse-scan.c \
-	    $(PARSE_RELDIR)/parse-scan.y
+	$$bison -t -v $(BISONFLAGS) -o parse-scan.c \
+	    S(srcdir)/parse-scan.y
 
 lex.c: keyword.h lex.h
 
	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner

Re: Can we remove bison output from cvs?

[Jeffrey A Law]

  In message < 369956F2.649575E5@tssc.co.nz >you write:
  > Is the same feasable for auto* generated files?  I doen't see the need
  > for *any* generated files to be included in CVS (gperf etc included). 
I think we can generally work in that direction over time.  However, some
files like configure will probably stay in the repository.

It's a matter of what makes the CVS repo easiest to use for the largest group
of people.  Being able to configure and build without retrieving gperf,
bison, autoconf, automake, autogen, m4 (must use gnu-m4 for autoconf), texinfo
etc is a good thing.

  > NOTE: there should probly be a prominent list of all required utils/libs
  > and there versions.
Yes.  Feel free to start one.  We should put it on the web page.

jeff

Re: Can we remove bison output from cvs?

[Joe Buck]

> I would like to propose that we remove bison-generated .c and .h
> files from the cvs repository.  They would still be in the release
> and snapshot tar-balls, generated by whatever script makes those
> tar fils.

I second the motion, with the following caveat: we can remove a
derived file from CVS if and only if --

* we are relying on a standard FSF-released version of a tool (bison
  in this case) to build the derived file;

* we specify, at any time, which version of this tool is to be used
  to make the file (so that we are all testing the same code).

This would mean that we can't do the same for configure files until
we get a new autoconf release (since we are currently relying
on a snapshot here).

Given this, it's a good thing, as small changes to the .y file
can generate large changes to parse.c (major changes to the tables),
and it's a waste of network bandwidth for everyone to suck down the
change.

Re: Can we remove bison output from cvs?

[Kaveh R. Ghazi]

 > From: Per Bothner <bothner@cygnus.com>
 >  
 > > I don't have the luxury of getting the sysadmin to install the latest
 > > copy of packageX and don't want to maintain these in my home directory. 
 > > Building egcs already takes up enough space without having 10 other
 > > tools installed in my home dir. 
 >  
 > In that case, test snapshots (which will have the generated files
 > in them), rather than the instanteous state of the cvs archive.
 > Testing the cvs archive is useful to quickly catch new bugs,
 > but in terms of QA it is better to test the snapshots, because it
 > tests what the users will downloading.
 >  
 > The cvs archive is primarily for developers, plus people who want to
 > be on the bleeding edgs.  In both cases, I think it is reasonable
 > to require that people have bison+makeinfo installed.



	It may be reasonable but its not necessary.  What is the benefit
derived which outweighs the simplicity of having the generated files
there already?

Looking back at http://www.cygnus.com/ml/egcs/1999-Jan/0299.html you
only propose doing it, you never state how it would improve things. 




 > > (Again this assumes I don't have the ability to upgrade the system
 > > bison or the quota to maintain my own copy.)
 >  
 > You have a quota big enough for gcc, but not big enough for
 > gcc+bison?  I'm sorry, but that does not make sense.  Add a
 > couple more gcc toolchains, and you will no longer have room
 > for gcc.
 >         --Per Bothner


	No, think of bison+gperf+autoconf+automake+gnum4+texinfo+etc. 
If bison files are okay to remove we've already seen there's a bandwagon
waiting to remove other generated files. 

Then multiply by five platforms, (all five share the same NFS home
directory in my case.) I'm already maintaining a private dejagnu
snapshot for each one and that's 5 * 16Mb right there.  I don't want to
install 5 * (10 other tools) on top of that. 

Then think hard quota vs soft quota.  My hard quota is much larger than
my soft one.  I remove snapshot builds daily.  So the 7 day limit for
exceeding the soft quota never comes into play.  I would have to keep
these tools permanently so its charged against the lower soft quota
amount. 

	Hopefully this makes clear my issue is not just the space for
gcc+bison. :-)  Your proposal would hose me (and I suspect others)
considerably.  Please don't do it. 

		--Kaveh
--
Kaveh R. Ghazi			Engagement Manager / Project Services
ghazi@caip.rutgers.edu		Icon CMT Corp.

Re: Can we remove bison output from cvs?

[Kaveh R. Ghazi]

 > From: Jeffrey A Law <law@hurl.cygnus.com>
 >  
 >   In message < 369956F2.649575E5@tssc.co.nz >you write:
 >   > Is the same feasable for auto* generated files?  I doen't see the need
 >   > for *any* generated files to be included in CVS (gperf etc included). 
 > I think we can generally work in that direction over time.  However, some
 > files like configure will probably stay in the repository.
 >  
 > It's a matter of what makes the CVS repo easiest to use for the largest group
 > of people.  Being able to configure and build without retrieving gperf,
 > bison, autoconf, automake, autogen, m4 (must use gnu-m4 for autoconf), texinfo
 > etc is a good thing.
 > jeff

	Agreed!

I'm able to test multiple platforms by mooching off of guest accounts. :-)

I don't have the luxury of getting the sysadmin to install the latest
copy of packageX and don't want to maintain these in my home directory. 
Building egcs already takes up enough space without having 10 other
tools installed in my home dir. 

You also have the problem of not having the right version of packageX
installed when you build.  If my system has bison < 1.25 installed, then
I am not testing what will eventually be the release.  (Again this
assumes I don't have the ability to upgrade the system bison or the
quota to maintain my own copy.)


	If its not already too late to stop this train (wreck), please
let's NOT remove generated files from the repo. 


		--Kaveh
--
Kaveh R. Ghazi			Engagement Manager / Project Services
ghazi@caip.rutgers.edu		Icon CMT Corp.

Re: Can we remove bison output from cvs?

[Per Bothner]

> Looking back at http://www.cygnus.com/ml/egcs/1999-Jan/0299.html you
> only propose doing it, you never state how it would improve things. 

Because the cvs archive should contain *source* files.
Files that can be generated should in principle not be there.
We may make some exceptions when it makes things easier,
but the general philosophy is that cvs should not contain
generated files.

Another lesser reason:  It would slightly simplify my life.
The current setup is inconsistent with how Cygnus maintains our
internal cvs tree.  Thus I need to do different things when
checking things into the internal tree and the Cygnus tree,
increasing the likelyhood of mistakes.  Since a large chunk
of Gcc work is done by Cygnus engineers, this is not just
for me.

> I remove snapshot builds daily.

But nobody is proposing removing the generated files from the
snapshots, only from the cvs archive!  If you want to test
the "instanteous" snapshots from the cvs archive, I think
it would make a lot of sense to have a main machine with cvs,
check it out, build a snapshot tarball from that (using
whatever script Jeff uses), and copy that tarball around
to your various machines.

Note:  Thus means we should also release a script that
produces a snapshot from the cvs source tree.  This should
ideally be a "dist" rule in the Makefile, for compatibility
with automake.

	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner

Re: Can we remove bison output from cvs?

[David Starner]

Joe Buck wrote:
> This would mean that we can't do the same for configure files until
> we get a new autoconf release (since we are currently relying
> on a snapshot here).

Does the new autoconf not work?

-- 
David Starner - dstarner98@aasaa.ofe.org
"There's another side of heaven this way - to technical paradise" -
Black Sabbath, "Computer God"

Re: Can we remove bison output from cvs?

[Gerald Pfeifer]

On Wed, 13 Jan 1999, Per Bothner wrote:
> The cvs archive is primarily for developers, plus people who want to
> be on the bleeding edgs.  In both cases, I think it is reasonable
> to require that people have bison+makeinfo installed.

I believe that requiring bison, and perhaps makeinfo as well, should not
be that much of a problem.

However, please do not make it a requirement to install various additional
packages like auto* for CVS users, both for reasons of disk space on guest
accounts and time to install these tools on various platforms.

This would be a big loss for people like Kaveh and myself. Moreover, even
if one is interested only in using snapshots, how about those using snap-
shots via CVS?

Gerald
-- 
Gerald Pfeifer (Jerry)      Vienna University of Technology
pfeifer@dbai.tuwien.ac.at   http://www.dbai.tuwien.ac.at/~pfeifer/

Re: Can we remove bison output from cvs?

[Jeffrey A Law]

  In message < 71B30885B657D111809D080009EEBBF34FE6D2@MAILSERV.molienergy.bc.ca >
you write:
  > It seems CVS 1.10 does not support what we need to do in the modules
  > file, namely:
  > 
  > #start CVSROOT/modules
  > auto_generated_files -a {big list of auto generated files}
  > 
  > egcs_noauto -a !auto_generated_files egcs
  > #start CVSROOT/modules
  > 
  > the cvs modules parsing code does support the ignore directive, but ONLY
  > for directories, NOT for aliases (as I show above) or for individual
  > file names.
Bummer.

  > I didn't bother looking and non-release versions of cvs
  > like 1.10.{1->4} since I suspect few if any egcs users will be running
  > these versions.
This kind of functionality would probably be server side only, which wouldn't
require an update of the end-user client.  We need to upgrade anyway, so if
some new version has such a feature, it's just one more reason to bite the
bullet and upgrade cvs on the server.

  > 	Anybody know what the cvs developers are up to in the area?
  > Would they accept a patch from an unknown?
Dunno.  You might head over to cyclic's home page and see if they mention
anything interesting.

  > 	As for checking out things like egcs-core, egcs-g++, egcs-java
  > etc. I can try and set that up if it is useful.
Yes, I think this would still be very useful

jeff

Re: Can we remove bison output from cvs?

[Kaveh R. Ghazi]

 > From: Per Bothner <bothner@cygnus.com>
 >  
 > > I remove snapshot builds daily.
 >  
 > But nobody is proposing removing the generated files from the
 > snapshots, only from the cvs archive!  

	I know.  By snapshot, I meant cvs ones.  I rebuild cvs checkouts
every night.  No need to rebuild a weekly tarball snapshot every night. :-)


 > If you want to test
 > the "instanteous" snapshots from the cvs archive, I think
 > it would make a lot of sense to have a main machine with cvs,
 > check it out, build a snapshot tarball from that (using
 > whatever script Jeff uses), and copy that tarball around
 > to your various machines.


	Okay that's fair.  If I use the same script Jeff uses to build
tarballs then I'm testing the same thing via cvs as what is released.



 > Note:  Thus means we should also release a script that
 > produces a snapshot from the cvs source tree.  This should
 > ideally be a "dist" rule in the Makefile, for compatibility
 > with automake.
 >         --Per Bothner


	If its a "dist" rule, then you have a circular dependency
situation where you have to configure egcs to create Makefile to call
the dist rule to create the generated files, one of which is configure
itself which is needed to configure egcs which...

	I think instead of (or in addition to) a dist rule, this should
be an external shell script in the contrib directory.  Then it could be
called by the egcs_update script which does the cvs checkout.  The
egcs_update script currently "touches" the generated files to make sure
the timestamps are correct.  It could instead call the "generator"
script to actually create the files. 

		--Kaveh
--
Kaveh R. Ghazi			Engagement Manager / Project Services
ghazi@caip.rutgers.edu		Icon CMT Corp.

Re: Can we remove bison output from cvs?

[N8TM]

In a message dated 1/10/99 2:04:11 PM Pacific Standard Time,
law@hurl.cygnus.com writes:

<< Has anyone tried 2.13 yet?   It's sitting on prep (told you they were close
 to a new release :-)
  >>
I have used autoconf-2.13 on various packages under cygwin-b20.1, including
egcs-19990103 under both NT and W95, and it works better than any previous
version.

Re: Can we remove bison output from cvs?

[Joe Buck]

> Joe Buck wrote:
> > This would mean that we can't do the same for configure files until
> > we get a new autoconf release (since we are currently relying
> > on a snapshot here).
> 
> Does the new autoconf not work?

The autoconf we are currently using is a CVS version.  The latest FSF
version doesn't work.  That's what I meant.  I don't think it's
desirable to require everyone to deal with this -- it's easy enough
to tell folks "you must have bison version x.xx", but with CVS it's
more likely that people will wind up with different versions of
the configure files.

Re: Can we remove bison output from cvs?

[Bill Currie]

Is the same feasable for auto* generated files?  I doen't see the need
for *any* generated files to be included in CVS (gperf etc included). 
For auto*, doing what the gtk/gnome people do seems to work (ie have an
`autogen.sh' script in the top level directory that runs aclocal,
automake and autoconf.  Their autogen.sh also runs configure, but I'm
not so sure that's a good idea.  As Per says, developers should have all
tools available anyway.

NOTE: there should probly be a prominent list of all required utils/libs
and there versions.

Bill
-- 
Leave others their otherness

Re: Can we remove bison output from cvs?

[Bruce Korb]

Joern Rennecke wrote:
> 
> > Can the repository be split?  I.e.  the main repository contains the
> > hand crafted text files and a secondary repository contains derived text
> > for people without the "proper" tools.  Releases would, of course,
> > have to contain a merge of the trees.  For anyone to build without
> > all the development tools, they would have to cvs-get both trees and
> > do a symlink merge a la "mkmerge" or  a fairly straight forward script.
> 
> can handle multiple modules which may overlap in directory trees.  So
> if things are handled that way, you'd just check out a different module
> to get the generated files alongside with the source.
> The main cost is checking in the generated files whenever the source files
> are changed.
> Although it should be possible to do this automatically on the server
> side, but that'll need someone to tinker with the configuration to get
> it running.


Gee.  I have had to generate and checkin all that stuff anyways. :)
It *would* be nice to automate it!!!  (see the gcc/fixinc/ directory)

Re: Can we remove bison output from cvs?

[Jeffrey A Law]

  In message < 199901132225.RAA24116@caip.rutgers.edu >you write:
  > 	Okay that's fair.  If I use the same script Jeff uses to build
  > tarballs then I'm testing the same thing via cvs as what is released.
You don't even want to look at those scripts.  They will burn your eyes
and melt your brain.

I rewrote the release script a while back to that it's only mildly revolting.
That rewrite will form the basis of the new snapshot scripts...    I'd like
to check both of them into the contrib repo.  Not that I expect anyone to work
on them, but you never know.

jeff

Re: Can we remove bison output from cvs?

[Mike Stump]

> Date: Sun, 10 Jan 1999 09:27:05 -0800
> From: Per Bothner <bothner@cygnus.com>

> This has been discussed before, and I don't recall any real objections.

While not in the exact form I was thinking, it does cover all the
bases I can think of right now.  My last proposal on the matter I
don't believe was dependent upon VPATH like this change.  All in all,
I support this change.

Re: Can we remove bison output from cvs?

[Jeffrey A Law]

  In message < 199901102154.NAA09424@atrus.synopsys.com >you write:
  > 
  > > Joe Buck wrote:
  > > > This would mean that we can't do the same for configure files until
  > > > we get a new autoconf release (since we are currently relying
  > > > on a snapshot here).
  > > 
  > > Does the new autoconf not work?
  > 
  > The autoconf we are currently using is a CVS version.  The latest FSF
  > version doesn't work.  That's what I meant.  I don't think it's
  > desirable to require everyone to deal with this -- it's easy enough
  > to tell folks "you must have bison version x.xx", but with CVS it's
  > more likely that people will wind up with different versions of
  > the configure files.
Has anyone tried 2.13 yet?   It's sitting on prep (told you they were close
to a new release :-)

jeff

Re: Can we remove bison output from cvs?

[Per Bothner]

> I don't have the luxury of getting the sysadmin to install the latest
> copy of packageX and don't want to maintain these in my home directory. 
> Building egcs already takes up enough space without having 10 other
> tools installed in my home dir. 

In that case, test snapshots (which will have the generated files
in them), rather than the instanteous state of the cvs archive.
Testing the cvs archive is useful to quickly catch new bugs,
but in terms of QA it is better to test the snapshots, because it
tests what the users will downloading.

The cvs archive is primarily for developers, plus people who want to
be on the bleeding edgs.  In both cases, I think it is reasonable
to require that people have bison+makeinfo installed.

> (Again this assumes I don't have the ability to upgrade the system
> bison or the quota to maintain my own copy.)

You have a quota big enough for gcc, but not big enough for
gcc+bison?  I'm sorry, but that does not make sense.  Add a
couple more gcc toolchains, and you will no longer have room
for gcc.

	--Per Bothner
Cygnus Solutions     bothner@cygnus.com     http://www.cygnus.com/~bothner

Re: Can we remove bison output from cvs?

[Jeffrey A Law]

  In message < 199901102053.MAA09101@atrus.synopsys.com >you write:
  > 
  > > I would like to propose that we remove bison-generated .c and .h
  > > files from the cvs repository.  They would still be in the release
  > > and snapshot tar-balls, generated by whatever script makes those
  > > tar fils.
  > 
  > I second the motion, with the following caveat: we can remove a
  > derived file from CVS if and only if --
  > 
  > * we are relying on a standard FSF-released version of a tool (bison
  >   in this case) to build the derived file;
I believe that is the case for bison (and autoconf now that 2.13 has hit
the streets).


  > * we specify, at any time, which version of this tool is to be used
  >   to make the file (so that we are all testing the same code).
I seem to have bison-1.25 installed on my boxes.  I don't know if that is
the preferred version or not.

  > This would mean that we can't do the same for configure files until
  > we get a new autoconf release (since we are currently relying
  > on a snapshot here).
No longer true.  autoconf-2.13 has been released.

However, it may still be a bad idea to remove configure from the repo since
you need configure to build a Makefile.

  > Given this, it's a good thing, as small changes to the .y file
  > can generate large changes to parse.c (major changes to the tables),
  > and it's a waste of network bandwidth for everyone to suck down the
  > change.
Yup.  More importantly, it reduces the chance of someone forgetting to check
in the output file and folks getting inconsistent sources.

The other issue at hand is where are the parser files built?

Then can be built in the build directory.  That's how Cygnus does things, but
I don't want to force that convention on anyone.  The downside here is that
cvs folks and snapshot/release folks would be using different procedures since
the snapshots & releases need to continue to include the generated files. 


The other possibility is to build them into the source tree.  The downside here
is cvs update will start complaining about the parser files since they won't
be in the repository.

Jeff

Re: Can we remove bison output from cvs?

[Joern Rennecke]

> I don't have the luxury of getting the sysadmin to install the latest
> copy of packageX and don't want to maintain these in my home directory. 
> Building egcs already takes up enough space without having 10 other
> tools installed in my home dir. 
> 
> You also have the problem of not having the right version of packageX
> installed when you build.  If my system has bison < 1.25 installed, then
> I am not testing what will eventually be the release.  (Again this
> assumes I don't have the ability to upgrade the system bison or the
> quota to maintain my own copy.)

I think a bison binary is much smnaller than you average bison-generated file,
so considering that you frequently see extra copies of the parser files when
you have a conflict (or rather, you don't see them unless you are looking
for them, because the filenames start with .# ), you should be really
better off without the bison-generated files in the repository.
Now, the other generated files are a whole different matter, indeed.
To get gperf compiled, you first have to install a c++ compiler
(cperf is no alternative when the reason to install your own gperf is that
egcs needs the bleeding edge of gperf), and autoconf and m4 are quite
large, too.