OT: How do people set up long term development

OT: How do people set up long term development

[Michael Meissner]

I am wondering how people outside of Red Hat do long term development of
compiler tools when you can't check the machine specific sources into the FSF
repository until the NDA on the machine has been lifted?  The intention is that
when the chipset I am developing the GNU toolchain for is formally announced, I
will submit the files to the FSF for consideration of inclusion in the
releases, and non-machine specific changes that I make will be contributed as
they are written.

I can see three different development models:

1) Create my own CVS repository of a checked out releases, and check the files
   into there.  Every so often, I will do a merge from the FSF sources to my
   own repository.  This is how we did development at OSF, Cygnus, and Red Hat.
   The problem is doing the merge step, and there isn't a direct correlation
   between the versions in the master repository and my system.  If all of the
   tools had CVS tags for nightly snapshots, it would make things easier, but
   the bintuils directories don't seem to have snapshot tags.  This is the way
   I'm leaning, since it will allow me to work on two systems (desk system,
   laptop) and using cvs to do the update.

2) Doing the work in a live tree, possibly using a CVS repository to hold the
   newly created files with symlinks.  CVS update would do the merges
   automatically, but it is hairier to work with on a day to day basis.

3) I could imagine an alternative to #1 of downloading the rcs files from the
   repository to use as a starting base, instead of creating the files from
   scratch.  This may make the eventual merge easier, but intermediate merges
   may be more difficult.

-- 
Michael Meissner
email: gnu@the-meissners.org
http://www.the-meissners.org

Re: OT: How do people set up long term development

[Jakub Jelinek]

On Wed, May 28, 2003 at 11:02:11AM -0400, Michael Meissner wrote:
> I am wondering how people outside of Red Hat do long term development of
> compiler tools when you can't check the machine specific sources into the FSF
> repository until the NDA on the machine has been lifted?  The intention is that
> when the chipset I am developing the GNU toolchain for is formally announced, I
> will submit the files to the FSF for consideration of inclusion in the
> releases, and non-machine specific changes that I make will be contributed as
> they are written.
> 
> I can see three different development models:
> 
> 1) Create my own CVS repository of a checked out releases, and check the files
>    into there.  Every so often, I will do a merge from the FSF sources to my
>    own repository.  This is how we did development at OSF, Cygnus, and Red Hat.
>    The problem is doing the merge step, and there isn't a direct correlation
>    between the versions in the master repository and my system.  If all of the
>    tools had CVS tags for nightly snapshots, it would make things easier, but
>    the bintuils directories don't seem to have snapshot tags.  This is the way
>    I'm leaning, since it will allow me to work on two systems (desk system,
>    laptop) and using cvs to do the update.

cvs update -jgcc-3_3-branch:2003{0517,0528} works IMHO just fine, you don't
need any nightly tags for it. All you need is to remember what was the last
date you did a cvs merge at, which can be recorded e.g. in the CVS commit
messages.

	Jakub

Re: OT: How do people set up long term development

[Michael Meissner]

On Wed, May 28, 2003 at 05:07:19PM +0200, Jakub Jelinek wrote:
> On Wed, May 28, 2003 at 11:02:11AM -0400, Michael Meissner wrote:
> > I am wondering how people outside of Red Hat do long term development of
> > compiler tools when you can't check the machine specific sources into the FSF
> > repository until the NDA on the machine has been lifted?  The intention is that
> > when the chipset I am developing the GNU toolchain for is formally announced, I
> > will submit the files to the FSF for consideration of inclusion in the
> > releases, and non-machine specific changes that I make will be contributed as
> > they are written.
> > 
> > I can see three different development models:
> > 
> > 1) Create my own CVS repository of a checked out releases, and check the files
> >    into there.  Every so often, I will do a merge from the FSF sources to my
> >    own repository.  This is how we did development at OSF, Cygnus, and Red Hat.
> >    The problem is doing the merge step, and there isn't a direct correlation
> >    between the versions in the master repository and my system.  If all of the
> >    tools had CVS tags for nightly snapshots, it would make things easier, but
> >    the bintuils directories don't seem to have snapshot tags.  This is the way
> >    I'm leaning, since it will allow me to work on two systems (desk system,
> >    laptop) and using cvs to do the update.
> 
> cvs update -jgcc-3_3-branch:2003{0517,0528} works IMHO just fine, you don't
> need any nightly tags for it. All you need is to remember what was the last
> date you did a cvs merge at, which can be recorded e.g. in the CVS commit
> messages.

Thanks.

Wouldn't it have problems if there was a commit that occured around midnight,
particularly a multi-file commit, one file being committed before midnight, and
the other after midnight.

-- 
Michael Meissner
email: gnu@the-meissners.org
http://www.the-meissners.org

Re: OT: How do people set up long term development

[Tom Lord]

    >>> I am wondering how people outside of Red Hat do long term
    >>> development of compiler tools when you can't check the machine
    >>> specific sources into the FSF repository until the NDA on the
    >>> machine has been lifted? 

This page:

     http://arch.fifthvision.net/bin/view/Main/InteroperatingWithCVS

describes an approach in which you `cvs update' from the central
repository for merging in FSF sources, but use arch privately.  arch
is _ideal_ for working on on two systems ("desktop, laptop") and nice
for other reasons as well.

I'm not sure I recommend this approach, quite yet, for GCC.   The size
of the GCC tree is, I suspect, right around the edge of
"uncomfortable" for the current release of arch.   Maybe not -- try it
out if you are so inclined -- but also be aware of the upcoming
quantum-leap in arch performance with the new C version (see
=READEME.new-stuff) in the distro.

(arch's "home" is http://regexps.srparish.net/www)

-t







    > From: Michael Meissner <gcc-mail@the-meissners.org>

    > On Wed, May 28, 2003 at 05:07:19PM +0200, Jakub Jelinek wrote:
    >> On Wed, May 28, 2003 at 11:02:11AM -0400, Michael Meissner wrote:

    >>> I am wondering how people outside of Red Hat do long term
    >>> development of compiler tools when you can't check the machine
    >>> specific sources into the FSF repository until the NDA on the
    >>> machine has been lifted?  The intention is that when the
    >>> chipset I am developing the GNU toolchain for is formally
    >>> announced, I will submit the files to the FSF for
    >>> consideration of inclusion in the releases, and non-machine
    >>> specific changes that I make will be contributed as they are
    >>> written.

    >>> I can see three different development models:

    >>> 1) Create my own CVS repository of a checked out releases, and
    >>> check the files into there.  Every so often, I will do a merge
    >>> from the FSF sources to my own repository.  This is how we did
    >>> development at OSF, Cygnus, and Red Hat.  The problem is doing
    >>> the merge step, and there isn't a direct correlation between
    >>> the versions in the master repository and my system.  If all
    >>> of the tools had CVS tags for nightly snapshots, it would make
    >>> things easier, but the bintuils directories don't seem to have
    >>> snapshot tags.  This is the way I'm leaning, since it will
    >>> allow me to work on two systems (desk system, laptop) and
    >>> using cvs to do the update.

    >> cvs update -jgcc-3_3-branch:2003{0517,0528} works IMHO just
    >> fine, you don't need any nightly tags for it. All you need is
    >> to remember what was the last date you did a cvs merge at,
    >> which can be recorded e.g. in the CVS commit messages.

    > Thanks.

    > Wouldn't it have problems if there was a commit that occured
    > around midnight, particularly a multi-file commit, one file
    > being committed before midnight, and the other after midnight.

Re: OT: How do people set up long term development

[Gerald Pfeifer]

On Wed, 28 May 2003, Michael Meissner wrote:
> I can see three different development models:

In your specific case, I can imagine a fourth one, based on your plan
to submit non-machine specific changes right away (which I definitely
consider a good idea) and the fact that nearly all other changes will
reside in a new subtree in gcc/config:

4) Use our (gcc.gnu.org) CVS tree to do development, and use another CVS
   repository just for that subtree.

   That way, you can also submit/commit changes to the gcc.gnu.org tree
   most easily and don't have to worry about importing changes from
   gcc.gnu.org into a local repository.

Gerald
-- 
Gerald "Jerry"   pfeifer@dbai.tuwien.ac.at   http://www.pfeifer.com/gerald/

Re: OT: How do people set up long term development

[cgd]

At Wed, 28 May 2003 15:02:32 +0000 (UTC), "Michael Meissner" wrote:
> 1) Create my own CVS repository of a checked out releases, and check the files
>    into there.  Every so often, I will do a merge from the FSF sources to my
>    own repository.  This is how we did development at OSF, Cygnus, and Red Hat.
>    The problem is doing the merge step, and there isn't a direct correlation
>    between the versions in the master repository and my system.  If all of the
>    tools had CVS tags for nightly snapshots, it would make things easier, but
>    the bintuils directories don't seem to have snapshot tags.  This is the way
>    I'm leaning, since it will allow me to work on two systems (desk system,
>    laptop) and using cvs to do the update.

This is what I do.

To facilitate independent updates of different components, I prefer to
import each package separately (i.e., not a unified tree), and have my
build system worry about building them in the right order, etc.

(I'd revisit that somewhat if i were doing it again from scratch, but
I think if i had the same goal in mind that you do or that I did when
i created my current source tree, I'd probably do the same.)


The two real drawbacks of this (IMO) are:

* the build system can get a bit bit hairy.  8-)

* you have to do a bit of extra work to merge changes to the common
  components between the different parts of the source tree.  (e.g.,
  make opcodes changes?  Gotta change the copies of opcodes in
  binutils and gdb.)

It also chews more disk space and builds a few extra copies of things,
but disk is cheap and CPU is pretty cheap, too.


I also do my cvs imports in a special way, so that I have as much time
as I need to merge things.

basically, do:

	cvs import ... >& log_file

then run the script below against that log file.  any changes in any
imports need to be explicitly merged on to the trunk.

(One can approximate this procedure by using 'cvs commit -f' on one's
source tree, but that leads to bogus commits after the initial import,
and also newly-imported files automatically show up on the trunk.
ugh.  8-)

That way, if you try a snapshot from a night and it turns out not to
work, you don't have to commit it to the trunk.

beware: the script below doesn't do any cvs locking itself, and
requires local access to the CVS repository.  8-)


(BTW, if somebody's looking for a project to hack on, inside CVS, I'd
love to see the equivalent of the script below implemented as a 'cvs
import' option.  8-)



cgd
--
#!/bin/sh
# $Id: cvsimport_killnew,v 1.5 2003/01/16 01:27:55 cgd Exp $

# Script to kill revision 1.1 in files newly imported by CVS.
# See http://www.demetriou.com/~cgd/cvs for details.
#
# Written by Chris G. Demetriou <cgd@netbsd.org>.  Public domain.

fatal() {
	echo '***' $0: ERROR: $1 1>&2
	exit 1
}

usage() {
	echo usage: $0 logfilename 1>&2
	fatal usage
	exit 1
}

if [ -z "${CVSROOT}" ]; then
	fatal "CVSROOT is unset"
fi

logfile=$1
if [ -z "$logfile" ]; then
	usage
fi
if [ ! -f "$logfile" ]; then
	fatal "$logfile does not exist"
fi

awk '/^N / { print $2 }' "$logfile" | (
	set -e
	while read file; do
		repos_path=${CVSROOT}/${file},v 
		repos_dir=`dirname ${repos_path}` 
		repos_file=`basename ${repos_path}` 

		if [ ! -f $repos_path ]; then
			echo "Warning: $repos_path does not exist."
			continue
		fi

		echo "Fixing $file..."
		(
			# XXX error handling in this block is bogus...
			# a dir that won't (likely) be used.
			mkdir -p $repos_dir/CVS
			cd $repos_dir/CVS

			# reset default branch.
			rcs -b ../${repos_file} 

			# check in new rev w/ same contents.
			co -l -M ../${repos_file}
			ci -r1.2 -d -sdead \
			    -m"revision 1.1 was added on the vendor branch; delete from trunk" \
			    ../${repos_file}
		)
		rmdir $repos_dir/CVS || true

		echo "Moving $file to the attic..."
		mkdir -p ${repos_dir}/Attic 
		mv ${repos_path} ${repos_dir}/Attic/${repos_file} 
	done
)

Re: OT: How do people set up long term development

[David O'Brien]

On Wed, May 28, 2003 at 11:02:11AM -0400, Michael Meissner wrote:
> I am wondering how people outside of Red Hat do long term development of
> compiler tools when you can't check the machine specific sources into the FSF
> repository until the NDA on the machine has been lifted?  The intention is that
> when the chipset I am developing the GNU toolchain for is formally announced, I
> will submit the files to the FSF for consideration of inclusion in the
> releases, and non-machine specific changes that I make will be contributed as
> they are written.
> 
> I can see three different development models:
> 
> 1) Create my own CVS repository of a checked out releases, and check the files
>    into there.  Every so often, I will do a merge from the FSF sources to my
>    own repository.  This is how we did development at OSF, Cygnus, and Red Hat.

This is how the FreeBSD's does it.  However, you didn't mention the very
important deatail of doing *vendor branch* imports into your private CVS
tree.  Perododically, take a GCC snapshot, and import it again onto the
*vendor branch*.  CVS will give you instructions on how to use "-j -j" to
merge the changes onto your mainline development branch.

Others do the same thing, but with Perforce over CVS.  Perforce's vastly
superior branching and merging abilities make doing this easier than it
is with CVS.

> 3) I could imagine an alternative to #1 of downloading the rcs files from the
>    repository to use as a starting base, instead of creating the files from
>    scratch.  This may make the eventual merge easier, but intermediate merges
>    may be more difficult.

You can also CVSup the GCC CVS repository (,v files); and follow the
instructions at http://www.cvsup.org/faq.html#canilocal for maintaining
your own local branch.  You would use CVS's -j merging to put FSF GCC
changes onto your local, private branch.

-- 
-- David  (obrien@FreeBSD.org)