Re: An article about the Cygnus tree

Re: An article about the Cygnus tree

[Michael Sokolov]

David Edelsohn <dje@watson.ibm.com> wrote:

> I do not understand why you incorrectly insist on using such
> hostile and confrontational language in your description of EGCS.

I certainly didn't mean to be hostile or confrontational to the former EGCS. If
something in my description of it appears hostile or confrontational to you,
please point it out, also indicating how you would like it to be different (in
diff -c format).

Now you may find hostile or confrontational my criticism of first EGCS' and now
GCC's reluctance to move to the unified tree, but I think it's fair criticism.

Sorry guys for hurting your feelings if I did. I didn't intend to. I just want
the unified tree to become a reality, and in the current state of affairs there
isn't even a site or a mailing list for the Cygnus tree overall, necessitating
something like my article to be written to get the ball rolling.

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Re: An article about the Cygnus tree

[David Edelsohn]

	I find your "Introduction to the Cygnus Tree" to be riddled with
incorrect statements.  I do not understand why you apparently made no
attempt to investigate the facts before publicly releasing a thoroughly
inaccurate document.

	Cygnus, as a company, never has been the official FSF-appointed
maintainer of *ANY* GNU Project package.  Daily and weekly snapshots of
the various packages (gcc, gas, gdb, and binutils) were available to all
active developers prior to the public CVS repositories; they were not
developed solely by Cygnus or the FSF behind closed doors.  EGCS was not
run by Cygnus and characterizing it as competition with FSF's gcc project
is just one of numerous places where the description is far from
objective.  The hosting of the gcc.gnu.org site by Cygnus/Red Hat is not a
"dirty little secret".

	Whatever ax you have to grind, I would appreciate if you would
take it elsewhere instead of applying your prism to everyone else.  I
personally find your terminology and inferences highly offensive.
Spreading this sort of disinformation does not help the Free Software
Movement or help encourage a unified source tree and CVS repository for
GNU toolchain packages.

	The various maintainers and steering committees are not composed
of public relations experts, so we probably have been less communicative
and less quick to refute incorrect rumours than we should have been.
Filling in the missing information with conspiracies and accepting the
negative rumours is a normal human instinct, but easily resolved with a
little bit of investigation before producing a draft document.  I would
appreciate if you would demonstrate some comprehension and productive
involvement before you start patting yourself on the back for getting
things rolling.

David

Re: An article about the Cygnus tree

[Alexandre Oliva]

On Sep  6, 2000, Jeffrey A Law <law@cygnus.com> wrote:

> Because CVS doesn't DTRT with symlinks.  At least that what I was
> told.

I've never had any problem with symlinks of directories.  It indeed
doesn't work well with symlinks of files.

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: An article about the Cygnus tree

[Jeffrey A Law]

  In message < 200009070049.RAA21087@kankakee.wrs.com >you write:
  > > To: msokolov@ivan.Harhan.ORG (Michael Sokolov)
  > > Date: Wed, 06 Sep 2000 16:05:27 -0400
  > > From: David Edelsohn <dje@watson.ibm.com>
  > 
  > > No one is arguing against merging the trees.  It simply takes time
  > > and effort which no one has time to tackle at the moment.
  > 
  > Out of curiosity, would a sym link in the repo from src/gcc over to
  > egcs/gcc not fix the bulk of the problem?
Because CVS doesn't DTRT with symlinks.  At least that what I was
told.

jeff

Re: An article about the Cygnus tree

[Mike Stump]

> To: msokolov@ivan.Harhan.ORG (Michael Sokolov)
> Date: Wed, 06 Sep 2000 16:05:27 -0400
> From: David Edelsohn <dje@watson.ibm.com>

> No one is arguing against merging the trees.  It simply takes time
> and effort which no one has time to tackle at the moment.

Out of curiosity, would a sym link in the repo from src/gcc over to
egcs/gcc not fix the bulk of the problem?  If it does, could that be
done to sync most of the files, with very little effort?  It sounds
like if that were done, and a forced merge for include and maybe two
more symlinks for libiberty and include, that almost the entire
problem would be solved.  From there, new modules work and
rationalizing the toplevel files, and testing, and then sym link from
cvs/egcs over to cvs/src would drive it the rest of the way home.

Re: An article about the Cygnus tree

[David Edelsohn]

>>>>> Michael Sokolov writes:

Michael> I'm sorry that you've failed to see the whole point of my article, which is to
Michael> persuade the powers that be to merge the src and gcc repos. What's keeping GCC
Michael> in its own repo are the people on the GNU side of things who see it as
Michael> independent from the Cygnus tree. But what I'm arguing in my article is that
Michael> this perception of GCC's independence from the Cygnus tree by some GCC
Michael> maintainers is bogus. To become really independent, they would have to really
Michael> break all ties with the Cygnus tree, removing the top-level configure script
Michael> and Makefile that support the rest of the tree and taking away the single tree
Michael> build. I very seriously doubt that they'll be able to do that, unless they
Michael> commit the change anonymously so that the angry mob of embedded developers
Michael> using the single tree build doesn't get them, and if they are not going to do
Michael> that, they are still tied to the Cygnus tree whether they are in the main repo
Michael> with the master copies of the top-level files or using stale mirrors of those
Michael> files in their own repo, and everyone would benefit and no one would lose
Michael> anything from switching to the former.

	No one sees the trees as independent and no one is trying to keep
the independent.  No one is arguing against merging the trees.  It simply
takes time and effort which no one has time to tackle at the moment.  The
only concern is being able to make a single, coordinated distribution in
the face of the different schedules of the various packages.  One still
can create a release branch and stabilize the various dependent components
in that branch for a particular package (gcc, binutils, gdb, etc.).

	I do not understand why you insist on looking for conspiracies and
attributing ulterior motives where there are none.  Have you ever heard of
"Occam's razor"?  If there is a simple explanation, it most likely is
correct.

David

Re: An article about the Cygnus tree

[Michael Sokolov]

Joe Buck <jbuck@racerx.synopsys.com> wrote:

> I am starting to see the problem.  You are confusing several things
> together and mashing them into one concept that you are calling "the
> Cygnus tree".

No, I'm not confusing anything.

> But
> what you're failing to see is that this system was designed so that
> a GNU source directory can be made a subdirectory and then built *without
> changing it*.

No, I'm not failing to see this, I know this very well.

> Since you don't appreciate that, you falsely believe that
> a huge change had to be made to gcc to get it to build within this
> structure.

No, I know that there was no big change at that early stage. The big changes
came when the original non-Cygnus-tree GNU packages went away and the ones in
the Cygnus tree became the only ones remaining, at the same time undergoing a
transformation to become dependent on the top level and on other modules. This
was certainly much more significant for Binutils and GDB, but I'm arguing that
GCC is really no different.

I'm sorry that you've failed to see the whole point of my article, which is to
persuade the powers that be to merge the src and gcc repos. What's keeping GCC
in its own repo are the people on the GNU side of things who see it as
independent from the Cygnus tree. But what I'm arguing in my article is that
this perception of GCC's independence from the Cygnus tree by some GCC
maintainers is bogus. To become really independent, they would have to really
break all ties with the Cygnus tree, removing the top-level configure script
and Makefile that support the rest of the tree and taking away the single tree
build. I very seriously doubt that they'll be able to do that, unless they
commit the change anonymously so that the angry mob of embedded developers
using the single tree build doesn't get them, and if they are not going to do
that, they are still tied to the Cygnus tree whether they are in the main repo
with the master copies of the top-level files or using stale mirrors of those
files in their own repo, and everyone would benefit and no one would lose
anything from switching to the former.

> Concept #2 is the CVS archive that Cygnus (now Red Hat) makes releases
> from for their own customers.  This tree is *not* the same as the "net"
> version of gcc (or egcs before it).  In many cases, work that Cygnus did
> went to customers first and only later was merged into the egcs or GCC
> distribution.

I know that there are two Cygnus trees, one internal and one public, with the
former being very old and the latter not being complete yet because of
someone's reluctance to merge /cvs/src and /cvs/gcc on the Sourceware box. This
is explained in my article too, clearly enough I think. And I also know that
the differences between the two are generally kept to a minimum. Also don't
forget that before the public CVS repos were created, there were still public
daily or weekly snapshots. I can bet that those were made directly from the
trunk of Cygnus' internal repo, proving that it can't be too far away from what
the public is allowed to see. As for customer stuff, that's on branches, not on
the trunk, or so I've heard.

And still note the big problem with your wording. When you talk about the
public tree, you are again talking about a separate tree for GCC or EGCS rather
the a public version of the full Cygnus tree. *There is no separate GCC or EGCS
tree, other than as a nuisance to the developers and users in the two-repo
arrangement*. There is a public Cygnus tree now, in /cvs/src on the Sourceware
box, which closely mirrors the structure of the old internal tree and contains
all of its components that Cygnus was willing to make public, whether they are
used in GNU projects (like bfd, opcodes, etc) or are completely non-GNU (like
newlib, libgloss, winsup, etc). GCC is the only remaining exception, and I'm
doing everything I can to persuade the powers that be to realise that it's no
different from everyone else (including Binutils and GDB, which are just as GNU
but happily live in the unified public Cygnus tree) and to merge the src and
gcc repos.

Haven't you noticed lately that every change to the top level has to be
carefully applied to both /cvs/src and /cvs/gcc repos? If you have, how can you
still think of a separate GCC tree?

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Re: An article about the Cygnus tree

[Russ.Shaw]

Its just that crossgcc traffic seems a bit on the low side
for some reason. BTW, is all this cross-posting bad? A
newsgroup is better suited for a wide range of questions
and seems more 'publicly accessible' for beginners to
ask all kinds of stupid questions.

Michael Sokolov wrote:
> 
> "Russ.Shaw" <russell@webaxs.net> wrote:
> 
> > I think there should be a newsgroup for the gnu tools etc, for applications
> > involving embedded systems. comp.arch.embedded is more hardware oriented.
> > What about a comp.gnu.embedded for all questions on installation, development,
> > use, and abuse of gnu/cygnus/open-source tools for embedded systems?
> 
> How is the crossgcc mailing list not enough for you for this purpose? That's
> exactly what it's for.

-- 
*******************************************
*   Russell Shaw, B.Eng, M.Eng(Research)  *
*      email: russell@webaxs.net          *
*      Victoria, Australia                *
*******************************************

Re: An article about the Cygnus tree

[Joe Buck]

> 
> Joe Buck <jbuck@racerx.synopsys.com> wrote:
> 
> > Still wrong.  The FSF never "closed their gcc project".  This *is* the gcc
> > project.  Same people for the most part, same code base.
> 
> Not quite. Yes, politically it's the same project, has many of the same people,
> and the core code is a version of the same thing. But this is not what my
> article is about. My article is about the Cygnus tree, the top-level
> architecture, and the fact that all these projects are now dependent on the
> tree architecture and belong in the single unified repo.

I am starting to see the problem.  You are confusing several things
together and mashing them into one concept that you are calling "the
Cygnus tree".

Concept #1 is the software architecture that allows for a top-level
makefile with tools placed underneath that top level that can be built
all in one step.  Yes, this was invented by folks at Cygnus.  But
what you're failing to see is that this system was designed so that
a GNU source directory can be made a subdirectory and then built *without
changing it*.  Since you don't appreciate that, you falsely believe that
a huge change had to be made to gcc to get it to build within this
structure.  Not at all: it just drops in.  The Cygnus folks had to do
it that way to work and play successfully with GNU maintainers that did
not use the structure.

Similarly, we noticed in the early days of egcs that the makeinfo sources
were a lot smaller than the .info files, so we decided to include the
source in the distribution (maybe not the greatest decision).  It was not
necessary to change texinfo/makeinfo to do this.

Because you make this mistake, you think that gcc underwent a larger
technical change than it did.  In fact, you appear to believe that this
was a revolution or something.  Surprise: it is no big deal.

Concept #2 is the CVS archive that Cygnus (now Red Hat) makes releases
from for their own customers.  This tree is *not* the same as the "net"
version of gcc (or egcs before it).  In many cases, work that Cygnus did
went to customers first and only later was merged into the egcs or GCC
distribution.

Re: An article about the Cygnus tree

[Michael Sokolov]

"Russ.Shaw" <russell@webaxs.net> wrote:

> I think there should be a newsgroup for the gnu tools etc, for applications
> involving embedded systems. comp.arch.embedded is more hardware oriented.
> What about a comp.gnu.embedded for all questions on installation, development,
> use, and abuse of gnu/cygnus/open-source tools for embedded systems?

How is the crossgcc mailing list not enough for you for this purpose? That's
exactly what it's for.

But what my article calls for is quite different. Here I'm concerned mostly
with the *core developers*. I'm calling for the /cvs/src and /cvs/gcc repos on
Sourceware to be merged and for creating a home for the Cygnus tree concerned
with its core development and maintenance, rather than "abuse" you are talking
about.

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Re: An article about the Cygnus tree

[Michael Sokolov]

Joe Buck <jbuck@racerx.synopsys.com> wrote:

> Still wrong.  The FSF never "closed their gcc project".  This *is* the gcc
> project.  Same people for the most part, same code base.

Not quite. Yes, politically it's the same project, has many of the same people,
and the core code is a version of the same thing. But this is not what my
article is about. My article is about the Cygnus tree, the top-level
architecture, and the fact that all these projects are now dependent on the
tree architecture and belong in the single unified repo. The old gcc which had
all the actual gcc code at the top directory level and no libiberty *is* dead.
The other, once forked, GCC that has the Cygnus configure script at the top
directory level is the only one remaining. That was my whole point.

> > BTW, what did EGCS stand for?
>
> eggs.

I've checked in this:

------- cygnus-tree-intro -------
*** /tmp/d01094	Tue Sep  5 11:28:02 2000
--- cygnus-tree-intro	Tue Sep  5 11:10:11 2000
***************
*** 42,52 ****
  its development, so Cygnus didn't take over its development by itself. First
  they synchronised their work on it (in their internal Cygnus tree) with the FSF
  maintainers. Then in August 1997 they created an open development project for
! it which they named EGCS (Extended GNU Compiler System). It was a public
! project and for a period of time, for better or worse, was in competition with
! FSF's gcc project. In spring 1999 FSF closed their gcc project and EGCS was
! renamed into GCC. At the same time GCC was changed to mean GNU Compiler
! Collection instead of GNU C Compiler.
  
  All these programs have been integrated into the Cygnus tree so completely that
  they no longer exist separately from it. Moreover, some of these programs are
--- 42,52 ----
  its development, so Cygnus didn't take over its development by itself. First
  they synchronised their work on it (in their internal Cygnus tree) with the FSF
  maintainers. Then in August 1997 they created an open development project for
! it which they named EGCS (eggs, just like the ones Americans eat for breakfast
! :-). It was a public project and for a period of time, for better or worse, was
! in competition with FSF's gcc project. In April 1999 FSF closed their gcc
! project and EGCS was renamed into GCC. At the same time GCC was changed to mean
! GNU Compiler Collection instead of GNU C Compiler.
  
  All these programs have been integrated into the Cygnus tree so completely that
  they no longer exist separately from it. Moreover, some of these programs are

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Re: An article about the Cygnus tree

[Russ.Shaw]

I think there should be a newsgroup for the gnu tools etc, for applications
involving embedded systems. comp.arch.embedded is more hardware oriented.
What about a comp.gnu.embedded for all questions on installation, development,
use, and abuse of gnu/cygnus/open-source tools for embedded systems?


Michael Sokolov wrote:
> 
> And while I think this is stated clearly enough in the article itself, let me
> repeat: the previous point (that I wasn't even sure what to call it) is the
> essense of the problem that needs a solution: there is a sore lack of public
> tutorial information on this animal. There is a home for the GNU project and
> there are home pages for all GNU projects, and as a result, everyone knows what
> they are and where to get them. But it's very hard to explain to a newcomer
> what the Cygnus tree is. See below.
> 

-- 
*******************************************
*   Russell Shaw, B.Eng, M.Eng(Research)  *
*      email: russell@webaxs.net          *
*      Victoria, Australia                *
*******************************************

Re: An article about the Cygnus tree

[Joe Buck]

> OK, I checked this in:

> ! was a public project and competed with FSF's gcc project. Finally, in spring
> ! 1999 FSF closed their gcc project and EGCS was renamed into GCC.

Still wrong.  The FSF never "closed their gcc project".  This *is* the gcc
project.  Same people for the most part, same code base.  We were always
careful with egcs to do things in a way that would prevent a permanent
fork.  What we were doing, in essence, was to demonstrate to the FSF that
there was a better way of doing things.  It is difficult to do these
things without offending people; there were some very sensitive politics
involved.  But the end result has been a great success.

What happened in April 1999 was that the 2.8.x and egcs lines of
development were merged and the egcs maintainers were given charge of the
whole project.  The current way of working is more in line with FSF
principles than what we had before (restricting who can get snapshots
seems at least to violate the spirit of the GPL, even if it is technically
allowed).

> BTW, what did EGCS stand for?

eggs.  (hence the logo).  Actually there were several suggested meanings
for the "E": "experimental" was the most common one, and it was presented
to RMS as an experiment.  He was kept informed and gave his sometimes
grudging approval for the major decisions.

Re: An article about the Cygnus tree

[Michael Sokolov]

David Edelsohn <dje@watson.ibm.com> wrote:

> I find your "Introduction to the Cygnus Tree" to be riddled with
> incorrect statements.  I do not understand why you apparently made no
> attempt to investigate the facts before publicly releasing a thoroughly
> inaccurate document.

I *did* investigate most of the facts. I have been watching these facts unfold
for a while now, I've been saving all the important bits of information about
this stuff I've come across in the past (even though I didn't need them then),
and I've done some CVS repo archaelogy. I did miss a few fine points, but you
could have certainly been a little more polite in pointing them out.

> Cygnus, as a company, never has been the official FSF-appointed
> maintainer of *ANY* GNU Project package.

I didn't say that they were FSF-appointed maintainers. That is politics that
I prefer to stay out of. I was and am focusing on the *technical* aspect of the
transition. First there were GNU packages with all their "meat" at the top
directory level. The Cygnus tree had versions of them grafted under the top-
level configure script and Makefile. What was so remarkable about the
transition from FSF's gcc2 to EGCS/GCC, and the much earlier transition in
Binutils and GDB that I, perhaps incorrectly, referred to as Cygnus takeover,
is that the FSF packages with the "meat" at the top directory level *went away*
and the version of the program in the Cygnus tree became the master one, with
all subsequent EGCS or FSF tarballs being made from the Cygnus tree and
carrying top-level configure scripts and Makefiles telling curious code readers
about the rest of the tree.

Also when I say "Cygnus tree", I'm not picking on them as a company. (And yes,
I know that they are Red Hat now.) I call it the Cygnus tree for exactly the
same reason why the configure script at its top is universally known as Cygnus
configure and called this way by everyone, and why the Automake option enabling
special rules for this tree is named cygnus. In fact, I took the name Cygnus
tree from configure.texi, before I read that I didn't know how to call it.

And while I think this is stated clearly enough in the article itself, let me
repeat: the previous point (that I wasn't even sure what to call it) is the
essense of the problem that needs a solution: there is a sore lack of public
tutorial information on this animal. There is a home for the GNU project and
there are home pages for all GNU projects, and as a result, everyone knows what
they are and where to get them. But it's very hard to explain to a newcomer
what the Cygnus tree is. See below.

> Daily and weekly snapshots of
> the various packages (gcc, gas, gdb, and binutils) were available to all
> active developers prior to the public CVS repositories;

I know this, and I never said otherwise.

> The hosting of the gcc.gnu.org site by Cygnus/Red Hat is not a
> "dirty little secret".

When writing that, I assumed that everyone reading those words would understand
them as tongue-in-cheek. Your reaction shows that I was wrong. Sorry 'bout
that. That and all the language about "conspiracies" was intended to convey
that the lack of a home for the tree overall, as opposed to its individual
modules, makes it look like a conspiracy. *Of course* I know that everyone
working on this code is extremely devoted to free software and that there are
no conspiracies. I was pointing it out that there is a problem with not having
a home for the unified Cygnus tree and that it makes it look like a conspiracy.

> Whatever ax you have to grind [...]

I don't have any axes to grind, and I'm very sorry that you have so completely
misinterpreted my article. I really don't think I could have made any clearer:
in the article itself I have clearly stated more than once why I wrote it and
what I'm trying to achieve.

I think I have explained it as well as it could be in the article, so repeating
it here would be just wasting bandwidth, but let me repeat it very briefly. I
have projects that use the Cygnus tree instead of discrete releases from
ftp.gnu.org. These projects have found it unacceptable to build bfd and opcodes
twice and libiberty three times. I, being a developer and knowing the "secrets"
(tongue-in-cheek) which most developers know but which are not written down in
ASCII anywhere, have no problem with this and have found the compilation time
reduction and ease of development tracking very pleasant. But right now I'm in
the process of preparing a release, and this is where I hit a problem. It's
trivial to tell your users "this system uses GCC, Binutils, and GDB, go get
them". But it's infinitely harder to tell them that a system uses the Cygnus
tree, given that there is no home for it where people can learn what it is,
that there is no single repository to get it from, and that a lot people don't
even know that such a thing exists (been there myself).

First all I wanted to do was to explain the procedure for piecing together the
tree from the two repos. But before I even got there, I at least had to say
*what* does one need to build. I needed to refer to the Cygnus tree. It's
trivial to say "this system uses GCC" and point to its WWW page. But for the
Cygnus tree there isn't any. In Free Software when you need something that does
not exist, you create it. This is exactly what I did in writing my article. And
of course after having explained the horrible procedure for piecing together
the tree when the correct solution (repo unification) is obvious, it was only
proper to make a call for action to actually implement this correct solution.

> [...] before you start patting yourself on the back for getting
> things rolling.

I was never planning on doing that actually.

Anyway, I have checked in version 1.4 of cygnus-tree-intro with the language
changed so that hopefully fewer people will get offended by it and more people
will gain useful insight from it, which was its original intent. It is in
ivan.Harhan.ORG:/pub/embedded/cygnus-tree-intro.

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Re: An article about the Cygnus tree

[Alexandre Oliva]

On Sep  4, 2000, Jan Dvorak <johnydog@go.cz> wrote:

> On Mon, Sep 04, 2000 at 07:30:59PM -0500, Michael Sokolov wrote:
>> BTW, what did EGCS stand for?

> EGCS stands for Extended GNU Compiler System

AFAIK, the E in EGCS could also stand for Experimental and Enhanced,
depending on the reader.  There was no official wording :-)

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                  aoliva@{cygnus.com, redhat.com}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist    *Please* write to mailing lists, not to me

Re: An article about the Cygnus tree

[Jan Dvorak]

On Mon, Sep 04, 2000 at 07:30:59PM -0500, Michael Sokolov wrote:
> BTW, what did EGCS stand for?

Yes, you should probably include meaning of these 'shortcuts' in your
article: GCC was GNU C Compiler, EGCS stands for Extended GNU Compiler
System, PGCC for Pentium GCC, and GCC shortut than become GNU Compiler
Collection.

If i remember correctly of course.

Jan Dvorak <johyndog@go.cz>

Re: An article about the Cygnus tree

[David Edelsohn]

	I do not understand why you incorrectly insist on using such
hostile and confrontational language in your description of EGCS.  EGCS
was a testbed for some new technology which needed a larger testing
audience than the GCC development environment of the time could provide.
Both the technology and the development method used by EGCS proved
effective, so EGCS and GCC merged back together, incorporating all of the
successes of EGCS.  This is not unlike the way gcc-2 was developed and
redesigned in the late 1980's and early 1990's.

David

Re: An article about the Cygnus tree

[Michael Sokolov]

Geoff Keating <geoffk@cygnus.com> wrote:

> This bit is not correct.  EGCS was never 'run by' Cygnus.  It has
> always been run by a steering committee on which Cygnus has been
> careful to ensure that its employees were never a majority.

OK, I checked this in:

------- cygnus-tree-intro -------
*** /tmp/d26930	Mon Sep  4 19:22:03 2000
--- cygnus-tree-intro	Mon Sep  4 19:21:02 2000
***************
*** 45,52 ****
  development and do it behind closed doors. First they synchronised their work
  on it (in their internal Cygnus tree) with the FSF maintainers. Then in late
  1997 they created an open development project for it which they named EGCS. It
! was run by Cygnus and competed with FSF's gcc project. Finally, in spring 1999
! FSF closed their gcc project and EGCS was renamed into GCC.
  
  All these programs have been integrated into the Cygnus tree so completely that
  they no longer exist separately from it. Moreover, some of these programs are
--- 45,52 ----
  development and do it behind closed doors. First they synchronised their work
  on it (in their internal Cygnus tree) with the FSF maintainers. Then in late
  1997 they created an open development project for it which they named EGCS. It
! was a public project and competed with FSF's gcc project. Finally, in spring
! 1999 FSF closed their gcc project and EGCS was renamed into GCC.
  
  All these programs have been integrated into the Cygnus tree so completely that
  they no longer exist separately from it. Moreover, some of these programs are

BTW, what did EGCS stand for?

> Well, I for one would like to see a combined tree.

That's why I wrote:

: Some of the GCC maintainers have said that they liked the idea. However,
: there seems to be some politics playing against it, apparently coming from
: FSF. Thus it appears that the next battle is going to be between us,
: developers, and the politicians.

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Re: An article about the Cygnus tree

[Geoff Keating]

> Date: Mon, 4 Sep 00 16:20:16 CDT
> From: msokolov@ivan.Harhan.ORG (Michael Sokolov)

> gcc is bigger and has many more people interested in its
> development, so Cygnus didn't take over its development and do it
> behind closed doors. First they synchronised their work on it (in
> their internal Cygnus tree) with the FSF maintainers. Then in late
> 1997 they created an open development project for it which they
> named EGCS. It was run by Cygnus and competed with FSF's gcc
> project. Finally, in spring 1999 FSF closed their gcc project and
> EGCS was renamed into GCC.

This bit is not correct.  EGCS was never 'run by' Cygnus.  It has
always been run by a steering committee on which Cygnus has been
careful to ensure that its employees were never a majority.

> Unfortunately, the GCC (former EGCS) maintainers seem to not have
> grasped yet that this arrangement is troublesome and just plain
> wrong.

Well, I for one would like to see a combined tree.

-- 
- Geoffrey Keating <geoffk@cygnus.com>

An article about the Cygnus tree

[Michael Sokolov]

Hi there,

In light of the question of merging the src and gcc repos recently raised again
in a thread on the GCC list about adding zlib to GCC, I've written an article
about the Cygnus tree. It appears below. I hope it will start a better
discussion about merging the repos and breaking the code of silence around the
Cygnus tree.

This article also appears on my FTP site in /pub/embedded/cygnus-tree-intro on
ivan.Harhan.ORG. Whenever I release a toolchain for one of the embedded systems
I'm working with based on the Cygnus tree, I have to explain to my users what
it is, given its obscurity. Now I can just refer them to my article.

Enjoy!

--
Michael Sokolov		Harhan Engineering Laboratory
Public Service Agent	International Free Computing Task Force
			International Engineering and Science Task Force
			615 N GOOD LATIMER EXPY STE #4
			DALLAS TX 75204-5852 USA

Phone: +1-214-824-7693 (Harhan Eng Lab office)
E-mail: msokolov@ivan.Harhan.ORG (ARPA TCP/SMTP) (UUCP coming soon)

Here is the article:

			An Introduction to the Cygnus Tree

				By Michael Sokolov
			International Free Computing Task Force

			@(#)cygnus-tree-intro	1.1	00/09/04


			1. What is the Cygnus tree?

"The GNU configure and build system" by Ian Lance Taylor gives the following
answer:

	The Cygnus tree is used for various packages including gdb, the GNU
	binutils, and egcs.  It is also, of course, used for Cygnus releases.
	It is the build system which was developed at Cygnus, using the Cygnus
	configure script.  It permits building many different packages with a
	single configure and make.  The configure scripts in the tree are being
	converted to autoconf, but the general build structure remains intact.

During the 1990s Cygnus Solutions (now part of Red Hat, Inc.) has created a
remarkable system that is usually called the Cygnus tree. They have taken many
GNU programs (which were all designed as completely self-contained stand-alone
packages), most importantly the ones used for software development (gcc, gas,
binutils, and gdb) and unified them into one source tree with a single top-
level configure script and a single top-level Makefile. This allows all these
packages to be configured, built, and installed together in one fell swoop.

Initially, the Cygnus tree existed inside Cygnus only and was distributed only
to their customers. At that point, most of the GNU programs were simply grafted
into the tree with no changes of much interest to people outside of Cygnus.
These programs existed and were completely public in their original GNU form,
and the Cygnus tree was just a fancy packaging option of not much interest to
people outside of Cygnus. Therefore, there was no pressure for a public Cygnus
tree.

However, as time went on, it turned out that Cygnus was the best and most
active contributor to some of the GNU programs involved. As a result, they have
taken over the maintenance of those programs and changed and enhanced them
significantly. These GNU programs were gcc, gas, binutils, and gdb. Cygnus took
over the maintenance of gas, binutils, and gdb back in early-mid 1990s, doing
development behind closed doors and making occasional public releases. They
were finally opened to the public in May 1999. gcc is bigger and has many more
people interested in its development, so Cygnus didn't take over its
development and do it behind closed doors. First they synchronised their work
on it (in their internal Cygnus tree) with the FSF maintainers. Then in late
1997 they created an open development project for it which they named EGCS. It
was run by Cygnus and competed with FSF's gcc project. Finally, in spring 1999
FSF closed their gcc project and EGCS was renamed into GCC.

All these programs have been integrated into the Cygnus tree so completely that
they no longer exist separately from it. Moreover, some of these programs are
not even single modules any more. The Cygnus tree consists of many
subdirectories called modules and some top-level glue. Initially, there was one
module for each GNU program grafted into the tree. Then, however, Cygnus added
some new modules, split some existing ones, and made some existing modules
dependent on some new ones. As a result, some of the modules that initially
(long ago) were grafted into the tree from stand-alone GNU packages can no
longer be pulled back out of the tree and used separately.

As Cygnus took over the maintenance of several GNU programs, they started
making new releases of them. However, these programs had already been converted
to work in the Cygnus tree only. How could they release them separately then?
The answer is that these releases are NOT like typical GNU packages that have
the program in the top-level directory of the distribution tarball. Instead,
these releases are actually pruned checkouts of the Cygnus tree, made to look
indistinguishable to the untrained eye from typical GNU packages. An important
property of the Cygnus tree is that it doesn't have to be complete. The top-
level configure script and Makefile check whether each directory they are about
to descend into is actually present, and if it isn't, it's silently skipped. As
a result, you can prune the Cygnus tree down to contain only the modules you
need, and then just those modules will be configured and built. Cygnus-made GNU
releases are all Cygnus tree checkouts pruned down to contain only the modules
needed by the GNU release in question.

GCC is still a single module in the Cygnus tree, but it now depends on the top
level and on libiberty, the miscellaneous support code module that virtually
every other module depends on. Current GCC releases consist of the top level,
libiberty, the actual gcc module, and several modules with target libraries for
different high-level languages that GCC now supports. Current Binutils releases
consist of a lot of modules. There is still a binutils module that loosely
corresponds to the old GNU binutils package, but GNU ld is now a separate
module. (gas has always been a separate module.) The binutils, gas, and ld
modules depend extensively on bfd and opcodes, two major host library modules
invented by Cygnus. Current GDB releases also consist of a lot of modules. gdb
itself is still a single module in the Cygnus tree, but it now depends
extensively on bfd and opcodes, same as Binutils, as well as some other Cygnus-
added modules that Binutils don't use. And of course on libiberty, which is
included in all current GCC, Binutils, and GDB releases.

In addition to the above GNU programs, the Cygnus tree contains many
interesting non-GNU modules developed by Cygnus, most of which have been opened
to the public. These include Newlib, Cygwin, a number of Tcl tools and GUI
libraries, an extensive testsuite framework, and CGEN.

	2. So where is this thing in terms of public CVS and mailing lists?

This is where we currently have a problem. When Cygnus tree-based EGCS/GCC,
Binutils, and GDB were first opened to the public, they were in the form of
pruned Cygnus tree checkouts. We ended up with three GNU projects each having
its own (stale, corresponding to a snapshot from the Cygnus tree at some point)
copy of the top-level files, libiberty, and most of the headers. With Binutils
and GDB it was even worse, as they had their own copies of bfd and opcodes,
both of which are actively maintained and rapidly changing modules.

This is OK for releases, but it's a problem for development. After all, the
whole point of release branches and release engineering is to produce stability
in a single software component, regardless of the staleness and deviation from
the mainline this almost always causes. Before Cygnus opened the development to
the public, they internally had a very sensible model: one master source tree
with one master copy of each module where all development is done, so all
developers are always on the same page, and specific bits of the tree are sent
off on release branches as releases are made. Each release will inevitably have
some oddities introduced into it by the release engineering process, and some
things may become less generic than they could be (for example, the top-level
configure and Makefile will still remind curious code readers of the other
modules in the tree, but because of the potentially incompatible changes on the
release branch, there may not be perfect interoperability with them). However,
developers always have one single master tree to work with, to fix major bugs
in, and to make major improvements to. It is perfectly synergistic and self-
consistent, at the price of less stability because it isn't release-engineered.

This means that releases have the property that some bits in them may be stale
or duplicated elsewhere, both of which are highly undesirable for developers.
The way releases should really work is by letting end users who are not
expected to do their own development and bugfixing have a stable release that
doesn't need any bugfixing. But as soon as a user does find a bug he/she wants
to fix him/herself, he/she should really put the release aside, get the master
copies of all components involved, fix any bugs there, and submit the fixes to
the respective maintainers. This is the Free Software way, and this is what
makes Free Software thrive.

However, this arrangement was hampered by EGCS/GCC, Binutils, and GDB starting
life as public GNU projects in the same branched form in which they exist in
releases. What they should have done was to create a public Cygnus tree, fully
explain to everyone what the Cygnus tree is, and have all development proceed
in it like it did inside Cygnus before. But instead each of EGCS, Binutils, and
GDB began life in its own CVS repository containing the same thing the FSF
release tarballs have in them. The renaming of EGCS back to GCC didn't change
anything.

Unfortunately, the GCC (former EGCS) maintainers seem to not have grasped yet
that this arrangement is troublesome and just plain wrong. (And they have been
living with it since the start of the EGCS project!) Fortunately, the Binutils
and GDB maintainers were much quicker to realise this. Almost immediately after
the opening of these projects it became clear that a single master copy of each
component is needed, instead of two teams working divergently on two branches
made off of a once unified Cygnus tree. Also at the same time parts of the
Cygnus tree other than GCC, Binutils, and GDB (i.e., the Cygnus-developed non-
GNU modules) were being opened to the public. Those aren't GNU projects and
virtually everyone doing significant work on them is from Cygnus. Those folks
are used to the Cygnus tree and know that doing it any other way is just plain
wrong. Therefore, it became clear that the Cygnus tree needed to be brought
back.

In February 2000, less than a year after the opening of Binutils and GDB, their
separate CVS repositories were liquidated. Instead, a new CVS repository was
created which was to be the public Cygnus tree. It is /cvs/src on
sources.redhat.com (formerly sourceware.cygnus.com), and from the start it was
designed as a real full Cygnus tree repository, rather than a repository for
just one project, which is what they did with all their public CVS repositories
before that. (You can check the CVS log on its modules file to convince
yourself.) The modules that make up Binutils and GDB were moved into it,
eliminating the separate Binutils and GDB repositories that existed before.
Immediately after that Newlib and Cygwin were imported from Cygnus' internal
tree (these are Cygnus-developed non-GNU tree modules), confirming without any
doubt that finally the public Cygnus tree was born.

After being born in February 2000, the public Cygnus tree in /cvs/src on the
sourceware machine matured quickly. It is now almost complete. Unfortunately,
there is still one omission. This omission and the need for users/developers to
compensate for it manually is the reason why I'm boring you here with history
lessons instead of just telling you where to get the public Cygnus tree and
what to do with it.

This omission is GCC. Ever since the start of EGCS in late 1997 it has lived in
its own CVS repository. Currently it is /cvs/gcc on the sourceware machine
(sources.redhat.com). Officially it's on gcc.gnu.org, but the dirty little
secret is that gcc.gnu.org is just a DNS record, it points to the very same
sourceware machine, same as sources.redhat.com. Everything else has now been
integrated into the /cvs/src repository, bringing back the Cygnus tree in all
its glory. However, there is also the /cvs/gcc repository duplicating a lot of
it. In effect, instead of one unified public Cygnus tree we, the non-Cygnus
folks who don't have access to their original internal tree, have two trees to
deal with: /cvs/gcc and /cvs/src. The former contains the gcc module and all
language library modules, the latter contains everything else. The two
duplicate all the top-level files and the libiberty module.

Most people working on this code have by now realised that it is really
designed to be in one Cygnus tree, and that's how they work on it. Our unspoken
convention is now to locally construct this tree from the two repositories,
work on it, and check the changes into the right repo(s). The parts that are in
only one of the repositories are simply taken from it and combined into one
tree. The trickier parts are the ones that are duplicated in the two repos.
These are the top-level files, the headers, and libiberty. /cvs/gcc's libiberty
is considered the master one, so that one is usually taken. However, most
commits to it are also simultaneously made to /cvs/src's copy, and the latter
is also periodically replaced with the former, so it usually works just as
well. The include directory in the Cygnus tree contains the public headers for
all modules. It currently exists in both repos. /cvs/src's copy contains the
headers for all modules and /cvs/gcc's copy contains only the headers for
libiberty. The latter follow the same rules as libiberty itself. Finally, there
are the top-level files. These must know about all the modules in tree. Most
people changing these files now keep checking each change into both repos.

The /cvs/src repo has one top-level directory in it also named src, and that
directory has the Cygnus tree in it (sans GCC). You can check it out in its
entirety with:

cvs -d :pserver:anoncvs@sources.redhat.com co src

This will take a lot of time and disk space. The /cvs/src repo also has a
modules file. Remember, the Cygnus tree has lots of modules in it, and most
people work only on those modules that interest them. The modules file in the
/cvs/src repo allows checking out partial Cygnus trees, and the comments in its
CVS log indicate that it is modeled after the modules file of Cygnus' internal
repo, meaning that this is how the modules file of a real Cygnus tree should
look like. It has CVS checkout modules defined for the most common Cygnus tree
module combinations that are normally checked out together. Since CVS checkout
module names exist in the same namespace as the top-level directories in the
repo, of which there is only one (src), there are no conflicts. (In particular,
there is no conflict between CVS checkout modules and the Cygnus tree modules,
the latter being one level below in the src directory.)

The /cvs/gcc repo has one top-level directory in it named egcs, and it has this
repo's version of the Cygnus tree in it. You can check it out in its entirety
with:

cvs -d :pserver:anoncvs@sources.redhat.com co egcs

The /cvs/gcc repo's modules file doesn't do much. It has an egcs-core CVS
checkout module defined that checks out the tree without the language front
ends and target libraries, but other than that, this repo is normally checked
out in its entirety.

As for mailing lists, currently most of the public projects in the Cygnus tree
have their own project-specific mailing lists, but there are no mailing lists
for the Cygnus tree overall, leaving the top-level files and many less popular
modules homeless.

				3. Our Current Solution

So what do we do about it? As I've just explained, we really want and need the
Cygnus tree, but there currently isn't a single public CVS repository for it.
The current solution is for people to construct full Cygnus trees on their
local machines from the two CVS repos and to keep track of these two repos in
development. Here is the procedure I use for constructing the full Cygnus tree
from the two repos:

1. Check out both repos (either in their entirety or only the parts you want).
You'll have two partial Cygnus trees in different directories.

2. Create a directory for the combined Cygnus tree.

3. Populate it with everything from the src repo except the include and
libiberty subdirectories.

4. Add gcc, libiberty, and the language target libraries from the gcc repo.

5. Create the include subdirectory and populate it by merging the include
subdirectories of the two repos. For files that are present in both, use the
gcc repo's version.

Explanation. This procedure is designed with the following two points in mind:

1. libiberty and its headers are taken from the gcc repo, which is considered
the master copy.

2. The procedure I just outlined uses the top-level files from the src repo.
The ones from the gcc repo could have been used instead. Most of the time both
will work equally well. I personally prefer to take them from the src repo
because it was specifically designed as the real full Cygnus tree repo.

				4. The Real Solution

The above procedure, with a few variations, is generally followed by most
developers working in the Cygnus tree. However, this doesn't make it any less
painful. It is just a nuisance for everyone to keep piecing the tree together
every time, then parsing back where to check in patches, and remembering to
keep the two repos in sync. There is absolutely no benefit to gain from the
current arrangement. It doesn't give GCC any more independence. GCC is now
critically dependent on the Cygnus tree (and has been so ever since the start
of EGCS), and by keeping their own copy of it the GCC maintainers are simply
closing their eyes to this. But the reality is that everyone still builds and
tests it together with the rest of the Cygnus tree (using the procedure from
the previous section), and the top-level files in the gcc repo are still kept
in sync with the ones in the src repo, manually and painfully. There is nothing
to lose by doing away with this and merging the repos, only a lot of
convenience and sanity to gain.

This is only one of the problems with the current arrangement. The other
problem is that there is no "home" for the Cygnus tree. There is no place where
people can learn what the Cygnus tree is and read all about it. There is no
mailing list to discuss it overall (as opposed to some particular module in
it). There is no clearly designed group responsible for the maintenance of the
top-level files.

In fact, it's even more than just not having a mailing list for the Cygnus tree
overall. There appears to be some sort of a code of silence around it. Many
people know what it is and do their development with it in mind, but there is
virtually no public mention of it, as if everyone is pretending that it doesn't
exist. This is actually why I had to write this article: to tell the people
what the Cygnus tree is. I have a number of projects that involve the Cygnus
tree, and when presenting them to the public, I found myself facing a strange
problem. I need to refer people to the Cygnus tree, but there is nothing to
refer them to! Their is no home (WWW page, FTP site, mailing list, or anything
at all) about the Cygnus tree that I can point people to. In fact, there is
nothing to even tell people what it is, aside from a very terse remark in Ian
Lance Taylor's "The GNU configure and build system", which while describing how
its configure scripts and Makefiles work, really fails to answer the question
of *what it is*.

As a developer highly interested in the Cygnus tree, I'm trying to do whatever
I can to help solve these problems. This article explaining what the Cygnus
tree is and what is its current situation is my first step. I will end it with
my proposal for what I believe is the right solution.

For the problem of two repos, the solution is to merge them. Given how the src
repo was intended from the start as the repo for the full Cygnus tree and how
it successfully does this now for everything except GCC, there is no need to do
anything with it. It is already exactly the way it should be. All that needs to
be done is to move the gcc and language library modules from the gcc repo to
the src one, replace libiberty and its headers in the src repo with the ones in
the gcc one, and put the gcc repo to rest.

Given how new modules have recently been added to the src repo with no fuss and
no problems, I don't think that the current inhabitants of the src repo would
object to welcoming a new member, or even that their consent would be required.
After all, they don't have to check out any modules other than the ones they
need, and the top-level configure script and Makefile already list everything
anyway. In fact, many src repo inhabitants will certainly like having the
master copy of libiberty in their repo, rather than a mirror that sometimes
gets stale. Thus the only ones who will have to be persuaded are the GCC
maintainers.

The question of merging the repos has come up more than once on the GCC mailing
lists. Some of the GCC maintainers have said that they liked the idea. However,
there seems to be some politics playing against it, apparently coming from FSF.
Thus it appears that the next battle is going to be between us, developers, and
the politicians. In order to fight and win this battle, we must actively push
our cause. This brings us to the second problem of not having a real home for
the Cygnus tree.

The solution for this problem is obvious: create one. This article is a start,
explaining publicly apparently for the first time what the Cygnus tree is and
trying to break the code of silence around it. Now we just need to make all
this better known to more people so that we can start a mailing list for the
Cygnus tree and decide what else do we need for a "home" for it. This
awareness-raising cause will probably have to be pursued on the GCC, Binutils,
and GDB mailing lists. This is because these are the parts of the Cygnus tree
where some people still live in the sandbox of separate GNU projects. Everyone
else, i.e., people working on the Cygnus-developed non-GNU modules like Newlib,
already come from the Cygnus tree background and would certainly be all for
bringing it back.

So, let's all try to do our best to enlighten the public about the Cygnus tree,
create a real home for it, and persuade the GCC maintainers to move to the src
repository!

				Appendix. References

"The GNU configure and build system" by Ian Lance Taylor is file configure.texi
in the etc subdirectory of the Cygnus tree. For those of you who prefer WWW,
the author has a WWW version on his page:

http://www.airs.com/ian/configure/

Once you know what the Cygnus tree is thanks to this article, Ian's superb
tutorial will tell you everything you need to know about its configure scripts
and Makefiles to master development in this tree.

libgloss/doc/porting.texi in the Cygnus tree gives a very good overview of how
the different pieces of the tree come together to support embedded systems and
how to port them to a new one.