Re: [maint] The GDB maintenance process

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Thu, Feb 20, 2003 at 08:55:21AM +0200, Zaretskii Eli wrote:
> 
> This message was scanned for viruses and other malicious code by PrivaWall.
> 
> 
> This mail was sent from ELTA SYS LTD.
> 
> 
> > From: Daniel Berlin [mailto:dberlin@dberlin.org] 
> > Sent: Wednesday, February 19, 2003 3:24 PM
> > 
> > > I guess I just don't see this to be as much of a problem as  others
> do.
> > > For one thing, with the higher entropy level, more development
> actually
> > > happens.
> > Bingo.
> > I don't think we should stall development (and in the 
> > extreme,  even if 
> > it means we can't make quality releases any day of the year) because 
> > mistakes occasionally happen in patches, or because not every 
> > maintainer in existence has said something about a patch.  That's a 
> > recipe for no progress.
> 
> For some definition of ``progress''.
> 
> Who said that adding code at a faster rate at the price of having more
> bugs is more ``progress'' than what we have now?  There are people out
> there who need GDB to actually do something _useful_, not just to debug
> and/or develop GDB itself, you know.  What about frustration of those
> GDB users when their favorite feature is broken by some
> committed-before-review patch that adds a hot new feature?  Does that
> ever count?

I wouldn't have suggested this if I really thought that would happen.

> Does anyone remember that latest GCC releases are practically unusable
> for any production-quality work due to bugs?  Does anyone even care?

And for the record, while I'd say that was true for 3.0, it was _not_
true for 3.1 or 3.2 or 3.2.1/3.2.2, which I consider production quality
compilers; and it won't be true for 3.3 either.

> Of course, if contributors are frustrated by the slow review rate, let's
> try to improve that (see my other mail).  But let's not obscure our view
> of the problem by discussing abstract issues of ``progress''.  An
> official release every 3 months is more than enough progress for my
> taste.

Not if there's nothing much new in it.  Which is a bit of an
exaggeration, before anyone calls me on it - but still pretty well
expresses my point.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Daniel Berlin]

>
>
>> From: Daniel Berlin [mailto:dberlin@dberlin.org]
>> Sent: Wednesday, February 19, 2003 3:24 PM
>>
>>> I guess I just don't see this to be as much of a problem as  others
> do.
>>> For one thing, with the higher entropy level, more development
> actually
>>> happens.
>> Bingo.
>> I don't think we should stall development (and in the
>> extreme,  even if
>> it means we can't make quality releases any day of the year) because
>> mistakes occasionally happen in patches, or because not every
>> maintainer in existence has said something about a patch.  That's a
>> recipe for no progress.
>
> For some definition of ``progress''.
>
> Who said that adding code at a faster rate at the price of having more
> bugs is more ``progress'' than what we have now?

Who said we'd necessarily have more bugs?
Can you back this up?

> There are people out
> there who need GDB to actually do something _useful_, not just to debug
> and/or develop GDB itself, you know.
You've assumed that it would make GDB completely unusable.
You know, it's funny you mention "people out there", since *every* GDB 
user i know not on the gdb mailing list is a desktop developer, working 
with C++ (KDE friends) or Java (Weirdo friends :P) , and yet these are 
*exactly* the portions of GDB that work least well.  It's almost as if 
our focus on what features are important is wrong.  But anyway, that's 
a rant for another time.
> What about frustration of those
> GDB users when their favorite feature is broken by some
> committed-before-review patch that adds a hot new feature?
> Does that
> ever count?

Does it ever happen?
I mean, seriously.
Ever?
I doubt it would have been suggested if anyone seriously though it was 
going to happen.
This isn't grade school anymore, we all know how to write good code.

>
> Does anyone remember that latest GCC releases are practically unusable
> for any production-quality work due to bugs?  Does anyone even care?
>
Latest?
There was *one* release that was like this, GCC 3.0.
And we *knew* it was going to be like this, because of the new ABI.  
You couldn't do much about it.
That's why 3.1 was planned as a completely "make it faster, fix bugs", 
not "add new features" release.


> I say thanks God for slower development of GDB.  At least I can _debug_
> buggy code produced by buggy development tools ;-)
>
> Of course, if contributors are frustrated by the slow review rate, 
> let's
> try to improve that (see my other mail).  But let's not obscure our 
> view
> of the problem by discussing abstract issues of ``progress''.  An
> official release every 3 months is more than enough progress for my
> taste.
Only if they contain some useful feature.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Of course, if contributors are frustrated by the slow review rate, let's
>> try to improve that (see my other mail).  But let's not obscure our view
>> of the problem by discussing abstract issues of ``progress''.  An
>> official release every 3 months is more than enough progress for my
>> taste.
> 
> 
> Not if there's nothing much new in it.  Which is a bit of an
> exaggeration, before anyone calls me on it - but still pretty well
> expresses my point.

It is closer to 6 months.  The releases are not tied to specific new 
features (as that is consistently a schedule derailing disaster).  Even 
with that time frame the releases consistently contain significant new 
features or fixes.

The next release, for instance, will contain a significant upgrade to 
MI.  That is critical to the GNU project and GDB.  It will also contain 
objective C.  Depending on how long this debate drags out for, it will 
also contain remote file I/O, remote gprov/gcov, Motorola's SPE and 
Altivec simulators, ....

(at a less visible level, it also contains a rewritten frame and 
register framework)

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

Daniel Berlin,

You and your track record are a case in point for why it is important to 
for GDB developers to both receive and respect peer review.  I find it 
extreamly ironic that you, of all people, should be arguing that the 
system is stifling.

Yes, the system was a barrier to you but that was for a very good 
reason.  Please don't try to use yourself as the sob story.  The only 
real mistake in your case was to not step in earlier and see you given 
the boot.

Andrew

Re: [maint] The GDB maintenance process

[Daniel Berlin]

On Thursday, February 20, 2003, at 11:23  AM, Andrew Cagney wrote:

> Daniel Berlin,
>
> You and your track record are a case in point for why it is important 
> to for GDB developers to both receive and respect peer review.
While that's kind of you Andrew, its also irrelevant.
>  I find it extreamly ironic that you, of all people, should be arguing 
> that the system is stifling.
That's also nice, but irrelevant.
Personal attacks do not answer any points raised.
> Yes, the system was a barrier to you but that was for a very good 
> reason.  Please don't try to use yourself as the sob story.
Where did I do so?

> The only real mistake in your case was to not step in earlier and see 
> you given the boot.
>
Grow up Andrew.
Do i answer your emails with personal attacks?

> Andrew
>

Re: [maint] The GDB maintenance process

[Daniel Berlin]

On Thursday, February 20, 2003, at 11:23  AM, Andrew Cagney wrote:

> Daniel Berlin,
>
> You and your track record are a case in point for why it is important 
> to for GDB developers to both receive and respect peer review.  I find 
> it extreamly ironic that you, of all people, should be arguing that 
> the system is stifling.
>
> Yes, the system was a barrier to you but that was for a very good 
> reason.  Please don't try to use yourself as the sob story.  The only 
> real mistake in your case was to not step in earlier and see you given 
> the boot.

Oh, and just to avoid you rewriting history, you'll note i quit, 
because Jim refused to even review my patches for months, because he 
felt they were ugly.  Which of course, regardless of whether they *are* 
ugly or not, doesn't change the fact that they are still supposed to be 
reviewed, and the reviews are supposed to be useful.  Nobody "booted" 
me.
Not that facts would have much influence on your viewpoint.

I've changed since I worked on GDB.  I'll happily admit i submitted 
tons of crappy code to GDB.  But it wasn't reviewed anyway.  It's only 
when i went to work on GCC, and actually got reviews of crappy patches 
telling me why they were crappy *constructively* (by people who 
actually clearly understood the code in question), rather than 
*destructively* (or by raising points that were completely irrelevant), 
that I actually became a better coder.

GDB as a project isn't good to contributors. It's processes are not 
helpful in maintaining or nurturing a community of maintainers and 
future maintainers.   Can you point to anyone who will take over GDB 
when the current maintainers are gone?  In GCC, I can.  You guys have 
basically trashed/ignored anyone who has had any potential.

I grew up Andrew.
Maybe you and GDB should too?

> Andrew

>

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Of course, if contributors are frustrated by the slow review rate, let's
> try to improve that (see my other mail).  But let's not obscure our view
> of the problem by discussing abstract issues of ``progress''.  An
> official release every 3 months is more than enough progress for my
> taste.
> 
> 
> Not if there's nothing much new in it.  Which is a bit of an
> exaggeration, before anyone calls me on it - but still pretty well
> expresses my point.
> 
> It is closer to 6 months.  The releases are not tied to specific new features (as that is consistently a schedule derailing disaster).  Even with that time frame the releases consistently contain significant new features or fixes.
> 
> The next release, for instance, will contain a significant upgrade to MI.  That is critical to the GNU project and GDB.  It will also contain objective C.  Depending on how long this debate drags out for, it will also contain remote file I/O, remote gprov/gcov, Motorola's SPE and Altivec simulators, ....
> 
> (at a less visible level, it also contains a rewritten frame and register framework)

PS: Hmm, this makes me think of `boiled frogs'.

Re: [maint] The GDB maintenance process

[Daniel Berlin]

On Thursday, February 20, 2003, at 11:23  AM, Andrew Cagney wrote:

> Daniel Berlin,
>
> You and your track record are a case in point for why it is important 
> to for GDB developers to both receive and respect peer review.  I find 
> it extreamly ironic that you, of all people, should be arguing that 
> the system is stifling.
>

And finally, I find it extremely ironic that, you, a person whose main 
complaint with me was that i wouldn't take criticism and i didn't 
respect peer review from people i apparently didn't respect, won't take 
criticism or respect peer review from people he apparently doesn't 
respect.

Pot, kettle, black.

--Dan

RE: [maint] The GDB maintenance process

[Zaretskii Eli]

This message was scanned for viruses and other malicious code by PrivaWall.


This mail was sent from ELTA SYS LTD.


> From: Daniel Berlin [mailto:dberlin@dberlin.org] 
> Sent: Wednesday, February 19, 2003 3:24 PM
> 
> > I guess I just don't see this to be as much of a problem as  others
do.
> > For one thing, with the higher entropy level, more development
actually
> > happens.
> Bingo.
> I don't think we should stall development (and in the 
> extreme,  even if 
> it means we can't make quality releases any day of the year) because 
> mistakes occasionally happen in patches, or because not every 
> maintainer in existence has said something about a patch.  That's a 
> recipe for no progress.

For some definition of ``progress''.

Who said that adding code at a faster rate at the price of having more
bugs is more ``progress'' than what we have now?  There are people out
there who need GDB to actually do something _useful_, not just to debug
and/or develop GDB itself, you know.  What about frustration of those
GDB users when their favorite feature is broken by some
committed-before-review patch that adds a hot new feature?  Does that
ever count?

Does anyone remember that latest GCC releases are practically unusable
for any production-quality work due to bugs?  Does anyone even care?

I say thanks God for slower development of GDB.  At least I can _debug_
buggy code produced by buggy development tools ;-)

Of course, if contributors are frustrated by the slow review rate, let's
try to improve that (see my other mail).  But let's not obscure our view
of the problem by discussing abstract issues of ``progress''.  An
official release every 3 months is more than enough progress for my
taste.


This message is processed by the PrivaWall Email Security Server. 

Re: [maint] The GDB maintenance process

[Eli Zaretskii]

> Date: Thu, 20 Feb 2003 10:16:46 -0500
> From: Daniel Berlin <dberlin@dberlin.org>
> >
> > Who said that adding code at a faster rate at the price of having more
> > bugs is more ``progress'' than what we have now?
> 
> Who said we'd necessarily have more bugs?
> Can you back this up?

I think we have too many bugs already.  How about computing some
statistics about how many changes repair breakage made by previous
changes?

Committing unreviewed changes will necessarily make the situation
worse.

> > There are people out
> > there who need GDB to actually do something _useful_, not just to debug
> > and/or develop GDB itself, you know.
> You've assumed that it would make GDB completely unusable.

I didn't say that.  But breaking even a single important feature
could make someone out there totally helpless to find some bug.

> > What about frustration of those
> > GDB users when their favorite feature is broken by some
> > committed-before-review patch that adds a hot new feature?
> > Does that
> > ever count?
> 
> Does it ever happen?

Yes, it does.  A simple reading of the ChangeLog will show you.

> This isn't grade school anymore, we all know how to write good code.

Yes, we do.  But GDB's internals being as complex and not too
documented as they are, it doesn't surprise me that excellent
programmers still break things.

> > Does anyone remember that latest GCC releases are practically unusable
> > for any production-quality work due to bugs?  Does anyone even care?
> >
> Latest?
> There was *one* release that was like this, GCC 3.0.

That was was simply broken.  But even the latest 3.2.x series are too
buggy for serious production-quality code, at least in systems where
reliability is a requirement.  People are still using 2.95.x for that
reason.

> And we *knew* it was going to be like this, because of the new ABI.  
> You couldn't do much about it.

Yes, you could.  You could refrain from releasing it.

Re: [maint] The GDB maintenance process

[Daniel Berlin]

On Sun, 23 Feb 2003, Eli Zaretskii wrote:

> > Date: Thu, 20 Feb 2003 10:16:46 -0500
> > From: Daniel Berlin <dberlin@dberlin.org>
> > >
> > > Who said that adding code at a faster rate at the price of having more
> > > bugs is more ``progress'' than what we have now?
> >
> > Who said we'd necessarily have more bugs?
> > Can you back this up?
>
> I think we have too many bugs already.  How about computing some
> statistics about how many changes repair breakage made by previous
> changes?

Sure, but it'll take me a few days to do so.
It's much harder to sort out fixes that repair breakage than it was to
compute stats on how long it takes to review patches.
>
> Committing unreviewed changes will necessarily make the situation
> worse.

>
> > > There are people out
> > > there who need GDB to actually do something _useful_, not just to debug
> > > and/or develop GDB itself, you know.
> > You've assumed that it would make GDB completely unusable.
>
> I didn't say that.  But breaking even a single important feature
> could make someone out there totally helpless to find some bug.
>
I doubt it would break a single important feature.

> > > What about frustration of those
> > > GDB users when their favorite feature is broken by some
> > > committed-before-review patch that adds a hot new feature?
> > > Does that
> > > ever count?
> >
> > Does it ever happen?
>
> Yes, it does.  A simple reading of the ChangeLog will show you.
Um, no, it doesn't.  I can't find any instances in the changelogs from the
past 2 years  of a non-target specific problem in a feature that was
introduced and then fixed.

Here's all the reversions and fixes caused by previous changes i can
findfrom the past 2 years:
I've annotated what feature each affected.
I've ignored clearly accidental checkins (IE those marked as accidental
and committed right after the accident), and documentation bugs (2
occurred).
From ChangeLog:
        lost in the previous patch. (extremely small annotation bug)
        compilation error in the previous revision (hpread bug).
        failure introduced in the previous change (alpha gdbarch bug).
From ChangeLog-2002
        valops.c, value.h: Revert previous change (Reversion of committing
	of objc stuff prematurely, not a feature breakage).
       * sh-tdep.c: Clean up code erroneously reintroduced by previous
	(SH-only bug)
        * kod.c (kod_set_os): Revert previous change.  Is called by ``info
(kod specific problem)
	  Fix last change to use regcache_collect instead of referencing
        registers[] directly (m68k-only bug).


Granted, i  just got new contacts and they are bothering my eyes, so i
only spent 30 minutes going over the changelogs, so it's possible i missed
one or two.

> That was was simply broken.  But even the latest 3.2.x series are too
> buggy for serious production-quality code, at least in systems where
> reliability is a requirement.  People are still using 2.95.x for that
> reason.

Sorry, i simply don't believe this.
Every piece of data we have shows exactly the opposite (that people have
abandonded 2.95 for 3.2.2 because of reliability reasons, and those that
stick to 2.95 are doing it not because of reliability but instead because
of speed).

I'd really like to know where you are getting this from.


> > And we *knew* it was going to be like this, because of the new ABI.
> > You couldn't do much about it.
>
> Yes, you could.  You could refrain from releasing it.

Actually, that wouldn't help.  You can't forsee every bug, no matter what
you try.  Not releasing it wouldn't enable us to find more bugs, since we
don't have every piece of code in existence that g++ gets run on.
The likelyhood that 3.0 would have more bugs, no matter when we released
it, was very high, simply based on the nature of the changes.
There were numerous pretests, and numerous weeks of people simply doing
only bug fixing and ensuring that it was as bug-free as possible.
The patches that went into 3.0 received *more* review than patches that
went into previous releases received.

Therefore, maybe you'd like to explain exactly what should have been
done while we "refrained from releasing it", given that there were no high
priority  bugs left to fix, no major bugs left to fix that affected a
primary release platform, and no large bases of code that anyone had
access to failing to run properly, since I can't exactly see what more
you wanted done.

We can't fix unknown, unreported bugs, that don't occur for anyone doing
development, on any piece of code they have, until they are reported.
Most aren't reported until you release (even pre-tests  and snapshots
don't receive the same level of use, no matter how long you put them out
for).

--Dan

Re: [maint] The GDB maintenance process

[Eli Zaretskii]

> Date: Sat, 22 Feb 2003 20:57:32 -0500 (EST)
> From: Daniel Berlin <dberlin@dberlin.org>
> 
> > > > What about frustration of those
> > > > GDB users when their favorite feature is broken by some
> > > > committed-before-review patch that adds a hot new feature?
> > > > Does that
> > > > ever count?
> > >
> > > Does it ever happen?
> >
> > Yes, it does.  A simple reading of the ChangeLog will show you.
> Um, no, it doesn't.  I can't find any instances in the changelogs from the
> past 2 years  of a non-target specific problem in a feature that was
> introduced and then fixed.
> 
> Here's all the reversions and fixes caused by previous changes i can
> findfrom the past 2 years:
> I've annotated what feature each affected.
> I've ignored clearly accidental checkins (IE those marked as accidental
> and committed right after the accident), and documentation bugs (2
> occurred).
> From ChangeLog:
>         lost in the previous patch. (extremely small annotation bug)
>         compilation error in the previous revision (hpread bug).
>         failure introduced in the previous change (alpha gdbarch bug).
> From ChangeLog-2002
>         valops.c, value.h: Revert previous change (Reversion of committing
> 	of objc stuff prematurely, not a feature breakage).
>        * sh-tdep.c: Clean up code erroneously reintroduced by previous
> 	(SH-only bug)
>         * kod.c (kod_set_os): Revert previous change.  Is called by ``info
> (kod specific problem)
> 	  Fix last change to use regcache_collect instead of referencing
>         registers[] directly (m68k-only bug).

Be sure not to miss entries like this:

	(dwarf2_build_frame_info): Corrected handling of eh_frame.

or this:

	* dwarf2cfi.c (context_cpy): Copy registers correctly.

or this:

	* symfile.c (add_filename_language): Fix wrong xrealloc size argument.

It is my personal impression that a very large portion of GDB
development is devoted to fix bugs in existing code.  Perhaps my
impression is somehow lopsided, I don't know.

> > That was was simply broken.  But even the latest 3.2.x series are too
> > buggy for serious production-quality code, at least in systems where
> > reliability is a requirement.  People are still using 2.95.x for that
> > reason.
> 
> Sorry, i simply don't believe this.
> Every piece of data we have shows exactly the opposite (that people have
> abandonded 2.95 for 3.2.2 because of reliability reasons, and those that
> stick to 2.95 are doing it not because of reliability but instead because
> of speed).

[Who are ``we''?]

> I'd really like to know where you are getting this from.

I'm getting this from my daily work.  Perhaps we write different types
of software.  I work for a large firm that develops software in C that
needs to be extremely reliable.  As long as people can break programs
by recompiling them with latest GCC versions, while it runs fine with
GCC 2.x (I still have 2.7.2.1 around, by the way), GCC 3.2.x will be
suspect as far as I'm concerned.  Sorry, can't tell more: I cannot
afford being a GCC hacker in addition to being a GDB hacker, an Emacs
hacker, and a Texinfo hacker (to name just a few).

> > > And we *knew* it was going to be like this, because of the new ABI.
> > > You couldn't do much about it.
> >
> > Yes, you could.  You could refrain from releasing it.
> 
> Actually, that wouldn't help.  You can't forsee every bug, no matter what
> you try.  Not releasing it wouldn't enable us to find more bugs, since we
> don't have every piece of code in existence that g++ gets run on.
> The likelyhood that 3.0 would have more bugs, no matter when we released
> it, was very high, simply based on the nature of the changes.

Yes, we all know the theory: there's a fine balance to strike between
reliability and development pace.  To me, the point where GCC cuts it
is a wrong one.

But I don't want to argue about GCC development too much.  We are
discussing GDB, not GCC.  I only mentioned GCC because it was used by
others in this thread to compare against GDB.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I mostly agree with Daniel here.  We have too many single points of
> failure.  I still have testsuite patches sitting in my tree, dating
> months back since they were never approved.  Similarly, Daniel
> probably has been held back more than once since I wasn't able to
> review threads-related patches in a timely fashion.  I think we should
> allow our global maintainers to approve patches even for parts of GDB
> where we have a specific maintainer, if they feel they have the
> necessary expertise.

Depends on what `single point failure' means.  If you look through the 
MAINTAINERS file you'll notice that all but one of the problem areas is 
double, if not triple maintained.  Even with all that dedundency, 
patches in certain areas continues to stall.  While there might be three 
maintainers, the reality is only one or none are reliable.

The first, and most obvious thing is for the global maintainers to 
review their current number of responsibilities and compare that against 
their actual level of commmitment.  I'm down to just target/arch, 
remote, mips and sim/ppc.

The second, is for maintainers to always be on the lookout for potential 
new maintainers (global or not) and, where possible, step aside to let 
new commers in.

Finally, the GDB `tradition' (that appears to have been forgotten) is to 
up front reach consensus on stuff so that the final patches are trivial 
to review.  cf, the block.[hc] change.

enjoy,
Andrew

Re: [maint] The GDB maintenance process

[Michael Elizabeth Chastain]

Oh, man, what a thread.

I think it's hopeless to argue about the nature of gdb on a mailing list.
We all have opinions and are unlikely to change them.  I think the best
we can actually do is craft proposals that are good in some people's
views, neutral in other people's, and aren't horrible in anybody's eyes.

My opinions:

. We need two or three test suite maintainers.  I think we should
  add some co-maintainers for the test suite right now, just as we
  did for the gdb.c++ subdirectory.

. We have a problem integrating new contributors.  This is tangled up
  with the bigger problem that the doco for new contributors is
  spread out over several places.  I would like to tackle this
  problem after the 5.4/6.0 release.

. The basic issue with patches is that they flow into our mailboxes
  faster than they flow out.  No patch management system can solve
  this problem directly; it simply introduces new and fancier buffer
  mechanisms.  The only ways to solve it are to devote more resources
  to patch review or find ways to review patches more quickly
  (for example, flat out reject patches with the note "we don't have
  the resources to review this").

. I am comfortable with a low degree of pre-commit testing as long as
  we have healthy regression coverage AND people prioritize the
  regression bugs in front of committing their next patch.  For example,
  I'd like someone to investigate pr gdb/1039.  I don't want this kind
  of stuff to build up.  It's very easy for a developer to have a
  never-ending stream of new stuff and never fix the problems introduced
  by last week's stuff.

. My estimate of gdb regressions is that in the area I cover
  (native i686-pc-linux-gnu) there is roughly 0.2 to 0.5 regressions
  per week.  See the gdb-testers archive and count "New Bugs Detected".

Michael C

Re: [maint] The GDB maintenance process

[Mark Kettenis]

I mostly agree with Daniel here.  We have too many single points of
failure.  I still have testsuite patches sitting in my tree, dating
months back since they were never approved.  Similarly, Daniel
probably has been held back more than once since I wasn't able to
review threads-related patches in a timely fashion.  I think we should
allow our global maintainers to approve patches even for parts of GDB
where we have a specific maintainer, if they feel they have the
necessary expertise.

Mark

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> -Branches should have an owner.
> The owner can set further policy for a branch, but may not change the 
> ground rules.  In particular, they can set a policy for commits (be it 
> adding more reviewers or deciding who can commit).

Yep.

> -All commits to a branch must be covered by an assignment
> This saves us from the situation where a branch might become contaminated.

Um, er, yes, `obviously' (How did I forget that one :-( :-).

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> When I was maintaining Guile, I ended up replacing my dictatorship
> with a group of four maintainers --- Mikael Djurfeldt, Maciej
> Stachowiak, and Marius Vollmer.  Any one of those guys I trusted as
> much as I trust myself.  There was no risk in making them my peers,
> since I was just as likely to make a poor decision as they were (if
> not more so), and if they didn't like something, it would certainly
> benefit from a re-examination.  Surely we have people in the GDB
> community in addition to Andrew who have earned that level of trust.

The global write maintainers already have this responsability.  Every so 
often there needs to be a `judgment call'(1) which falls to the gdb head 
(again `the buck stops').  On the whole, though, the group should be 
discusses technical issues selecting designs based solely on their 
relative merits.

One thing I do do though, is nip debates that are re-flogging head 
issues, in the bud.  Proposals for making GDB LGPL, for instance.

Andrew

(1) The last one I remember making was adopting gdb/cli/cli-*.[hc] for 
cli file names and for that one I tossed a coin.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I was talking to a friend of mine about the way GDB is run, and he was
> amazed that we give individual people complete power, and complete
> responsibility, for sections of code.  Everyone is going to be wrong
> sometimes, he said, and it's easy to protect against, without being
> too bureaucratic.

Interesting spin.

I think the concept such as `the Buck stops' and having the final 
decision, are very different to `complete power'.   The former is about 
responsibility.

A maintainer, who is a member of a larger team, takes on certain 
responsibilities, such making the final decision when reviewing a patch. 
  The maintainer is assumed to have sufficiently good judgment, however, 
they are also assumed to be taking on board the comments and suggestions 
being made by their peers.

Andrew

RE: [maint] The GDB maintenance process

[Zaretskii Eli]

This message was scanned for viruses and other malicious code by PrivaWall.


This mail was sent from ELTA SYS LTD.


> From: Jim Blandy [mailto:jimb@redhat.com] 
> Sent: Wednesday, February 19, 2003 4:18 AM
> 
> - It's true that "... some maintainers should try to review patches in
>   their areas of responsibility more often", but merely saying so
>   doesn't have any effect.  Folks have been saying that ever since
>   Cygnus loosened its grip on GDB and the process opened to the
>   public.  That statement seems to express a hope that the maintainers
>   will somehow "wake up" and everything will get better.  It's been
>   years, now, and we need to stop waiting for this to happen.  Let's
>   work with the people we've got, rather than hoping they'll transform
>   themselves somehow.

GDB maintenance is not the only place that needs to deal with that
problem.  We don't have to reinvent the wheel, or feel that we face an
insoluble problem, and neither do we need to make drastic changes in the
current commit paradigms.

More to the point, if we think that some maintainers don't respond fast
enough, we could bring more maintainers on board.  The primary
candidates for becoming new maintainers are those people who push
significant patches in the areas where the current maintainers are slow.
In other words, let those who are unhappy with the current situation put
their money where their mouth is and help with the burden.

We could also ask a maintainer to step down if he/she cannot keep up
with his/her duties in a timely manner, or we could grant the head
maintainer (or a group of veteran maintainers) powers to declare that
someone has effectively stepped down due to lack of responsiveness.

These are all minor modifications to the existing model.  If someone
thinks that they will never be effective enough, let them explain why.

I'm not against changing the model to something like commit-then-review,
mind you.  But please don't underestimate the implications of this on
the social dynamics within our team: reverting changes, while
technically easy, tends to leave bad after-taste and hurts
relationships; the arguments about whether to revert a patch tend to be
ugly.  We should ask ourselves if we are willing to pay that price.


This message is processed by the PrivaWall Email Security Server. 

Re: [maint] The GDB maintenance process

[Richard Earnshaw]

> One thing GCC(4) and GDB are now is encouraging exprementation on 
> branches development to always occure on branches cut from the the 
> relevant repository.  For GDB we've had both success stories but also 
> disasters.  With that in mind, and looking at the GCC / GDB success 
> stories, I'd suggest the following guidelines:
> 
> - branches shall post all commits
> They don't need approval but can be commented on.
> 
> - branches shall to be focused
> The interps branch started out too large with too many changes - look at 
> the size of the final commit compared to that branch at its peak.  Much 
> time was lost because the branch started with too much lint :-(
> 
> - branches shall track mainline.
> This keeps the level of divergence under control.  It also keeps the 
> pressure on developers to push cleanups and other stuff into the mainline.

I think there's a few other ground-rules that ought to apply, particularly 
with respect to the first point:

-Branches should have an owner.
The owner can set further policy for a branch, but may not change the 
ground rules.  In particular, they can set a policy for commits (be it 
adding more reviewers or deciding who can commit).

-All commits to a branch must be covered by an assignment
This saves us from the situation where a branch might become contaminated.

R.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I agree with what Daniel has said here.  I'm concerned that some
> people have misunderstood his points, so I'll put it differently.
> 
> I think GDB could get better use of the contributors it has now by
> adjusting the rules for patch approval.
> 
> - Slow patch review is a real problem on GDB --- even acknowledging
>   the legitimate reasons for some delays that Elena has mentioned.

I found myself babysitting a short list of largely dormant maintainers 
using regular pings.  That was grosely inefficient.  I've now given that 
one up and have moved on to filing unreviewed patches in the bug 
database - worse for the contributor waiting on a dormant maintainer but 
better for me and the other active maintainers.

It has been in place for ~a month and, in my opinion, has definitly 
improved things.  There is a one-stop-shop where active [global] 
maintainers can either find a backlog of patches or areas that need work.

I just wish that the tool being used was less primative :-(

> - It's true that "... some maintainers should try to review patches in
>   their areas of responsibility more often", but merely saying so
>   doesn't have any effect.  Folks have been saying that ever since
>   Cygnus loosened its grip on GDB and the process opened to the
>   public.  That statement seems to express a hope that the maintainers
>   will somehow "wake up" and everything will get better.  It's been
>   years, now, and we need to stop waiting for this to happen.  Let's
>   work with the people we've got, rather than hoping they'll transform
>   themselves somehow.

And the vast majority of maintainers do a pretty good job.

> - If we accept that the maintainers' behavior is stable, then the next
>   alternative is to adjust the organization that they operate under.
>   Is there some way to re-organize the people we have, accepting their
>   job committments and personal limitations (I have myself in mind
>   here as much as anyone else, so don't be affronted), so that things
>   progress better?  Is the current organization optimal?

Stable or reliable?  Most maintainers are reliable, a small few, though, 
seem to keep falling asleep at the wheel.

Having such people as developers isn't a problem.  Having such people 
hold key responsabilities (such as being in the loop to review patches) is.

Andrew

Re: [maint] The GDB maintenance process

[Christopher Faylor]

On Wed, Feb 19, 2003 at 03:17:09PM -0800, Jason Molenda wrote:
>On Wed, Feb 19, 2003 at 03:39:50PM -0500, Christopher Faylor wrote:
>>I know that Jason likes svn but I haven't used it.  I'm not sure that
>>it would be easy to convince me to spend any time trying to convert
>>everything to a new source control system.  CVS is nice because it
>>lives everywhere and is a known commodity.
>
>I don't think I saw anyone suggesting alternative SCMs...  cagney wrote
>"sware, subversion, sourceforge", but he probably meant to write
>"subversions", as in "subversions.gnu.org".

I probably responded one message too late.  David Carlton said this:

>It might be nice if GDB were using a source code management tool that
>didn't depend on having a single repository, making it easier for
>people to maintain public branches elsewhere but to still sync them
>with an official branch.

So when I saw Andrew mention subversion, I thought we were still talking
about source control systems rather than systems which run source
control systems.

cgf

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> How is the 6 months inactivity period going to help? None of the
> current gdb maintainers perticipating in this discussion has ever
> disappeared for that long, really.  If some person has disappeared
> from radar, like relocated, changed e-mail, etc, the person has been
> removed from the list. I can think of one such case.

There was one such case.  There have also been two dummy spits; and, 
after lobying, a number of people `stepping down'.

One thing I've noticed is that people struggle to appreciate the 
difference between `no maintainer' and `inactive maintainer'.  There is 
a definite resistance to `letting go'.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> On Wed, Feb 19, 2003 at 06:57:01PM -0500, Andrew Cagney wrote:
> 
> 
>> > Using GNATS as the infrastructure to track patches is pathetic.
> 
>> 
>> Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
>> failing hard disk'.
> 
> 
> Well, yes. :-)  I didn't mean "you, the fellow who has put patches
> into gnats, are a fool" -- I meant that the overhead over putting
> patches in gnats is too high compared with just sending them to
> gdb-patches.  IMHO this is a method that will fail, which is why
> I dragged my feet when Elena originally requested the gdb-patches
> gnats database be set up.  Ignoring the fact that gnats is a bug
> tracker--not a magical patch tracking database--as long as it isn't
> at the center of every developer/maintainer's patch workflow, it
> will be doomed to irrelevance.

Actually the overhead is effectively zero.  Any patch not touched for a 
week gets run through a script that turns it into a gnats entry.

Once there, I and everyone else can search it.  I know of non-mainstream 
developers that perfer this as it is easier to pick up a task (and 
easier to track than either gdb@ or gdb-patches@).

It sux, but sux less than not doing it.  It is interum measure that will 
hopefully be replaced by bugzilla.

Andrew

Re: [maint] The GDB maintenance process

[Elena Zannoni]

Jason Molenda writes:
 > On Wed, Feb 19, 2003 at 06:57:01PM -0500, Andrew Cagney wrote:
 > 
 > > > Using GNATS as the infrastructure to track patches is pathetic.
 > > 
 > > Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
 > > failing hard disk'.
 > 
 > Well, yes. :-)  I didn't mean "you, the fellow who has put patches
 > into gnats, are a fool" -- I meant that the overhead over putting
 > patches in gnats is too high compared with just sending them to
 > gdb-patches.  IMHO this is a method that will fail, which is why
 > I dragged my feet when Elena originally requested the gdb-patches
 > gnats database be set up.  Ignoring the fact that gnats is a bug

It must have been my evil twin, because I don't remember asking for
this (I am in favor of something like it, though). To be honest, it
was a project that Jim Blandy started but wasn't finished. It was
abandoned because there were problems with including a patch
preserving spaces, or something like that.

 > tracker--not a magical patch tracking database--as long as it isn't
 > at the center of every developer/maintainer's patch workflow, it
 > will be doomed to irrelevance.
 > 
 > It's got to be easy, it's got to be relevant, and it's gotta be the
 > way everything is done.
 > 
 > > > Using mailing lists to track patches is annoying.
 > > 
 > > Er, you can't track patches using a mailing list.  A mailing list can be 
 > > used to submit/discuss patches.  It can't be used to track their state. 
 > >   that needs a database.
 > 
 > I was speaking loosely - I meant the combination of the mailing
 > list and the web archives of that mailing list.  The mailing list
 > web archives are a being used as the patch repository right
 > now--people use URLs into the archives to refer to old patches,
 > they use google or the htdig search engine to find old patches,
 > and they grope around blindly to figure out what ever happened with
 > a given patch.
 > 
 > > Time to install aegis, ay?
 > 
 > I've never looked at Aegis, so I can't say.  First the gdb maintainers
 > and developers need to decide what they want and will use, then
 > make it exist; not look at what exists and settle for it.  Maybe
 > Aegis is exactly what we'd all love in a magical patch tracking
 > database and we can use it as-is, but IMHO it's too early in that
 > discussion to care one way or another.
 > 
 > 
 > J

Re: [maint] The GDB maintenance process

[Jason Molenda]

On Wed, Feb 19, 2003 at 06:57:01PM -0500, Andrew Cagney wrote:

> > Using GNATS as the infrastructure to track patches is pathetic.
> 
> Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
> failing hard disk'.

Well, yes. :-)  I didn't mean "you, the fellow who has put patches
into gnats, are a fool" -- I meant that the overhead over putting
patches in gnats is too high compared with just sending them to
gdb-patches.  IMHO this is a method that will fail, which is why
I dragged my feet when Elena originally requested the gdb-patches
gnats database be set up.  Ignoring the fact that gnats is a bug
tracker--not a magical patch tracking database--as long as it isn't
at the center of every developer/maintainer's patch workflow, it
will be doomed to irrelevance.

It's got to be easy, it's got to be relevant, and it's gotta be the
way everything is done.

> > Using mailing lists to track patches is annoying.
> 
> Er, you can't track patches using a mailing list.  A mailing list can be 
> used to submit/discuss patches.  It can't be used to track their state. 
>   that needs a database.

I was speaking loosely - I meant the combination of the mailing
list and the web archives of that mailing list.  The mailing list
web archives are a being used as the patch repository right
now--people use URLs into the archives to refer to old patches,
they use google or the htdig search engine to find old patches,
and they grope around blindly to figure out what ever happened with
a given patch.

> Time to install aegis, ay?

I've never looked at Aegis, so I can't say.  First the gdb maintainers
and developers need to decide what they want and will use, then
make it exist; not look at what exists and settle for it.  Maybe
Aegis is exactly what we'd all love in a magical patch tracking
database and we can use it as-is, but IMHO it's too early in that
discussion to care one way or another.


J

Re: [maint] The GDB maintenance process

[Elena Zannoni]

Jim Blandy writes:
 > Andrew Cagney <ac131313@redhat.com> writes:
 > 
 > > > - It's true that "... some maintainers should try to review patches in
 > > >   their areas of responsibility more often", but merely saying so
 > > >   doesn't have any effect.
 > > 
 > > For the record your name is top of the list of that `some maintainers'.
 > 
 > 
 > I think the explicit hierarchy we have now, outlined in MAINTAINERS,
 > is a real problem in this sense.  It's a big, public, political deal
 > to rearrange that hierarchy.  There are other systems where the
 > processes of promoting promising contributors and clearing dead wood
 > happen smoothly and automatically, without confrontation.  People
 > contribute as they are able, and leaders emerge and recede in a
 > natural way, not by fiat.  The Apache system, for example, encourages
 > newcomers to acquire expertise in different areas, and allows less
 > responsive people to simply fall to the wayside as irrelevant.
 > 
 > These systems are in widespread use, and I think some are even
 > well-documented, like: http://httpd.apache.org/dev/guidelines.html

"Membership as a Committer is by invitation only and must be approved
 by consensus of the active Apache PMC members. A Committer is
 considered inactive by their own declaration or by not contributing
 in any form to the project for over six months. An inactive member
 can become active again by reversing whichever condition made them
 inactive (i.e., by reversing their earlier declaration or by once
 again contributing toward the project's work). Membership can be
 revoked by a unanimous vote of all the active PMC members (except the
 member in question if they are a PMC member)."


How is being a member of the 'Committers' by "invitation only" not
going to be another political deal?

How is the 6 months inactivity period going to help? None of the
current gdb maintainers perticipating in this discussion has ever
disappeared for that long, really.  If some person has disappeared
from radar, like relocated, changed e-mail, etc, the person has been
removed from the list. I can think of one such case.


I don't like this one:

"Ideas must be review-then-commit; patches can be
 commit-then-review. With a commit-then-review process, we trust that
 the developer doing the commit has a high degree of confidence in the
 change."

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> 
> 
> We've got two separate discussions here:  The social conventions
> and procedures for how patches are discussed/applied, and the
> infrastructure to support those things.
> 
> Using GNATS as the infrastructure to track patches is pathetic.

Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
failing hard disk'.

> Using mailing lists to track patches is annoying.

Er, you can't track patches using a mailing list.  A mailing list can be 
used to submit/discuss patches.  It can't be used to track their state. 
  that needs a database.

Time to install aegis, ay?

Andrew

Re: [maint] The GDB maintenance process

[Jason Molenda]

On Wed, Feb 19, 2003 at 11:29:13AM -0500, Andrew Cagney wrote:
> 
> > Which reminds me.  We've got two GNATS databases set up for GDB: 'gdb'
> > and 'gdb-patches'.  Should we use the gdb-patches GNATS database to
> > separate them from bug reports?
> 
> No!!!!  That gdb-patches database should be deleted.  It's dead.
> 
> People already have to track:
> 
> - gdb@
> - gdb-patches@
> - gnats@
> 
> and that is to complicated for some.  At least, by having both patches 
> and bugs in a single database, we've a one stop-shop.  A better change 
> is to dump gdb-patches@


We've got two separate discussions here:  The social conventions
and procedures for how patches are discussed/applied, and the
infrastructure to support those things.

Using GNATS as the infrastructure to track patches is pathetic.
Using mailing lists to track patches is annoying.

For a minute, though, imagine a magical patch tracking database.
It would be at the core of the patch workflow, not something glommed
on to the side--anything that says "you send your note to gdb-patches
and then you enter it into the patch database" is missing the point.
It wouldn't be a bug database, it would be a patch database--it's
got to track updates to the patch, it's got to relate groups of
patches or patches dependant on other patches.  It'll track the
discussion about patches.  It could track various maintainer's
opinions on a patch (I like the -1/+0/+1 Apache scheme); it could
track when the patch is blocked on a particular person (submitter,
a maintainer), or maybe when the patch is waiting approval by any
one of several people.  It could even relate GNATS bugs and patches
that are attempting to fix those bugs, so when someone sees a bug
in the GNATS database that hasn't yet been fixed, they can click
over to the magical patch tracking database and see why that patch
isn't in yet.

All of these transactions are still visible on a gdb-patches list,
of course, but the mailing list is a read-only view on to what's
happening.  All additional comments or follow-ups are routed through
the magical patch tracking database, and reflected back out to the
gdb-patches list.


I've heard Sourceforge has such a tool.  JimI was telling me at
lunch how they track the tcl patches this way.  And there is a
separate mechanism that tcl and Python both use for larger-scope
proposals ("PEP" for Python, a "Python Enhancement Proposal").
For instance, here's a PEP to add a boolean type to Python:
	http://www.python.org/peps/pep-0285.html
And it has a link into the Patch Tracker sourceforget database 
for the patch to implement the proposal:
	http://python.org/sf/528022



Don't fall into the trap of saying "the only two ways patches can
be tracked are the gdb-patches mailing list or gnats".  Let's start
by imagining a patch database designed around our actual workflow,
one which would require a minor change in how we work and add a
tiny bit of extra patch-management overhead, and yield benefits of
not losing track of patches.  

If such a magical patch tracking database exists, would it be worth
changing our workflow to use it?  I know that patch submitters would
eagerly jump on such a system, but unless all the actual maintainers
think it's a net gain it's not going anywhere.

Once that's all decided - what we want, and if we'd even use it if
we had it - then there's the minor matter of making it exist. :-)
Maybe bugzilla or gnats can be adapted, maybe another program can
be found from the net or a free bit of code from Sourceforge's
PatchTracker can be appropriated, or maybe someone gets the joyus
chance to write it from scratch.  


Before we get bogged down in implementation details and discussions,
it's critical to discuss what such a system would do, and if we're
all willing to push gdb patch work through it.  If we can't dream
up such a system, then let's be happy with gdb-patches and URLs
for patch references.

J

Re: [maint] The GDB maintenance process

[Jason Molenda]

On Wed, Feb 19, 2003 at 02:01:14PM -0500, Andrew Cagney wrote:
>Red Hat makes sourceware available as a public service.  It funds it 
>rather than owns it.  The machine is adminstered by a group of overseers 
>(chrisf and jasonm are two).

Minor correction - RH owns the physical sources.redhat.com system.
It pays for the network bandwidth, electricity, and the necessary
network IS resources.  The time spent actually administering the
box (almost entirely Chris Faylor these days) is outside the scope
of cgf's Day Job, so most administration is on a volunteer basis.

On Wed, Feb 19, 2003 at 03:39:50PM -0500, Christopher Faylor wrote:

> I know that Jason likes svn but I haven't used it.  I'm not sure that it
> would be easy to convince me to spend any time trying to convert
> everything to a new source control system.  CVS is nice because it lives
> everywhere and is a known commodity.  

I don't think I saw anyone suggesting alternative SCMs... cagney
wrote "sware, subversion, sourceforge", but he probably meant
to write "subversions", as in "subversions.gnu.org".


(svn is really interesting, and I expect I'll be playing with it
more this year, but it'll be a while before anyone wants to switch
their project to using it.  If nothing else, each developer in that
project has to install/set up svn before they can access the
repository.)

Jason

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Has anyone taken the time to go through the list of outstanding patches
> and categorize them by maintainership?

Already done, see the bug database.   Mind you only very recently but I 
think we're already seeing results.

> I'm wondering if this is just a problem with a couple of maintainers
> being unresponsive.
> 
> For instance, as a not-so-wild guess, I'd say that the symtab part of
> gdb frequently falls behind.  That may be because there are more
> symtab-related patches being supplied or it could be that I'm just
> imagining things.

There are more symtab patches but, thanks to one maintainer, they are 
moving.  Not sure about other categories, some do appear stalled :-(

> If there are some maintainers who just aren't responsive enough, for
> whatever reason, then the simple expedient of adding another maintainer
> for that specific part of gdb might go a long way towards fixing the
> backlog.

There is responsive and then there is reliable :-/

Andrew

Re: [maint] The GDB maintenance process

[Christopher Faylor]

Has anyone taken the time to go through the list of outstanding patches
and categorize them by maintainership?

I'm wondering if this is just a problem with a couple of maintainers
being unresponsive.

For instance, as a not-so-wild guess, I'd say that the symtab part of
gdb frequently falls behind.  That may be because there are more
symtab-related patches being supplied or it could be that I'm just
imagining things.

If there are some maintainers who just aren't responsive enough, for
whatever reason, then the simple expedient of adding another maintainer
for that specific part of gdb might go a long way towards fixing the
backlog.

However, if the patches are all over the board then this technique
probably won't work.

cgf

Re: [maint] The GDB maintenance process

[Jim Blandy]

Andrew Cagney <ac131313@redhat.com> writes:

> > - It's true that "... some maintainers should try to review patches in
> >   their areas of responsibility more often", but merely saying so
> >   doesn't have any effect.
> 
> For the record your name is top of the list of that `some maintainers'.

If you like.  I pick on myself often enough elsewhere in that message
that it should be clear I'm not trying to make others feel bad for
shortcomings I have myself.

In fact, this is exactly what we need to put behind us: we can spend
another few years feeling bad for being insufficiently responsive, and
treating it as a personal failure.  But that approach hasn't improved
things noticeably, so, is there something else we can do?  Given the
people we have, the community we have, and their known strengths and
weaknesses, what is the best way to organize them?  Can we improve on
what we're doing now?

I think the explicit hierarchy we have now, outlined in MAINTAINERS,
is a real problem in this sense.  It's a big, public, political deal
to rearrange that hierarchy.  There are other systems where the
processes of promoting promising contributors and clearing dead wood
happen smoothly and automatically, without confrontation.  People
contribute as they are able, and leaders emerge and recede in a
natural way, not by fiat.  The Apache system, for example, encourages
newcomers to acquire expertise in different areas, and allows less
responsive people to simply fall to the wayside as irrelevant.

These systems are in widespread use, and I think some are even
well-documented, like: http://httpd.apache.org/dev/guidelines.html

Re: [maint] The GDB maintenance process

[Andrew Cagney]

>> Fernando does this already.  He also sends out e-mail saying he is off 
>> line for a bit (he is down the hall from me which is an advantage). 
>> Other maintainers do not, sigh!  More of a concern is that repeated 
>> pings often result in radio silence.
> 
> 
> Yes, Fernando does this.  He also tends to disappear for long periods
> of time without doing it.

Fernando and I finding that my e-mail pings to him got black holed for 6 
months didn't help (this is where being able to walk down the hall is a 
disadvantage :-).   I don't know that others have the luxury of that excuse.

Andrew

Re: [maint] The GDB maintenance process

[Christopher Faylor]

On Wed, Feb 19, 2003 at 02:01:14PM -0500, Andrew Cagney wrote:
>Red Hat makes sourceware available as a public service.  It funds it 
>rather than owns it.  The machine is adminstered by a group of overseers 
>(chrisf and jasonm are two).

I know that Jason likes svn but I haven't used it.  I'm not sure that it
would be easy to convince me to spend any time trying to convert
everything to a new source control system.  CVS is nice because it lives
everywhere and is a known commodity.  Do we really want to think about
the headache of having people set up svn (or arch, or whatever) if they
want to develop with gdb?  Do we want to make gdb's repository separate
from, say, cygwin's or newlib's?  Seems like a lot of tricky work to me.

cgf

Re: [maint] The GDB maintenance process

[David Carlton]

On Wed, 19 Feb 2003 10:57:20 -0700, Kevin Buettner <kevinb@redhat.com> said:

> For the contract work that Red Hat itself does, Red Hat does not
> depend solely on the external (sources.redhat.com) respository.  Red
> Hat has an internal repository that it uses for customer work.  From
> time to time, the public repository is merged to Red Hat's internal
> repository.  Also, from time to time, work that was performed for a
> customer which couldn't immediately go into the public repository is
> pushed out to the public.

Thanks for the info; that sounds sensible.

It would be nice if there were a scheme that allowed such development
to go on smoothly while still making it as easy as possible for
changes in separate repositories to make it into the mainline.  I
don't really know to do that, though.

David Carlton
carlton@math.stanford.edu

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> On Feb 19,  9:33am, David Carlton wrote:
> 
> 
>> On Wed, 19 Feb 2003 09:54:50 -0500, Andrew Cagney <ac131313@redhat.com> said:
>> 
> 
>> > One thing GCC(4) and GDB are now is encouraging exprementation on
>> > branches development to always occure on branches cut from the the
>> > relevant repository.
> 
>> 
>> Would Red Hat be happy hosting significant branches for other
>> companies?

FYI, it isn't a Red Hat decision.

> I assume you're referring to the sources.redhat.com repository, right?
> 
> I think the answer is "yes", so long as the branch in question is
> intended to (eventually) further mainline gdb development.  If the
> branch is just some custom bit of work that is likely to never make
> it to the mainline, probably not.

And that decision is up to this group.

>> Would those other companies be happy depending on Red
>> Hat's CVS servers?

Should money making corporate types be willing to bet the crown jewls on 
those same resources (sware, subversion, sourceforge)?  Interesting 
question.

Red Hat makes sourceware available as a public service.  It funds it 
rather than owns it.  The machine is adminstered by a group of overseers 
(chrisf and jasonm are two).

Andrew

Re: [maint] The GDB maintenance process

[Christopher Faylor]

On Wed, Feb 19, 2003 at 11:29:13AM -0500, Andrew Cagney wrote:
>That gdb-patches database should be deleted.  It's dead.

Done.

cgf

Re: [maint] The GDB maintenance process

[Kevin Buettner]

On Feb 19,  9:33am, David Carlton wrote:

> On Wed, 19 Feb 2003 09:54:50 -0500, Andrew Cagney <ac131313@redhat.com> said:
> 
> > One thing GCC(4) and GDB are now is encouraging exprementation on
> > branches development to always occure on branches cut from the the
> > relevant repository.
> 
> Would Red Hat be happy hosting significant branches for other
> companies?

I assume you're referring to the sources.redhat.com repository, right?

I think the answer is "yes", so long as the branch in question is
intended to (eventually) further mainline gdb development.  If the
branch is just some custom bit of work that is likely to never make
it to the mainline, probably not.

> Would those other companies be happy depending on Red
> Hat's CVS servers?

I doubt it.  But...

For the contract work that Red Hat itself does, Red Hat does not depend
solely on the external (sources.redhat.com) respository.  Red Hat has
an internal repository that it uses for customer work.  From time to time,
the public repository is merged to Red Hat's internal repository.  Also,
from time to time, work that was performed for a customer which couldn't
immediately go into the public repository is pushed out to the public.

> (Maybe a reasonable answer here is "yes, but money
> should change hands".)  It might be nice if GDB were using a source
> code management tool that didn't depend on having a single repository,
> making it easier for people to maintain public branches elsewhere but
> to still sync them with an official branch.

Hmm.  Something to think about.

Kevin

Re: [maint] The GDB maintenance process

[David Carlton]

On Wed, 19 Feb 2003 09:54:50 -0500, Andrew Cagney <ac131313@redhat.com> said:

> One thing GCC(4) and GDB are now is encouraging exprementation on
> branches development to always occure on branches cut from the the
> relevant repository.

Would Red Hat be happy hosting significant branches for other
companies?  Would those other companies be happy depending on Red
Hat's CVS servers?  (Maybe a reasonable answer here is "yes, but money
should change hands".)  It might be nice if GDB were using a source
code management tool that didn't depend on having a single repository,
making it easier for people to maintain public branches elsewhere but
to still sync them with an official branch.

Anyways, I'd been planning for a while to write up some documentation
about creating branches; I'll try to do that sooner than later.  I was
somewhat resistant to the idea myself, but in retrospect it was
unquestionably the right way to go for my project.

David Carlton
carlton@math.stanford.edu

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> - It's true that "... some maintainers should try to review patches in
>   their areas of responsibility more often", but merely saying so
>   doesn't have any effect.

For the record your name is top of the list of that `some maintainers'.

Andrew

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Wed, Feb 19, 2003 at 11:19:07AM -0500, Andrew Cagney wrote:
> >ac131313@redhat.com (Andrew Cagney) writes:
> >
> >>> Some noticeable differences between these two models:
> >>>   - In the GCC model, more people are able/likely to check in patches 
> >>which
> >>>     break things.
> >>>   - But in the GCC model, more people are able/likely to check in 
> >>patches to
> >>>     fix it afterwards.
> >
> >>
> >>(ROFL.)
> >>
> >>The GCC model involves a number of development phases and the above
> >>comments would only relate to one of those phases.  At other times
> >>increasingly strict controls are placed on what can be
> >>committed/approved.  The GCC group spend a significant (out of
> >>control?)  amount of their time trying to re-stablize GCC for their
> >>releases.
> >>
> >>For GDB, on the other hand, interesting development can and does get
> >>approved/committed at any time.  GDB snaps are of such quality that we
> >>can confidently refer someone to current sources for fixes (except
> >>when I have a bad day like today :-).  Further, instead of using
> >>official releases (and as you yourself have done) arbitrary snaps can
> >>even make their way into a distro.
> >
> >
> >The problem is, being that stable has a cost associated with it.  GCC
> >pays that cost at certain parts in their cycle; we pay that cost all
> >the time, every day.
> 
> GDB is less stable then you might think.  Right now while both:
> 
> - interps
> - frame
> 
> are causing problems they are not getting in the way of DavidC's dwarf2 
> stuff (gee wiz, both my doing :-/).  GDB always builds, gdb always 
> `break main; run'.  Is that too much to ask?

Of course not.  If someone breaks that, they (or we) fix it quickly. 
GCC's no different.

> The problem with GDB's stability is that allows people to quickly forget:
> 
> - what it is like with out it
> - how much gain there is from it
> - how relatively small the pain
> - how much more expensive it is to have to re-do something later
> - how, with a bit of peer revew, problematic code could have been done 
> right the first time (and how much that fallout costs).

I don't see where any of this is coming from.  As you point out above,
in a lot of respects GDB isn't all that stable.  What are we risking
here?

It also seems that Jim and I don't agree that the gain outweighs the
pain.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Wed, Feb 19, 2003 at 10:47:46AM -0500, Andrew Cagney wrote:
> 
> >With my modest experience on the GDB project, and without any experience
> >on any other GNU project like this, it's difficult to make any
> >recommendation. My feeling is that we could try relaxing a bit the
> >rules, and allow global maintainers to approves changes if the
> >associated maintainer is unable to review them in say, a few (couple?)
> >of weeks.
> 
> Fernando does this already.  He also sends out e-mail saying he is off 
> line for a bit (he is down the hall from me which is an advantage). 
> Other maintainers do not, sigh!  More of a concern is that repeated 
> pings often result in radio silence.

Yes, Fernando does this.  He also tends to disappear for long periods
of time without doing it.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Which reminds me.  We've got two GNATS databases set up for GDB: 'gdb'
> and 'gdb-patches'.  Should we use the gdb-patches GNATS database to
> separate them from bug reports?

No!!!!  That gdb-patches database should be deleted.  It's dead.

People already have to track:

- gdb@
- gdb-patches@
- gnats@

and that is to complicated for some.  At least, by having both patches 
and bugs in a single database, we've a one stop-shop.  A better change 
is to dump gdb-patches@

> TBH, I've been avoiding it because I don't know the slightest thing
> about the h8300 and it seemed like someone (I don't remember who -
> MichaelS maybe?) did.  If I'm wrong then one of us is just going to
> have to suck it up and learn about the h8300... yay, another project.

No worse than having me review them.  All that can be done is review 
them at a coding / broad architectural level and assume that the 
contributor, given that they are using the target, have it working. 
Knowing that they run the testsuite also helps.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> ac131313@redhat.com (Andrew Cagney) writes:
> 
>> > Some noticeable differences between these two models:
>> >   - In the GCC model, more people are able/likely to check in patches which
>> >     break things.
>> >   - But in the GCC model, more people are able/likely to check in patches to
>> >     fix it afterwards.
> 
>> 
>> (ROFL.)
>> 
>> The GCC model involves a number of development phases and the above
>> comments would only relate to one of those phases.  At other times
>> increasingly strict controls are placed on what can be
>> committed/approved.  The GCC group spend a significant (out of
>> control?)  amount of their time trying to re-stablize GCC for their
>> releases.
>> 
>> For GDB, on the other hand, interesting development can and does get
>> approved/committed at any time.  GDB snaps are of such quality that we
>> can confidently refer someone to current sources for fixes (except
>> when I have a bad day like today :-).  Further, instead of using
>> official releases (and as you yourself have done) arbitrary snaps can
>> even make their way into a distro.
> 
> 
> The problem is, being that stable has a cost associated with it.  GCC
> pays that cost at certain parts in their cycle; we pay that cost all
> the time, every day.

GDB is less stable then you might think.  Right now while both:

- interps
- frame

are causing problems they are not getting in the way of DavidC's dwarf2 
stuff (gee wiz, both my doing :-/).  GDB always builds, gdb always 
`break main; run'.  Is that too much to ask?

The problem with GDB's stability is that allows people to quickly forget:

- what it is like with out it
- how much gain there is from it
- how relatively small the pain
- how much more expensive it is to have to re-do something later
- how, with a bit of peer revew, problematic code could have been done 
right the first time (and how much that fallout costs).

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> A few days ago, I actually ran statistics on how long it takes for a patch to get first review in gcc vs gdb over the past year, and for gcc, it's 2.some odd days.  Believe it or not, for gdb, it was well over two weeks.
> That's not good.

Ah, statistics.  Ah metrics.

Given most outliers are now in the bug database can you perhaphs break 
it down by area and, hopefully, more usefuly, look at each area's change 
rate.

It is the change/progress that is important, not the absolute numbers. 
A lack of change that is of concern.  For instance, I worry about build 
and remote problems.  The former definitly makes progress, the latter 
less so :-(

Andrew

PS: And metrics are made to be abused.  As soon as people know the 
metric that they are being measured by, the quickly addapt their 
behavour to make that specific metric look better.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> With my modest experience on the GDB project, and without any experience
> on any other GNU project like this, it's difficult to make any
> recommendation. My feeling is that we could try relaxing a bit the
> rules, and allow global maintainers to approves changes if the
> associated maintainer is unable to review them in say, a few (couple?)
> of weeks.

Fernando does this already.  He also sends out e-mail saying he is off 
line for a bit (he is down the hall from me which is an advantage). 
Other maintainers do not, sigh!  More of a concern is that repeated 
pings often result in radio silence.

I don't think having a global maintainer step in after N weeks will 
really help.  It just butters over what may be a more serious underlying 
problem.

Andrew

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Wed, Feb 19, 2003 at 10:17:12AM -0500, Andrew Cagney wrote:
> 
> >> > Right now, we use stricter policies to prevent problems which cause
> >> > breakage.  I think these policies are stifling us.  Loosening them (and
> >> > maybe adding a formal patch reversion policy) would let more people fix
> >> > problems more easily, as they arise, without slowing development.
> >> > 
> >>
> >>I really think that having the bug database track patches is a very
> >>big improvement. We should try to push its use a bit more before
> >>giving up on the current rules. 
> >
> >
> >I don't like it, but that's just my general anti-GNATS rancor.  Maybe
> >Bugzilla will be better.
> 
> The idea of tracking things as bugs or GNATS?

Mostly, GNATS.

> Tracking this stuff in a public database is, I think, clearly better 
> than the status quo (my mailbox).  It's already been mentioned that 
> there is possibly a lack of response with some maintainers.  At least 
> with me putting unreviewed patches in the database, we can figure out 
> what patches there are, and if there is a problem.  (my todo list 
> includes checking that all the key developers are on the bug tracking 
> list, I suspect that some are not.)
> 
> Anyway, have a look in the database, you'll notice all sorts of 
> interesting things.

Which reminds me.  We've got two GNATS databases set up for GDB: 'gdb'
and 'gdb-patches'.  Should we use the gdb-patches GNATS database to
separate them from bug reports?

> For instance search for tdep & h8300.  There is an obvious backlog and 
> someone (a global maintainer like yourself or I) needs to step up and 
> work through them with the contributors.  That task, while no where near 
> as glamerous as a new feature, is absolutly needed and maintainers do 
> need to be willing to chip in (in fact I think that task and 
> documentation should be given a higher profile when handing out 
> `credit').  Hopefuly these patches will even yield a new developer.

TBH, I've been avoiding it because I don't know the slightest thing
about the h8300 and it seemed like someone (I don't remember who -
MichaelS maybe?) did.  If I'm wrong then one of us is just going to
have to suck it up and learn about the h8300... yay, another project.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Andrew Cagney]

>>  > Right now, we use stricter policies to prevent problems which cause
>>  > breakage.  I think these policies are stifling us.  Loosening them (and
>>  > maybe adding a formal patch reversion policy) would let more people fix
>>  > problems more easily, as they arise, without slowing development.
>>  > 
>> 
>> I really think that having the bug database track patches is a very
>> big improvement. We should try to push its use a bit more before
>> giving up on the current rules. 
> 
> 
> I don't like it, but that's just my general anti-GNATS rancor.  Maybe
> Bugzilla will be better.

The idea of tracking things as bugs or GNATS?

Tracking this stuff in a public database is, I think, clearly better 
than the status quo (my mailbox).  It's already been mentioned that 
there is possibly a lack of response with some maintainers.  At least 
with me putting unreviewed patches in the database, we can figure out 
what patches there are, and if there is a problem.  (my todo list 
includes checking that all the key developers are on the bug tracking 
list, I suspect that some are not.)

Anyway, have a look in the database, you'll notice all sorts of 
interesting things.

For instance search for tdep & h8300.  There is an obvious backlog and 
someone (a global maintainer like yourself or I) needs to step up and 
work through them with the contributors.  That task, while no where near 
as glamerous as a new feature, is absolutly needed and maintainers do 
need to be willing to chip in (in fact I think that task and 
documentation should be given a higher profile when handing out 
`credit').  Hopefuly these patches will even yield a new developer.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Sure.  But my portrayal isn't entirely inaccurate, either.  One example
> - inter-compilation-unit references in DWARF-2.  It's been posted at
> least twice, by different contributors.  When asked for changes,
> they've come back with changes.  It's still not supported, over a year
> later.

Yes, I think it is fair to say that we've recently dropped baton on that 
one :-(  JimB, what's the current status?

Anyway a mid-mortem is in order.

GDB has a serious problem with _old_ forks.  cf:

- hp fork
- apple fork
- act's fork
- caldera's fork

You'll note that, in each of these cases, they really are forks(1).  The 
relevent developers created their own repository and then went off into 
the wilderness only to mysteriously appear later with a jumbo patch.  I 
believe that decision for forking was largly driven by the desire to, 
due to short term commercial pressure, avoid doing the right thing.

Not unreasonably, the GDB developers asked that such jumbo patches be 
broken down into something more managable, more up-to-date, and (too 
often) `lint' free(2).

Also, not unreasonably, the GDB developers asked/questioned 
architectural decisions made on those branches.

Less realistic, perhaps, is an over expectation of how much a 
contributor can reasonably be asked.  Even ignoring those `comercial 
pressures', many contributors figure `take it or leave it'.  Maintainers 
need to be willing to not only spend time adding glamerous new features, 
but also when an important patch appears, pick it up and run with it(3).

One thing GCC(4) and GDB are now is encouraging exprementation on 
branches development to always occure on branches cut from the the 
relevant repository.  For GDB we've had both success stories but also 
disasters.  With that in mind, and looking at the GCC / GDB success 
stories, I'd suggest the following guidelines:

- branches shall post all commits
They don't need approval but can be commented on.

- branches shall to be focused
The interps branch started out too large with too many changes - look at 
the size of the final commit compared to that branch at its peak.  Much 
time was lost because the branch started with too much lint :-(

- branches shall track mainline.
This keeps the level of divergence under control.  It also keeps the 
pressure on developers to push cleanups and other stuff into the mainline.

Andrew

--

(1) I was discussing this with a GCC developer and they pointed out, for 
the reasons I note, that these are not branches.

(2) As in the stuff you find on your clothing (not the program).  Forks 
and branches, like navel's, are shockers for breeding lint - entropy 
that serves no useful purpose.

(4) Yes, as I mentioned, I discuss this with GCC developers to see what 
lessons can be learnt.

(3) Note that if I pick up a really old patch, I'll often review, tweak 
and commit it.  I figure that if a patch is more than a month old, it 
isn't realistic to be asking a contributor to make minor changes.

Re: [maint] The GDB maintenance process

[Daniel Berlin]

On Tuesday, February 18, 2003, at 08:49  PM, Daniel Jacobowitz wrote:

> Thanks to everyone who responded; I'll try to address everything.
>
> I want to share a piece of perspective on why I raise this issue now.
> I'm finding the GDB development process to be too slow to be workable -
> patches take a month or more routinely, and I don't have the time to
> juggle them for that long.  I'm facing that I may need to back down my
> level of involvement as a consequence of how awkward the development
> model is for getting any forward progress.
>

A few days ago, I actually ran statistics on how long it takes for a 
patch to get first review in gcc vs gdb over the past year, and for 
gcc, it's 2.some odd days.  Believe it or not, for gdb, it was well 
over two weeks.
That's not good.
Outliers are the norm in gdb, there was no middle ground.
Either something takes 1 day to get a review, or it takes months.
The only reason the average is actually only a little over two weeks is 
because of a string of 0 day turnarounds that occasionally happen.
> Maybe that means that I just don't have the time and the patience to be
> a useful contributor to GDB.  Me, I think that it means that we need to
> make the process less painful for contributors.
FWIW, I agree, but i hold no hope it will happen.
I'm not a cynic, i'm just experienced.
I'd be a cynic if it wasn't based on experience, of course.
>
>> I have seen sometimes very quick turnarounds on patches during
>> holidays, and maybe some of such patches should have been thought
>> through more carefully. If you don't give time to the appropriate
>> maintainers to chime in, the entropy can become way too high, with
>> patches, and reverted patches going around.
>
> I guess I just don't see this to be as much of a problem as others do.
> For one thing, with the higher entropy level, more development actually
> happens.
Bingo.
I don't think we should stall development (and in the extreme,  even if 
it means we can't make quality releases any day of the year) because 
mistakes occasionally happen in patches, or because not every 
maintainer in existence has said something about a patch.  That's a 
recipe for no progress.  The old "adding more developers to a late 
project makes it later because of communication overhead" problem.

>
>>> Another thing to think about: because of the layout of the above, 
>>> there is
>>> frequently no one who has the _right_ to approve a patch.  They 
>>> require
>>> buy-in from a number of additional maintainers.  In addition our 
>>> volunteers
>>> are often too busy to find time to respond to patches.  This impacts 
>>> patches
>>> from other maintainers (frequently, but generally a small impact) 
>>> and from
>>> outside contributors (happens less frequently, but larger impact - 
>>> most of
>>> these never get approved at all, from what I've seen).
>>>
>>
>> Wait, lots of external contributor's patches never make it in because
>> of the copyright assignment problems.

Bull.
This counts for *maybe* 10% of patches that don't make it in, if that.
>> Also I see external people
>> dropping patches, not because the are not reviewed, but because they
>> *are reviewed*. I.e. a patch is submitted, I ask for some changes, and
>> the person never comes back with a new patch.

Or maybe it's because frequently the review took >1 month, and they 
just don't want to spend that much more time waiting again after making 
changes, or they moved on to other projects.

Have you noticed GDB *very rarely* keeps minor contributors coming back?
OTOH, most opensource projects (including gcc) i am a part of 
frequently have the same minor contributors, again and again.

Maybe we need community exit interviews to back this point up.

>
> Are we talking about the same thing?  I don't think I understand you.
>
> First of all, the idea of a small, fully funded elite taking control of
> the project doesn't make any sense to me.  Either their changes are
> worthwhile - the existing maintainers will cooperate with them - or
> they overstep their boundaries - they are asked to cease.  We don't
> hand out maintainership to all comers.

Well, they gave it to me (and Stan Shebs :P) at one point, so ...
:)


>
> That's not accurate.  First of all, the comments relate to all three of
> the phases in the GCC development process, although the effect is
> mostly damped down in the third phase by the release manager's hand.
> Even in stage 3, other maintainers are free to exercise their
> judgement.
>
> Also, the GCC tree spends most of its time in a more useful state than
> the above really portrays.
Yup.
Andrew's just plain wrong on this one.
All patches are bootstrapped and reg-tested on at least one platform, 
and those that are larger or could affect multiple platforms are 
bootstrapped and regtested on multiple platforms.

Any needed stabilization is generally because of latent bugs in 
*other*, less tested platforms, that improvements expose.
*Not* problems in the patches themselves.


>
>> For GDB, on the other hand, interesting development can and does get
>> approved/committed at any time.  GDB snaps are of such quality that we
>> can confidently refer someone to current sources for fixes (except
>> when I have a bad day like today :-).  Further, instead of using
>> official releases (and as you yourself have done) arbitrary snaps can
>> even make their way into a distro.
>
> [Red Hat does this too, by the way.]  Having done it, I've resolved to
> avoid it in the future.  Costs outweighed benefits.
>
I agree that it should be avoided, too.
It's just painful and not necessary here.
Our users don't clamor for a release on a certain date, nor have we 
ever had a need to make one.
The only advantage this adds is to companies using gdb sources and 
wanting to do merges to an internal tree.

Re: [maint] The GDB maintenance process

[David Carlton]

Here is some free-associating that I have about GDB's approval
process.  I'm not sure yet what I think about various concrete
proposals out there; I would like to see more existing models
discussed, because it seems unlikely to me that GDB's current model is
optimal (though, of course, what is?), and because recent history
seems to be full of interesting and potentially effective software
development models, models that I wish I knew more about.

* Overall, I'm quite happy with my experience working with GDB.  On
  the other hand, I suspect that various idiosyncracies in my
  situation make me a bit of an outlier in terms of ability and
  willingness to stick through the process.  It does seem to me that
  there have been other people's patches that would be valuable but
  that have died through lack of timely follow-through (whether by the
  approvers, by the submitters, or both, not to mention issues like
  copyright approval that we're unlikely to change).  The more people
  who are willing and allowed to help newcomers through this process,
  the better GDB will be.

* Sometimes the division of responsibility seems to me a bit strange.
  If I were a C++ maintainer, for example, it might drive me a bit
  bonkers having to wait for other people's approval for patches that
  only affect C++ but that are located in code that is in somebody
  else's domain.  (I was impressed with the way Daniel got the
  'demangling partial symbols' patch through.)

* The most serious strangeness of division of responsibility is having
  separate maintainers for the testsuite whose approval is required.
  We should do whatever we can to encourage people to write tests,
  whether to test new features or to fill in gaps in existing
  features; the more people who can approve tests, the better.  (I
  might be imagining things, but it seems to me like people have been
  frequently adding tests without waiting for approval recently; this
  seems like a good thing to me.)  It seems to me that, at the very
  least, anybody who can approve patches in a specific area should
  also be able to commit patches for tests in that area.  (Though of
  course it's good to, in addition, allow people to approve testsuite
  patches even if they can't approve non-testsuite patches: like I
  said, the more tests, the better.)

* Speaking of tests, the better the testsuite is, the less likely that
  patches will break things.  (Some currently popular development
  methodologies are rooted on this idea.)  On the other hand, there
  are good reasons (the history of GDB, its architecture, the test
  suite's architecture, and the nature of debuggers) that the
  testsuite's coverage will never be perfect, setting aside the issue
  that "not breaking behavior" isn't the same as "good".

* I have been impressed with GDB's stability.  Of course, it doesn't
  hurt that I'm using GDB on i686-pc-gnu-linux.  (Or that, frankly,
  I'm not a very heavy GDB user.)

* By temperament and programming style, I prefer to break up my
  patches into lots of small changes whose correctness should be
  relatively easy to verify.  Unfortunately, this means that it takes
  a long time for the sum of my changes to be approved.  For example,
  it's been more than 3 months since I posted a version of linespec.c
  that I think most people would agree was a significant improvement
  to its predecessor; I've submitted 14 patches to that file so far,
  with approximately an equal number to come.  I actually rather enjoy
  that process when it's going well - I log in each day, read my
  email, get an approval message for my latest patch, take a bit of
  time to generate and test my next patch, and then move on to
  something else - but it gets annoying when I have to wait for
  approval for simple, obviously correct patches.

  I've had a similar experience with cleaning up lookup_symbol_aux:
  there are bugs in that code, and I have patches to fix them, but I'm
  not sure when they'll make it into GDB.  I sigh with envy as I watch
  Andrew committing simple frame patch after simple frame patch
  without needing anybody else's approval.  A series of small patches
  like that can be a wonderful way to cleanse code.

* Continuing my last bullet point, however, it's not obvious to me
  that the changes in the approval process that Daniel suggests would
  help.  After all, there are already 3 people who can approve patches
  to linespec.c (so I'm certainly not mad at Elena - she's the only
  one taking time to actually approve the patches in question!), and I
  don't see why increasing that number would decrease the approval
  time.  So the obvious place to look for improvement is for me to
  change my behavior; I have some thoughts along those lines that I'm
  going to try to implement over the next few weeks.

* Having patches in the bug database is probably a good idea, but it
  means that somebody has to take the effort to put patches in the bug
  database.  It was great when Andrew cleaned out his mailbox and
  dumped it into the bug database; I'm not optimistic about the bug
  database working well for this in the longer term.

* Increasing the number of people who are familiar with a given
  segment of GDB's code can only improve GDB's health.  The current
  approval process seems to me to work against this, to some extent.

David Carlton
carlton@math.stanford.edu

Re: [maint] The GDB maintenance process

[Jim Blandy]

ac131313@redhat.com (Andrew Cagney) writes:
> > Some noticeable differences between these two models:
> >   - In the GCC model, more people are able/likely to check in patches which
> >     break things.
> >   - But in the GCC model, more people are able/likely to check in patches to
> >     fix it afterwards.
> 
> (ROFL.)
> 
> The GCC model involves a number of development phases and the above
> comments would only relate to one of those phases.  At other times
> increasingly strict controls are placed on what can be
> committed/approved.  The GCC group spend a significant (out of
> control?)  amount of their time trying to re-stablize GCC for their
> releases.
> 
> For GDB, on the other hand, interesting development can and does get
> approved/committed at any time.  GDB snaps are of such quality that we
> can confidently refer someone to current sources for fixes (except
> when I have a bad day like today :-).  Further, instead of using
> official releases (and as you yourself have done) arbitrary snaps can
> even make their way into a distro.

The problem is, being that stable has a cost associated with it.  GCC
pays that cost at certain parts in their cycle; we pay that cost all
the time, every day.

Sure, it's nice being able to grab an arbitrary GDB snapshot and
distribute it.  But the rules which give us that property also slow
GDB's development into a more capable system.  The decision to pay
that particular price for that particular benefit is what we want to
open up for discussion.

Stability needs to be balanced with progress in features; although I'd
hate to see it go too far towards the latter, I think the balance we
have now is tipped too far in favor of the former.

Re: [maint] The GDB maintenance process

[Joel Brobecker]

Daniel said:
> I want to share a piece of perspective on why I raise this issue now. 
> I'm finding the GDB development process to be too slow to be workable -
> patches take a month or more routinely,

> Maybe that means that I just don't have the time and the patience to be
> a useful contributor to GDB.  Me, I think that it means that we need to
> make the process less painful for contributors.

So far, I have only really started making some contributions maybe a
year and a half ago, and my level of involvment is very far from a lot
of the developpers I see on this list. But I tend to agree with Daniel
that patches _sometimes_ take a very long time to be included, and that
it is easy to be discouraged. 

I think there are also a lot of patches floating around that are waiting
for at least a review. One of the ACT developpers actually stopped
submitting his patches, just because he did not receive any feedback.
And yet he spent the effort of creating a PR and attaching the patch to
this PR! So I am submitting the changes on his behalf now, and took on
the job of trying to push for these patches to be reviewed.

I agree that we are all very busy, and that it's natural that reviews
do not always happen in a timely manor. In fact, I am generally happy
with the delay-to-review, but my feeling is that the GDB community is
losing a lot of valuable work because it never gets looked at.

With my modest experience on the GDB project, and without any experience
on any other GNU project like this, it's difficult to make any
recommendation. My feeling is that we could try relaxing a bit the
rules, and allow global maintainers to approves changes if the
associated maintainer is unable to review them in say, a few (couple?)
of weeks.

Look at http://sources.redhat.com/ml/gdb-patches/2002-10/msg00586.html
for intance. This patch addresses a GDB crash, and the fix has been
sitting since Oct 30th, despite 2 pings, 1 recommendation for approval,
and a message to the maintainers. I do not want to blame the maintainers
here, but in my opinion the global maintainers should be able to act as
surrogates when they see that a patch has been sitting for more than a
certain duration, and they have the knowledge to review it.

-- 
Joel

Re: [maint] The GDB maintenance process

[Jim Blandy]

I agree with what Daniel has said here.  I'm concerned that some
people have misunderstood his points, so I'll put it differently.

I think GDB could get better use of the contributors it has now by
adjusting the rules for patch approval.

- Slow patch review is a real problem on GDB --- even acknowledging
  the legitimate reasons for some delays that Elena has mentioned.

- It's true that "... some maintainers should try to review patches in
  their areas of responsibility more often", but merely saying so
  doesn't have any effect.  Folks have been saying that ever since
  Cygnus loosened its grip on GDB and the process opened to the
  public.  That statement seems to express a hope that the maintainers
  will somehow "wake up" and everything will get better.  It's been
  years, now, and we need to stop waiting for this to happen.  Let's
  work with the people we've got, rather than hoping they'll transform
  themselves somehow.

- If we accept that the maintainers' behavior is stable, then the next
  alternative is to adjust the organization that they operate under.
  Is there some way to re-organize the people we have, accepting their
  job committments and personal limitations (I have myself in mind
  here as much as anyone else, so don't be affronted), so that things
  progress better?  Is the current organization optimal?

Set that line of thinking aside, and consider another problem:

As far as I can tell, the Global Maintainers don't really have the
privileges that one would expect.  A Global Maintainer ought to be
someone who can make or approve a change anywhere in GDB.  This means,
in addition to having good technical judgement, that we trust them to
ask others as appropriate, and that we consider them good people to
debate with when a disagreement does come up.

This is different from the current definition --- as far as I can
tell, Global Maintainers aren't really that much different from anyone
else.  I've seen them criticized for approving changes, not because of
any specific problem with the changes, but because it was seen as
outside their purview.

I would personally be happy to see the current list of Global
Maintainers, of which I am a member, emptied out, and have people
re-establish their reputations to be added to the list, if it would
make folks more comfortable about straightening out the definition
this way.

So, to bring the two threads together: I think GDB could get better
use of the contributors it has now by relaxing the rules about who is
allowed to approve a patch.  With the redefinition of Global
Maintainer we're suggesting, more people would have the right to
review and approve a patch, so there are more chances someone will.

When I was maintaining Guile, I ended up replacing my dictatorship
with a group of four maintainers --- Mikael Djurfeldt, Maciej
Stachowiak, and Marius Vollmer.  Any one of those guys I trusted as
much as I trust myself.  There was no risk in making them my peers,
since I was just as likely to make a poor decision as they were (if
not more so), and if they didn't like something, it would certainly
benefit from a re-examination.  Surely we have people in the GDB
community in addition to Andrew who have earned that level of trust.


Just to add another idea to the mix:

I was talking to a friend of mine about the way GDB is run, and he was
amazed that we give individual people complete power, and complete
responsibility, for sections of code.  Everyone is going to be wrong
sometimes, he said, and it's easy to protect against, without being
too bureaucratic.

In the Apache system, which is used now in a lot of projects, they
have a pretty big group of global maintainers.  Any non-obvious core
change needs to be described on the list, and get three "+1" votes
from other global maintainers.  But if anyone posts a "-1", the change
is vetoed: it gets discussed for a longer period, and then put up for
a straight vote.

The nice thing here is that *nobody* has absolute power --- come on,
even the most talented among us is going to do something questionable
every so often, and would benefit from some more discussion --- but
it's still easy enough to get buy-in for most changes that it's
efficient.

But whether or not you like that idea, GDB has been operating under
basically the same system, and suffering the same problems, long
enough to confidently conclude that we, at least, are always going to
operate the way we do now under the system we have now.  This
delegated dictatiorship (with all due respect to our hard-working and
conscientious dictator) is more of an inherited legacy from ancient
days than a well-tuned process.  I think it's time to try to bring in
good ideas from other projects, like GCC and Apache.

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

Thanks to everyone who responded; I'll try to address everything.

I want to share a piece of perspective on why I raise this issue now. 
I'm finding the GDB development process to be too slow to be workable -
patches take a month or more routinely, and I don't have the time to
juggle them for that long.  I'm facing that I may need to back down my
level of involvement as a consequence of how awkward the development
model is for getting any forward progress.

Maybe that means that I just don't have the time and the patience to be
a useful contributor to GDB.  Me, I think that it means that we need to
make the process less painful for contributors.

On Mon, Feb 17, 2003 at 09:35:53PM -0500, Andrew Cagney wrote:
> > I've discussed this general situation with a (small) sampling of other GDB
> > developers and contributors - enough to know that I'm not alone in my
> > concerns.  These aren't entirely my own words, either.  I'll let other
> > people take credit/blame for them if they want to, and if I've represented
> > their opinions accurately.
> 
> I recall you rasing this previously with me but I don't recall
> agreeing with your opinions (my concerns run along similar lines to
> what Elena posted).

Hmm, I don't remember actually discussing this with you before.  Must
be going soft.  We've certainly butted heads over related topics.

> Normally political issues get discussed with the global maintainers
> (so that the mailing lists can focus on technical discussion).
> However, if you feel that it is better discussed here, then so be it.

If you feel that's more appropriate, just say so.  I've no particular
preference for where the conversation happens, as long as it happens.



On Mon, Feb 17, 2003 at 04:05:40PM -0500, Elena Zannoni wrote:
> Daniel Jacobowitz writes:
>  > I get the feeling I'm already pretty unpopular for some of my opinions on
>  > how GDB maintenane should work.  This isn't going to make it any better, but
>  > I feel it should be said.
>  > 
> 
> OK, I'll take the bait. No offense intended. Just read until the end.
> 
>  > I believe that our current process has some problems.  Let me try to
>  > explain.  First, to make sure we're all on the same page...
>  > 
>  >  What does it mean to be a Global Maintainer, in practice?
>  >   - A certain amount of autonomy in the areas of GDB that no one wants to
>  >     take specific responsibility for.  There's no specific maintainer for
>  >     things like the frame support or the type/value systems.
>  >   - A little more freedom in approving patches to other people's areas of
>  >     GDB - not a lot, but it's definitely there.
>  >     [In practice, this depends on:
>  >      o How much risk you're willing to take of annoying people.
>  >      o How likely other maintainers are to shout at you about it.]
> 
> The above point is not part of the global maintainer description. I
> take you want it to be? I don't agree. It just makes it possible for
> particularly 'active' individuals to dominate the list and removes
> the more egalitarian feel that this community has. You can say that
> the linux kernel community is like that, and that gcc is like that,
> etc, but we don't have to be. 

I don't especially want it to be; I want something different, farther
down.  This is my vision of how we work right now.  The above is
informal but in my experience a pretty accurate summary of the way
things really are.

> I have seen sometimes very quick turnarounds on patches during
> holidays, and maybe some of such patches should have been thought
> through more carefully. If you don't give time to the appropriate
> maintainers to chime in, the entropy can become way too high, with
> patches, and reverted patches going around.

I guess I just don't see this to be as much of a problem as others do.
For one thing, with the higher entropy level, more development actually
happens.

>  > Another thing to think about: because of the layout of the above, there is
>  > frequently no one who has the _right_ to approve a patch.  They require
>  > buy-in from a number of additional maintainers.  In addition our volunteers
>  > are often too busy to find time to respond to patches.  This impacts patches
>  > from other maintainers (frequently, but generally a small impact) and from
>  > outside contributors (happens less frequently, but larger impact - most of
>  > these never get approved at all, from what I've seen).
>  > 
> 
> Wait, lots of external contributor's patches never make it in because
> of the copyright assignment problems. Also I see external people
> dropping patches, not because the are not reviewed, but because they
> *are reviewed*. I.e. a patch is submitted, I ask for some changes, and
> the person never comes back with a new patch.
> 
> There would be also something to say about championing your patches,
> ie, push a bit. For instance this is what David is doing with the
> namespaces stuff, he is keeping me posted, and he makes sure I don't
> fall asleep at the wheel. 
> 
> Just saying, things are not as black and white as they are portraited.

Sure.  But my portrayal isn't entirely inaccurate, either.  One example
- inter-compilation-unit references in DWARF-2.  It's been posted at
least twice, by different contributors.  When asked for changes,
they've come back with changes.  It's still not supported, over a year
later.

On the subject of David, infinite kudos to him for his
single-mindedness and patience.  When I look at the amount of forward
progress he's made compared to the time he's invested in making it, I
become even more convinced that we have a problem.

>  > Some other GNU projects have a similar setup and don't have this problem. 
>  > GCC and binutils are my usual examples.  How do they avoid it?  They have a
>  > different definition of global maintainer.  That's what ours used to be
>  > called - Blanket Write Privileges.  The system works a little differently:
>  >   - Maintainers for specific areas of the compiler can commit/approve
>  >     patches to the areas they maintain without buy-in from a blanket
>  >     maintainer.
> 
> I think you are misunderstanding. Gdb is like that. If one maintains
> one area, that's it, period. His/her approval is enough. Even if the
> maintainer is not global, the authority is the same *in that specific
> area*.

I know that.  That's no different between GCC and GDB; I'm not
discussing a radical 100% turnaround here.

>  >   - Blanket maintainers can commit/approve patches anywhere without buy-in
>  >     from a specific area maintainer.
>  > 
>  > [I hope Richard will forgive me for using him as an example and for putting
>  > words in his mouth...] This doesn't replace common sense - you generally
>  > won't find Richard Henderson approving patches to the C++ frontend, because:
>  >   - He knows he isn't familiar with it
>  >   - He knows it has an active set of maintainers at all times
>  > 
>  > Similarly, just because he can check in patches to any target backend, that
>  > doesn't mean he won't ask a target maintainer to look over it first.  If
>  > someone objects to a patch in their area, he would generally not just check
>  > it in anyway.  If they object to it after he checks it in, the two will
>  > discuss the problem like reasonable people and come to some agreement.
>  > 
>  > 
>  > Some noticeable differences between these two models:
>  >   - In the GCC model, more people are able/likely to check in patches which
>  >     break things.
>  >   - But in the GCC model, more people are able/likely to check in patches to
>  >     fix it afterwards.
>  >   - Because more people have the privilege of approving a given patch,
>  >     and fewer people's approvals are needed for any particular patch,
>  >     patches (usually) get approved more quickly.
>  >   - Development can happen more quickly, and does not get slowed to a
>  >     standstill when (say) one of us is pulled off of community GDB work for
>  >     an urgent customer project.  This happens all the time - I've never seen
>  >     all the GDB maintainers with time for GDB at the same time.
>  > 
> 
> Yes, we have day jobs, that are not 'work on FSF gdb'. If you loosen
> the rules, then it becomes possible for a small, fully funded, elite to
> take control.  
> 
> And how long would you wait before stepping in? That's a hard problem.
> 1 hour, one day, one working day, one week? 

Are we talking about the same thing?  I don't think I understand you.

First of all, the idea of a small, fully funded elite taking control of
the project doesn't make any sense to me.  Either their changes are
worthwhile - the existing maintainers will cooperate with them - or
they overstep their boundaries - they are asked to cease.  We don't
hand out maintainership to all comers.

It's not about control of GDB, it's about the growth and quality of
GDB!  I don't care who controls it.  I'll mostly resist making a
comment here about who currently controls GDB - more than half the
global maintainers work at the same place - because it's an entirely
benign and open control.

Second, what do you mean by "step in"?  Step in in response to what?  A
broken patch?  A violation of authority?  The latter and partly the
former are what GCC has a formal steering committee for.  We could do
the same thing, or use a vote of the global maintainers.  In practice
it's rarely necessary.

>  > Right now, we use stricter policies to prevent problems which cause
>  > breakage.  I think these policies are stifling us.  Loosening them (and
>  > maybe adding a formal patch reversion policy) would let more people fix
>  > problems more easily, as they arise, without slowing development.
>  > 
> 
> I really think that having the bug database track patches is a very
> big improvement. We should try to push its use a bit more before
> giving up on the current rules. 

I don't like it, but that's just my general anti-GNATS rancor.  Maybe
Bugzilla will be better.

>  > If there are people on our Global Maintainer list that we don't think should
>  > be trusted with the extra responsibility of the above, then perhaps we need
>  > to rethink who belongs in that list.  I'm not pointing any fingers - I don't
>  > have anyone in mind, and I've been quite happy working with absolutely all
>  > of the current team.  Just putting the idea out.
>  > 
>  > 
>  > I've discussed this general situation with a (small) sampling of other GDB
>  > developers and contributors - enough to know that I'm not alone in my
>  > concerns.  These aren't entirely my own words, either.  I'll let other
>  > people take credit/blame for them if they want to, and if I've represented
>  > their opinions accurately.
> 
> Hmm, I am not too happy with this behind the scenes discontent, if
> there is something wrong, let's hear it.

That's what I'm doing!  I assure you, I don't enjoy generating this
much rancor.

On Mon, Feb 17, 2003 at 11:28:46PM -0500, Andrew Cagney wrote:
> 
> >  What does it mean to be a Global Maintainer, in practice?
> >   - A certain amount of autonomy in the areas of GDB that no one wants to
> >     take specific responsibility for.  There's no specific maintainer for
> >     things like the frame support or the type/value systems.
> >   - A little more freedom in approving patches to other people's areas of
> >     GDB - not a lot, but it's definitely there.
> >     [In practice, this depends on:
> >      o How much risk you're willing to take of annoying people.
> >      o How likely other maintainers are to shout at you about it.]
> >   - Authority to approve patches covering general debugger issues.
> > 
> >  What does it mean to be a maintainer for a specific host/target/subsystem,
> >  in practice?
> >   - The authority to approve patches and apply your own patches to that area
> >     of the debugger.
> > 
> > I'd like everyone to notice one thing missing from the above list.  No one
> > has the _responsibility_ for approving patches.  This is a volunteer
> > project, and anyone who's watched it in action for a little while will see
> > that the volunteers are often busy and distracted.  There's no one who
> > can handle or should have to handle the responsibilities of patch approval.
> 
> This isn't correct.  The MAINTAINERS file reads:
> 
>                         Various Maintainers
> 
>   Note individuals who maintain parts of the debugger need approval to
>   check in changes outside of the immediate domain that they maintain.
> 
>   If there is no maintainer for a given domain then the responsibility
>   falls to a global maintainer.
> 
>   If there are several maintainers for a given domain then
>   responsibility falls to the first maintainer.  The first maintainer is
>   free to devolve that responsibility among the other maintainers.
> 
> so it makes definite references to `responsibility'.  Put simply, `the
> buck stops'.  Perhaps you should ask a different question `is the buck
> ignored?' and what should be done if/when it is.
> 
> Even though GDB is developed by volunteers, it is entirely reasonable
> to expect the volunteers to take their duties seriously.  Monitoring
> the bug data base; approving patches; where helpful, contributing to
> discussions.  Now that I typically file patches into the bug database,
> this should be much easier.
> 
> However, what is unreasonable is to place un-realistic expectations on
> the said maintainers.  The assumption that a volunteer is working 24x7
> and living in north america isn't, for instance, reasonable.  A
> significant number of GDB maintainers have real jobs and real
> timezones.

[North America doesn't have real jobs and real timezones?  They're just
different...]  OK, I withdraw my comment about no one having
responsibility.  Let me try from another side.

The volunteers do take their duty seriously, but with the density of
responsibility involved [i.e., when a particular person or people are
responsible for a decision, there tend to be just one or two of them]
it's very hard to make forward progress when people are busy.  And
we're all busy people.  Significant patches almost always take three
weeks or more if they can't be approved by their author; some take
months, even with repeated prodding.

We don't just have a bottleneck, we have a bottleneck per maintainer.

> > Some other GNU projects have a similar setup and don't have this problem. 
> > GCC and binutils are my usual examples.  How do they avoid it?  They have a
> > different definition of global maintainer.  That's what ours used to be
> > called - Blanket Write Privileges.  The system works a little differently:
> >   - Maintainers for specific areas of the compiler can commit/approve
> >     patches to the areas they maintain without buy-in from a blanket
> >     maintainer.
> >   - Blanket maintainers can commit/approve patches anywhere without buy-in
> >     from a specific area maintainer.
> 
> `write' was a simple typo.  From the outset (i.e., from when I got
> handed this job) it was made very clear that neither I, nor any other
> global maintainer, could override the decision of a specific
> maintainer.
> 
> This was, and still is, very important.  It ensures that the area has
> a certain level of continuity, and it ensures that I (or another
> global maintainer) doesn't ride gunshot over a specific maintainer.

Which raises two thoughts in my mind.
 - What are the benefits of this continuity, and do they really
outweigh the disadvantages (slow progress, primarily)?  My position is
that they do _not_.
 - Why do we need rules to prevent global maintainers riding gunshot
over area maintainers?  If people are willing (AND ABLE - people's
circumstances, however regrettable, aren't relevant to the GDB project
as a whole) to maintain their area responsively, and if the global
maintainers are reasonable people, there are few problems and problems
can be easily resolved.

> > [I hope Richard will forgive me for using him as an example and for putting
> > words in his mouth...] This doesn't replace common sense - you generally
> > won't find Richard Henderson approving patches to the C++ frontend, because:
> >   - He knows he isn't familiar with it
> >   - He knows it has an active set of maintainers at all times
> > 
> > Similarly, just because he can check in patches to any target backend, that
> > doesn't mean he won't ask a target maintainer to look over it first.  If
> > someone objects to a patch in their area, he would generally not just check
> > it in anyway.  If they object to it after he checks it in, the two will
> > discuss the problem like reasonable people and come to some agreement.
> 
> The interaction dynamics of a committed patch are very different to
> those of an unapproved patch.  Once a patch is committed, the person
> with objections is, unequestionably, put in the back seat.

I disagree.  Especially in the presence of a reversion policy.

> > Some noticeable differences between these two models:
> >   - In the GCC model, more people are able/likely to check in patches which
> >     break things.
> >   - But in the GCC model, more people are able/likely to check in patches to
> >     fix it afterwards.
> 
> (ROFL.)
> 
> The GCC model involves a number of development phases and the above
> comments would only relate to one of those phases.  At other times
> increasingly strict controls are placed on what can be
> committed/approved.  The GCC group spend a significant (out of
> control?)  amount of their time trying to re-stablize GCC for their
> releases.

That's not accurate.  First of all, the comments relate to all three of
the phases in the GCC development process, although the effect is
mostly damped down in the third phase by the release manager's hand. 
Even in stage 3, other maintainers are free to exercise their
judgement.

Also, the GCC tree spends most of its time in a more useful state than
the above really portrays.  Things tend to work.  The restabilization
is 75% a function of less-frequently-tested platforms - and you can't
realistically claim that the stability of GDB development keeps THEM
working, since we have evidence to the contrary for each of the last
several releases.

> For GDB, on the other hand, interesting development can and does get
> approved/committed at any time.  GDB snaps are of such quality that we
> can confidently refer someone to current sources for fixes (except
> when I have a bad day like today :-).  Further, instead of using
> official releases (and as you yourself have done) arbitrary snaps can
> even make their way into a distro.

[Red Hat does this too, by the way.]  Having done it, I've resolved to
avoid it in the future.  Costs outweighed benefits.

> >   - Because more people have the privilege of approving a given patch,
> >     and fewer people's approvals are needed for any particular patch,
> >     patches (usually) get approved more quickly.
> >   - Development can happen more quickly, and does not get slowed to a
> >     standstill when (say) one of us is pulled off of community GDB work for
> >     an urgent customer project.  This happens all the time - I've never seen
> >     all the GDB maintainers with time for GDB at the same time.
> 
> The only way I've got my patches into GCC is by acting as a champion
> for their cause.  Exactly like Elena described.  Having worked on
> other open source projects I'd have to conclude the same also applied
> to them.

Let me put it this way.  When I initially became a GDB developer, I was
in the process of merging back MontaVista patches to the community for
all of the toolchain packages.  For GCC and binutils, I ended up with
(eventually) write-after-approval, which means exactly nothing. 
Depending on the patch, I have to nag people about them for a day or
several.  The only way I could get patches into GDB efficiently enough
for it to be worth my time was to invest a great deal of effort into
the GDB project; to become one of the maintainers in question.  Should
that really be necessary?

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

RE: [maint] The GDB maintenance process

[Zaretskii Eli]

This message was scanned for viruses and other malicious code by PrivaWall.


This mail was sent from ELTA SYS LTD.


> From: Daniel Jacobowitz [mailto:drow@mvista.com] 
> Sent: Monday, February 17, 2003 8:07 PM
> 
> Right now, we use stricter policies to prevent problems which cause
> breakage.  I think these policies are stifling us.

I don't think so.  I do agree that some maintainers should try to
review patches in their areas of responsibility more often, but
that doesn't necessarily mean we need to change the
maintenance model.

P.S. Sorry for a possibly messed-up message, this one and
Possibly future ones.  We had major changes in our mail
setup, which will take me some time to repair...


This message is processed by the PrivaWall Email Security Server. 

Re: [maint] The GDB maintenance process

[Andrew Cagney]

>  What does it mean to be a Global Maintainer, in practice?
>   - A certain amount of autonomy in the areas of GDB that no one wants to
>     take specific responsibility for.  There's no specific maintainer for
>     things like the frame support or the type/value systems.
>   - A little more freedom in approving patches to other people's areas of
>     GDB - not a lot, but it's definitely there.
>     [In practice, this depends on:
>      o How much risk you're willing to take of annoying people.
>      o How likely other maintainers are to shout at you about it.]
>   - Authority to approve patches covering general debugger issues.
> 
>  What does it mean to be a maintainer for a specific host/target/subsystem,
>  in practice?
>   - The authority to approve patches and apply your own patches to that area
>     of the debugger.
> 
> I'd like everyone to notice one thing missing from the above list.  No one
> has the _responsibility_ for approving patches.  This is a volunteer
> project, and anyone who's watched it in action for a little while will see
> that the volunteers are often busy and distracted.  There's no one who
> can handle or should have to handle the responsibilities of patch approval.

This isn't correct.  The MAINTAINERS file reads:

                        Various Maintainers

  Note individuals who maintain parts of the debugger need approval to
  check in changes outside of the immediate domain that they maintain.

  If there is no maintainer for a given domain then the responsibility
  falls to a global maintainer.

  If there are several maintainers for a given domain then
  responsibility falls to the first maintainer.  The first maintainer is
  free to devolve that responsibility among the other maintainers.

so it makes definite references to `responsibility'.  Put simply, `the
buck stops'.  Perhaps you should ask a different question `is the buck
ignored?' and what should be done if/when it is.

Even though GDB is developed by volunteers, it is entirely reasonable
to expect the volunteers to take their duties seriously.  Monitoring
the bug data base; approving patches; where helpful, contributing to
discussions.  Now that I typically file patches into the bug database,
this should be much easier.

However, what is unreasonable is to place un-realistic expectations on
the said maintainers.  The assumption that a volunteer is working 24x7
and living in north america isn't, for instance, reasonable.  A
significant number of GDB maintainers have real jobs and real
timezones.

> Some other GNU projects have a similar setup and don't have this problem. 
> GCC and binutils are my usual examples.  How do they avoid it?  They have a
> different definition of global maintainer.  That's what ours used to be
> called - Blanket Write Privileges.  The system works a little differently:
>   - Maintainers for specific areas of the compiler can commit/approve
>     patches to the areas they maintain without buy-in from a blanket
>     maintainer.
>   - Blanket maintainers can commit/approve patches anywhere without buy-in
>     from a specific area maintainer.

`write' was a simple typo.  From the outset (i.e., from when I got
handed this job) it was made very clear that neither I, nor any other
global maintainer, could override the decision of a specific
maintainer.

This was, and still is, very important.  It ensures that the area has
a certain level of continuity, and it ensures that I (or another
global maintainer) doesn't ride gunshot over a specific maintainer.

> [I hope Richard will forgive me for using him as an example and for putting
> words in his mouth...] This doesn't replace common sense - you generally
> won't find Richard Henderson approving patches to the C++ frontend, because:
>   - He knows he isn't familiar with it
>   - He knows it has an active set of maintainers at all times
> 
> Similarly, just because he can check in patches to any target backend, that
> doesn't mean he won't ask a target maintainer to look over it first.  If
> someone objects to a patch in their area, he would generally not just check
> it in anyway.  If they object to it after he checks it in, the two will
> discuss the problem like reasonable people and come to some agreement.

The interaction dynamics of a committed patch are very different to
those of an unapproved patch.  Once a patch is committed, the person
with objections is, unequestionably, put in the back seat.

> Some noticeable differences between these two models:
>   - In the GCC model, more people are able/likely to check in patches which
>     break things.
>   - But in the GCC model, more people are able/likely to check in patches to
>     fix it afterwards.

(ROFL.)

The GCC model involves a number of development phases and the above
comments would only relate to one of those phases.  At other times
increasingly strict controls are placed on what can be
committed/approved.  The GCC group spend a significant (out of
control?)  amount of their time trying to re-stablize GCC for their
releases.

For GDB, on the other hand, interesting development can and does get
approved/committed at any time.  GDB snaps are of such quality that we
can confidently refer someone to current sources for fixes (except
when I have a bad day like today :-).  Further, instead of using
official releases (and as you yourself have done) arbitrary snaps can
even make their way into a distro.

>   - Because more people have the privilege of approving a given patch,
>     and fewer people's approvals are needed for any particular patch,
>     patches (usually) get approved more quickly.
>   - Development can happen more quickly, and does not get slowed to a
>     standstill when (say) one of us is pulled off of community GDB work for
>     an urgent customer project.  This happens all the time - I've never seen
>     all the GDB maintainers with time for GDB at the same time.

The only way I've got my patches into GCC is by acting as a champion
for their cause.  Exactly like Elena described.  Having worked on
other open source projects I'd have to conclude the same also applied
to them.

Sounds like it really is time to drag out "aegis".

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I've discussed this general situation with a (small) sampling of other GDB
> developers and contributors - enough to know that I'm not alone in my
> concerns.  These aren't entirely my own words, either.  I'll let other
> people take credit/blame for them if they want to, and if I've represented
> their opinions accurately.

I recall you rasing this previously with me but I don't recall
agreeing with your opinions (my concerns run along similar lines to
what Elena posted).

Normally political issues get discussed with the global maintainers
(so that the mailing lists can focus on technical discussion).
However, if you feel that it is better discussed here, then so be it.

enjoy,
Andrew

Re: [maint] The GDB maintenance process

[Elena Zannoni]

Daniel Jacobowitz writes:
 > I get the feeling I'm already pretty unpopular for some of my opinions on
 > how GDB maintenane should work.  This isn't going to make it any better, but
 > I feel it should be said.
 > 

OK, I'll take the bait. No offense intended. Just read until the end.

 > I believe that our current process has some problems.  Let me try to
 > explain.  First, to make sure we're all on the same page...
 > 
 >  What does it mean to be a Global Maintainer, in practice?
 >   - A certain amount of autonomy in the areas of GDB that no one wants to
 >     take specific responsibility for.  There's no specific maintainer for
 >     things like the frame support or the type/value systems.
 >   - A little more freedom in approving patches to other people's areas of
 >     GDB - not a lot, but it's definitely there.
 >     [In practice, this depends on:
 >      o How much risk you're willing to take of annoying people.
 >      o How likely other maintainers are to shout at you about it.]

The above point is not part of the global maintainer description. I
take you want it to be? I don't agree. It just makes it possible for
particularly 'active' individuals to dominate the list and removes
the more egalitarian feel that this community has. You can say that
the linux kernel community is like that, and that gcc is like that,
etc, but we don't have to be. 

I have seen sometimes very quick turnarounds on patches during
holidays, and maybe some of such patches should have been thought
through more carefully. If you don't give time to the appropriate
maintainers to chime in, the entropy can become way too high, with
patches, and reverted patches going around.

 >   - Authority to approve patches covering general debugger issues.
 > 
 >  What does it mean to be a maintainer for a specific host/target/subsystem,
 >  in practice?
 >   - The authority to approve patches and apply your own patches to that area
 >     of the debugger.
 > 
 > I'd like everyone to notice one thing missing from the above list.  No one
 > has the _responsibility_ for approving patches.  This is a volunteer
 > project, and anyone who's watched it in action for a little while will see
 > that the volunteers are often busy and distracted.  There's no one who
 > can handle or should have to handle the responsibilities of patch approval.
 > 

So, you are proposing that we let the vacuum be filled with heroics?
I.e. quantity vs quality.

 > Another thing to think about: because of the layout of the above, there is
 > frequently no one who has the _right_ to approve a patch.  They require
 > buy-in from a number of additional maintainers.  In addition our volunteers
 > are often too busy to find time to respond to patches.  This impacts patches
 > from other maintainers (frequently, but generally a small impact) and from
 > outside contributors (happens less frequently, but larger impact - most of
 > these never get approved at all, from what I've seen).
 > 

Wait, lots of external contributor's patches never make it in because
of the copyright assignment problems. Also I see external people
dropping patches, not because the are not reviewed, but because they
*are reviewed*. I.e. a patch is submitted, I ask for some changes, and
the person never comes back with a new patch.

There would be also something to say about championing your patches,
ie, push a bit. For instance this is what David is doing with the
namespaces stuff, he is keeping me posted, and he makes sure I don't
fall asleep at the wheel. 

Just saying, things are not as black and white as they are portraited.

 > 
 > Some other GNU projects have a similar setup and don't have this problem. 
 > GCC and binutils are my usual examples.  How do they avoid it?  They have a
 > different definition of global maintainer.  That's what ours used to be
 > called - Blanket Write Privileges.  The system works a little differently:
 >   - Maintainers for specific areas of the compiler can commit/approve
 >     patches to the areas they maintain without buy-in from a blanket
 >     maintainer.

I think you are misunderstanding. Gdb is like that. If one maintains
one area, that's it, period. His/her approval is enough. Even if the
maintainer is not global, the authority is the same *in that specific
area*.

 >   - Blanket maintainers can commit/approve patches anywhere without buy-in
 >     from a specific area maintainer.
 > 
 > [I hope Richard will forgive me for using him as an example and for putting
 > words in his mouth...] This doesn't replace common sense - you generally
 > won't find Richard Henderson approving patches to the C++ frontend, because:
 >   - He knows he isn't familiar with it
 >   - He knows it has an active set of maintainers at all times
 > 
 > Similarly, just because he can check in patches to any target backend, that
 > doesn't mean he won't ask a target maintainer to look over it first.  If
 > someone objects to a patch in their area, he would generally not just check
 > it in anyway.  If they object to it after he checks it in, the two will
 > discuss the problem like reasonable people and come to some agreement.
 > 
 > 
 > Some noticeable differences between these two models:
 >   - In the GCC model, more people are able/likely to check in patches which
 >     break things.
 >   - But in the GCC model, more people are able/likely to check in patches to
 >     fix it afterwards.
 >   - Because more people have the privilege of approving a given patch,
 >     and fewer people's approvals are needed for any particular patch,
 >     patches (usually) get approved more quickly.
 >   - Development can happen more quickly, and does not get slowed to a
 >     standstill when (say) one of us is pulled off of community GDB work for
 >     an urgent customer project.  This happens all the time - I've never seen
 >     all the GDB maintainers with time for GDB at the same time.
 > 

Yes, we have day jobs, that are not 'work on FSF gdb'. If you loosen
the rules, then it becomes possible for a small, fully funded, elite to
take control.  

And how long would you wait before stepping in? That's a hard problem.
1 hour, one day, one working day, one week? 

 > 
 > Right now, we use stricter policies to prevent problems which cause
 > breakage.  I think these policies are stifling us.  Loosening them (and
 > maybe adding a formal patch reversion policy) would let more people fix
 > problems more easily, as they arise, without slowing development.
 > 

I really think that having the bug database track patches is a very
big improvement. We should try to push its use a bit more before
giving up on the current rules. 

 > If there are people on our Global Maintainer list that we don't think should
 > be trusted with the extra responsibility of the above, then perhaps we need
 > to rethink who belongs in that list.  I'm not pointing any fingers - I don't
 > have anyone in mind, and I've been quite happy working with absolutely all
 > of the current team.  Just putting the idea out.
 > 
 > 
 > I've discussed this general situation with a (small) sampling of other GDB
 > developers and contributors - enough to know that I'm not alone in my
 > concerns.  These aren't entirely my own words, either.  I'll let other
 > people take credit/blame for them if they want to, and if I've represented
 > their opinions accurately.

Hmm, I am not too happy with this behind the scenes discontent, if
there is something wrong, let's hear it.

Ok, that said, I too am seeing patches that are not reviewed. So there
are wrinkles in the process. But I don't think that the 'let's take
over' approach is an appropriate solution.

elena


 > 
 > -- 
 > Daniel Jacobowitz
 > MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Kevin Buettner]

On Feb 17,  1:07pm, Daniel Jacobowitz wrote:

> Right now, we use stricter policies to prevent problems which cause
> breakage.  I think these policies are stifling us.  Loosening them (and
> maybe adding a formal patch reversion policy) would let more people fix
> problems more easily, as they arise, without slowing development.

I agree.

Kevin

[maint] The GDB maintenance process

[Daniel Jacobowitz]

I get the feeling I'm already pretty unpopular for some of my opinions on
how GDB maintenane should work.  This isn't going to make it any better, but
I feel it should be said.

I believe that our current process has some problems.  Let me try to
explain.  First, to make sure we're all on the same page...

 What does it mean to be a Global Maintainer, in practice?
  - A certain amount of autonomy in the areas of GDB that no one wants to
    take specific responsibility for.  There's no specific maintainer for
    things like the frame support or the type/value systems.
  - A little more freedom in approving patches to other people's areas of
    GDB - not a lot, but it's definitely there.
    [In practice, this depends on:
     o How much risk you're willing to take of annoying people.
     o How likely other maintainers are to shout at you about it.]
  - Authority to approve patches covering general debugger issues.

 What does it mean to be a maintainer for a specific host/target/subsystem,
 in practice?
  - The authority to approve patches and apply your own patches to that area
    of the debugger.

I'd like everyone to notice one thing missing from the above list.  No one
has the _responsibility_ for approving patches.  This is a volunteer
project, and anyone who's watched it in action for a little while will see
that the volunteers are often busy and distracted.  There's no one who
can handle or should have to handle the responsibilities of patch approval.

Another thing to think about: because of the layout of the above, there is
frequently no one who has the _right_ to approve a patch.  They require
buy-in from a number of additional maintainers.  In addition our volunteers
are often too busy to find time to respond to patches.  This impacts patches
from other maintainers (frequently, but generally a small impact) and from
outside contributors (happens less frequently, but larger impact - most of
these never get approved at all, from what I've seen).


Some other GNU projects have a similar setup and don't have this problem. 
GCC and binutils are my usual examples.  How do they avoid it?  They have a
different definition of global maintainer.  That's what ours used to be
called - Blanket Write Privileges.  The system works a little differently:
  - Maintainers for specific areas of the compiler can commit/approve
    patches to the areas they maintain without buy-in from a blanket
    maintainer.
  - Blanket maintainers can commit/approve patches anywhere without buy-in
    from a specific area maintainer.

[I hope Richard will forgive me for using him as an example and for putting
words in his mouth...] This doesn't replace common sense - you generally
won't find Richard Henderson approving patches to the C++ frontend, because:
  - He knows he isn't familiar with it
  - He knows it has an active set of maintainers at all times

Similarly, just because he can check in patches to any target backend, that
doesn't mean he won't ask a target maintainer to look over it first.  If
someone objects to a patch in their area, he would generally not just check
it in anyway.  If they object to it after he checks it in, the two will
discuss the problem like reasonable people and come to some agreement.


Some noticeable differences between these two models:
  - In the GCC model, more people are able/likely to check in patches which
    break things.
  - But in the GCC model, more people are able/likely to check in patches to
    fix it afterwards.
  - Because more people have the privilege of approving a given patch,
    and fewer people's approvals are needed for any particular patch,
    patches (usually) get approved more quickly.
  - Development can happen more quickly, and does not get slowed to a
    standstill when (say) one of us is pulled off of community GDB work for
    an urgent customer project.  This happens all the time - I've never seen
    all the GDB maintainers with time for GDB at the same time.


Right now, we use stricter policies to prevent problems which cause
breakage.  I think these policies are stifling us.  Loosening them (and
maybe adding a formal patch reversion policy) would let more people fix
problems more easily, as they arise, without slowing development.

If there are people on our Global Maintainer list that we don't think should
be trusted with the extra responsibility of the above, then perhaps we need
to rethink who belongs in that list.  I'm not pointing any fingers - I don't
have anyone in mind, and I've been quite happy working with absolutely all
of the current team.  Just putting the idea out.


I've discussed this general situation with a (small) sampling of other GDB
developers and contributors - enough to know that I'm not alone in my
concerns.  These aren't entirely my own words, either.  I'll let other
people take credit/blame for them if they want to, and if I've represented
their opinions accurately.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer