[maint] The GDB maintenance process

[maint] The GDB maintenance process

[Daniel Jacobowitz]

I get the feeling I'm already pretty unpopular for some of my opinions on
how GDB maintenane should work.  This isn't going to make it any better, but
I feel it should be said.

I believe that our current process has some problems.  Let me try to
explain.  First, to make sure we're all on the same page...

 What does it mean to be a Global Maintainer, in practice?
  - A certain amount of autonomy in the areas of GDB that no one wants to
    take specific responsibility for.  There's no specific maintainer for
    things like the frame support or the type/value systems.
  - A little more freedom in approving patches to other people's areas of
    GDB - not a lot, but it's definitely there.
    [In practice, this depends on:
     o How much risk you're willing to take of annoying people.
     o How likely other maintainers are to shout at you about it.]
  - Authority to approve patches covering general debugger issues.

 What does it mean to be a maintainer for a specific host/target/subsystem,
 in practice?
  - The authority to approve patches and apply your own patches to that area
    of the debugger.

I'd like everyone to notice one thing missing from the above list.  No one
has the _responsibility_ for approving patches.  This is a volunteer
project, and anyone who's watched it in action for a little while will see
that the volunteers are often busy and distracted.  There's no one who
can handle or should have to handle the responsibilities of patch approval.

Another thing to think about: because of the layout of the above, there is
frequently no one who has the _right_ to approve a patch.  They require
buy-in from a number of additional maintainers.  In addition our volunteers
are often too busy to find time to respond to patches.  This impacts patches
from other maintainers (frequently, but generally a small impact) and from
outside contributors (happens less frequently, but larger impact - most of
these never get approved at all, from what I've seen).


Some other GNU projects have a similar setup and don't have this problem. 
GCC and binutils are my usual examples.  How do they avoid it?  They have a
different definition of global maintainer.  That's what ours used to be
called - Blanket Write Privileges.  The system works a little differently:
  - Maintainers for specific areas of the compiler can commit/approve
    patches to the areas they maintain without buy-in from a blanket
    maintainer.
  - Blanket maintainers can commit/approve patches anywhere without buy-in
    from a specific area maintainer.

[I hope Richard will forgive me for using him as an example and for putting
words in his mouth...] This doesn't replace common sense - you generally
won't find Richard Henderson approving patches to the C++ frontend, because:
  - He knows he isn't familiar with it
  - He knows it has an active set of maintainers at all times

Similarly, just because he can check in patches to any target backend, that
doesn't mean he won't ask a target maintainer to look over it first.  If
someone objects to a patch in their area, he would generally not just check
it in anyway.  If they object to it after he checks it in, the two will
discuss the problem like reasonable people and come to some agreement.


Some noticeable differences between these two models:
  - In the GCC model, more people are able/likely to check in patches which
    break things.
  - But in the GCC model, more people are able/likely to check in patches to
    fix it afterwards.
  - Because more people have the privilege of approving a given patch,
    and fewer people's approvals are needed for any particular patch,
    patches (usually) get approved more quickly.
  - Development can happen more quickly, and does not get slowed to a
    standstill when (say) one of us is pulled off of community GDB work for
    an urgent customer project.  This happens all the time - I've never seen
    all the GDB maintainers with time for GDB at the same time.


Right now, we use stricter policies to prevent problems which cause
breakage.  I think these policies are stifling us.  Loosening them (and
maybe adding a formal patch reversion policy) would let more people fix
problems more easily, as they arise, without slowing development.

If there are people on our Global Maintainer list that we don't think should
be trusted with the extra responsibility of the above, then perhaps we need
to rethink who belongs in that list.  I'm not pointing any fingers - I don't
have anyone in mind, and I've been quite happy working with absolutely all
of the current team.  Just putting the idea out.


I've discussed this general situation with a (small) sampling of other GDB
developers and contributors - enough to know that I'm not alone in my
concerns.  These aren't entirely my own words, either.  I'll let other
people take credit/blame for them if they want to, and if I've represented
their opinions accurately.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Elena Zannoni]

Daniel Jacobowitz writes:
 > I get the feeling I'm already pretty unpopular for some of my opinions on
 > how GDB maintenane should work.  This isn't going to make it any better, but
 > I feel it should be said.
 > 

OK, I'll take the bait. No offense intended. Just read until the end.

 > I believe that our current process has some problems.  Let me try to
 > explain.  First, to make sure we're all on the same page...
 > 
 >  What does it mean to be a Global Maintainer, in practice?
 >   - A certain amount of autonomy in the areas of GDB that no one wants to
 >     take specific responsibility for.  There's no specific maintainer for
 >     things like the frame support or the type/value systems.
 >   - A little more freedom in approving patches to other people's areas of
 >     GDB - not a lot, but it's definitely there.
 >     [In practice, this depends on:
 >      o How much risk you're willing to take of annoying people.
 >      o How likely other maintainers are to shout at you about it.]

The above point is not part of the global maintainer description. I
take you want it to be? I don't agree. It just makes it possible for
particularly 'active' individuals to dominate the list and removes
the more egalitarian feel that this community has. You can say that
the linux kernel community is like that, and that gcc is like that,
etc, but we don't have to be. 

I have seen sometimes very quick turnarounds on patches during
holidays, and maybe some of such patches should have been thought
through more carefully. If you don't give time to the appropriate
maintainers to chime in, the entropy can become way too high, with
patches, and reverted patches going around.

 >   - Authority to approve patches covering general debugger issues.
 > 
 >  What does it mean to be a maintainer for a specific host/target/subsystem,
 >  in practice?
 >   - The authority to approve patches and apply your own patches to that area
 >     of the debugger.
 > 
 > I'd like everyone to notice one thing missing from the above list.  No one
 > has the _responsibility_ for approving patches.  This is a volunteer
 > project, and anyone who's watched it in action for a little while will see
 > that the volunteers are often busy and distracted.  There's no one who
 > can handle or should have to handle the responsibilities of patch approval.
 > 

So, you are proposing that we let the vacuum be filled with heroics?
I.e. quantity vs quality.

 > Another thing to think about: because of the layout of the above, there is
 > frequently no one who has the _right_ to approve a patch.  They require
 > buy-in from a number of additional maintainers.  In addition our volunteers
 > are often too busy to find time to respond to patches.  This impacts patches
 > from other maintainers (frequently, but generally a small impact) and from
 > outside contributors (happens less frequently, but larger impact - most of
 > these never get approved at all, from what I've seen).
 > 

Wait, lots of external contributor's patches never make it in because
of the copyright assignment problems. Also I see external people
dropping patches, not because the are not reviewed, but because they
*are reviewed*. I.e. a patch is submitted, I ask for some changes, and
the person never comes back with a new patch.

There would be also something to say about championing your patches,
ie, push a bit. For instance this is what David is doing with the
namespaces stuff, he is keeping me posted, and he makes sure I don't
fall asleep at the wheel. 

Just saying, things are not as black and white as they are portraited.

 > 
 > Some other GNU projects have a similar setup and don't have this problem. 
 > GCC and binutils are my usual examples.  How do they avoid it?  They have a
 > different definition of global maintainer.  That's what ours used to be
 > called - Blanket Write Privileges.  The system works a little differently:
 >   - Maintainers for specific areas of the compiler can commit/approve
 >     patches to the areas they maintain without buy-in from a blanket
 >     maintainer.

I think you are misunderstanding. Gdb is like that. If one maintains
one area, that's it, period. His/her approval is enough. Even if the
maintainer is not global, the authority is the same *in that specific
area*.

 >   - Blanket maintainers can commit/approve patches anywhere without buy-in
 >     from a specific area maintainer.
 > 
 > [I hope Richard will forgive me for using him as an example and for putting
 > words in his mouth...] This doesn't replace common sense - you generally
 > won't find Richard Henderson approving patches to the C++ frontend, because:
 >   - He knows he isn't familiar with it
 >   - He knows it has an active set of maintainers at all times
 > 
 > Similarly, just because he can check in patches to any target backend, that
 > doesn't mean he won't ask a target maintainer to look over it first.  If
 > someone objects to a patch in their area, he would generally not just check
 > it in anyway.  If they object to it after he checks it in, the two will
 > discuss the problem like reasonable people and come to some agreement.
 > 
 > 
 > Some noticeable differences between these two models:
 >   - In the GCC model, more people are able/likely to check in patches which
 >     break things.
 >   - But in the GCC model, more people are able/likely to check in patches to
 >     fix it afterwards.
 >   - Because more people have the privilege of approving a given patch,
 >     and fewer people's approvals are needed for any particular patch,
 >     patches (usually) get approved more quickly.
 >   - Development can happen more quickly, and does not get slowed to a
 >     standstill when (say) one of us is pulled off of community GDB work for
 >     an urgent customer project.  This happens all the time - I've never seen
 >     all the GDB maintainers with time for GDB at the same time.
 > 

Yes, we have day jobs, that are not 'work on FSF gdb'. If you loosen
the rules, then it becomes possible for a small, fully funded, elite to
take control.  

And how long would you wait before stepping in? That's a hard problem.
1 hour, one day, one working day, one week? 

 > 
 > Right now, we use stricter policies to prevent problems which cause
 > breakage.  I think these policies are stifling us.  Loosening them (and
 > maybe adding a formal patch reversion policy) would let more people fix
 > problems more easily, as they arise, without slowing development.
 > 

I really think that having the bug database track patches is a very
big improvement. We should try to push its use a bit more before
giving up on the current rules. 

 > If there are people on our Global Maintainer list that we don't think should
 > be trusted with the extra responsibility of the above, then perhaps we need
 > to rethink who belongs in that list.  I'm not pointing any fingers - I don't
 > have anyone in mind, and I've been quite happy working with absolutely all
 > of the current team.  Just putting the idea out.
 > 
 > 
 > I've discussed this general situation with a (small) sampling of other GDB
 > developers and contributors - enough to know that I'm not alone in my
 > concerns.  These aren't entirely my own words, either.  I'll let other
 > people take credit/blame for them if they want to, and if I've represented
 > their opinions accurately.

Hmm, I am not too happy with this behind the scenes discontent, if
there is something wrong, let's hear it.

Ok, that said, I too am seeing patches that are not reviewed. So there
are wrinkles in the process. But I don't think that the 'let's take
over' approach is an appropriate solution.

elena


 > 
 > -- 
 > Daniel Jacobowitz
 > MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

Thanks to everyone who responded; I'll try to address everything.

I want to share a piece of perspective on why I raise this issue now. 
I'm finding the GDB development process to be too slow to be workable -
patches take a month or more routinely, and I don't have the time to
juggle them for that long.  I'm facing that I may need to back down my
level of involvement as a consequence of how awkward the development
model is for getting any forward progress.

Maybe that means that I just don't have the time and the patience to be
a useful contributor to GDB.  Me, I think that it means that we need to
make the process less painful for contributors.

On Mon, Feb 17, 2003 at 09:35:53PM -0500, Andrew Cagney wrote:
> > I've discussed this general situation with a (small) sampling of other GDB
> > developers and contributors - enough to know that I'm not alone in my
> > concerns.  These aren't entirely my own words, either.  I'll let other
> > people take credit/blame for them if they want to, and if I've represented
> > their opinions accurately.
> 
> I recall you rasing this previously with me but I don't recall
> agreeing with your opinions (my concerns run along similar lines to
> what Elena posted).

Hmm, I don't remember actually discussing this with you before.  Must
be going soft.  We've certainly butted heads over related topics.

> Normally political issues get discussed with the global maintainers
> (so that the mailing lists can focus on technical discussion).
> However, if you feel that it is better discussed here, then so be it.

If you feel that's more appropriate, just say so.  I've no particular
preference for where the conversation happens, as long as it happens.



On Mon, Feb 17, 2003 at 04:05:40PM -0500, Elena Zannoni wrote:
> Daniel Jacobowitz writes:
>  > I get the feeling I'm already pretty unpopular for some of my opinions on
>  > how GDB maintenane should work.  This isn't going to make it any better, but
>  > I feel it should be said.
>  > 
> 
> OK, I'll take the bait. No offense intended. Just read until the end.
> 
>  > I believe that our current process has some problems.  Let me try to
>  > explain.  First, to make sure we're all on the same page...
>  > 
>  >  What does it mean to be a Global Maintainer, in practice?
>  >   - A certain amount of autonomy in the areas of GDB that no one wants to
>  >     take specific responsibility for.  There's no specific maintainer for
>  >     things like the frame support or the type/value systems.
>  >   - A little more freedom in approving patches to other people's areas of
>  >     GDB - not a lot, but it's definitely there.
>  >     [In practice, this depends on:
>  >      o How much risk you're willing to take of annoying people.
>  >      o How likely other maintainers are to shout at you about it.]
> 
> The above point is not part of the global maintainer description. I
> take you want it to be? I don't agree. It just makes it possible for
> particularly 'active' individuals to dominate the list and removes
> the more egalitarian feel that this community has. You can say that
> the linux kernel community is like that, and that gcc is like that,
> etc, but we don't have to be. 

I don't especially want it to be; I want something different, farther
down.  This is my vision of how we work right now.  The above is
informal but in my experience a pretty accurate summary of the way
things really are.

> I have seen sometimes very quick turnarounds on patches during
> holidays, and maybe some of such patches should have been thought
> through more carefully. If you don't give time to the appropriate
> maintainers to chime in, the entropy can become way too high, with
> patches, and reverted patches going around.

I guess I just don't see this to be as much of a problem as others do.
For one thing, with the higher entropy level, more development actually
happens.

>  > Another thing to think about: because of the layout of the above, there is
>  > frequently no one who has the _right_ to approve a patch.  They require
>  > buy-in from a number of additional maintainers.  In addition our volunteers
>  > are often too busy to find time to respond to patches.  This impacts patches
>  > from other maintainers (frequently, but generally a small impact) and from
>  > outside contributors (happens less frequently, but larger impact - most of
>  > these never get approved at all, from what I've seen).
>  > 
> 
> Wait, lots of external contributor's patches never make it in because
> of the copyright assignment problems. Also I see external people
> dropping patches, not because the are not reviewed, but because they
> *are reviewed*. I.e. a patch is submitted, I ask for some changes, and
> the person never comes back with a new patch.
> 
> There would be also something to say about championing your patches,
> ie, push a bit. For instance this is what David is doing with the
> namespaces stuff, he is keeping me posted, and he makes sure I don't
> fall asleep at the wheel. 
> 
> Just saying, things are not as black and white as they are portraited.

Sure.  But my portrayal isn't entirely inaccurate, either.  One example
- inter-compilation-unit references in DWARF-2.  It's been posted at
least twice, by different contributors.  When asked for changes,
they've come back with changes.  It's still not supported, over a year
later.

On the subject of David, infinite kudos to him for his
single-mindedness and patience.  When I look at the amount of forward
progress he's made compared to the time he's invested in making it, I
become even more convinced that we have a problem.

>  > Some other GNU projects have a similar setup and don't have this problem. 
>  > GCC and binutils are my usual examples.  How do they avoid it?  They have a
>  > different definition of global maintainer.  That's what ours used to be
>  > called - Blanket Write Privileges.  The system works a little differently:
>  >   - Maintainers for specific areas of the compiler can commit/approve
>  >     patches to the areas they maintain without buy-in from a blanket
>  >     maintainer.
> 
> I think you are misunderstanding. Gdb is like that. If one maintains
> one area, that's it, period. His/her approval is enough. Even if the
> maintainer is not global, the authority is the same *in that specific
> area*.

I know that.  That's no different between GCC and GDB; I'm not
discussing a radical 100% turnaround here.

>  >   - Blanket maintainers can commit/approve patches anywhere without buy-in
>  >     from a specific area maintainer.
>  > 
>  > [I hope Richard will forgive me for using him as an example and for putting
>  > words in his mouth...] This doesn't replace common sense - you generally
>  > won't find Richard Henderson approving patches to the C++ frontend, because:
>  >   - He knows he isn't familiar with it
>  >   - He knows it has an active set of maintainers at all times
>  > 
>  > Similarly, just because he can check in patches to any target backend, that
>  > doesn't mean he won't ask a target maintainer to look over it first.  If
>  > someone objects to a patch in their area, he would generally not just check
>  > it in anyway.  If they object to it after he checks it in, the two will
>  > discuss the problem like reasonable people and come to some agreement.
>  > 
>  > 
>  > Some noticeable differences between these two models:
>  >   - In the GCC model, more people are able/likely to check in patches which
>  >     break things.
>  >   - But in the GCC model, more people are able/likely to check in patches to
>  >     fix it afterwards.
>  >   - Because more people have the privilege of approving a given patch,
>  >     and fewer people's approvals are needed for any particular patch,
>  >     patches (usually) get approved more quickly.
>  >   - Development can happen more quickly, and does not get slowed to a
>  >     standstill when (say) one of us is pulled off of community GDB work for
>  >     an urgent customer project.  This happens all the time - I've never seen
>  >     all the GDB maintainers with time for GDB at the same time.
>  > 
> 
> Yes, we have day jobs, that are not 'work on FSF gdb'. If you loosen
> the rules, then it becomes possible for a small, fully funded, elite to
> take control.  
> 
> And how long would you wait before stepping in? That's a hard problem.
> 1 hour, one day, one working day, one week? 

Are we talking about the same thing?  I don't think I understand you.

First of all, the idea of a small, fully funded elite taking control of
the project doesn't make any sense to me.  Either their changes are
worthwhile - the existing maintainers will cooperate with them - or
they overstep their boundaries - they are asked to cease.  We don't
hand out maintainership to all comers.

It's not about control of GDB, it's about the growth and quality of
GDB!  I don't care who controls it.  I'll mostly resist making a
comment here about who currently controls GDB - more than half the
global maintainers work at the same place - because it's an entirely
benign and open control.

Second, what do you mean by "step in"?  Step in in response to what?  A
broken patch?  A violation of authority?  The latter and partly the
former are what GCC has a formal steering committee for.  We could do
the same thing, or use a vote of the global maintainers.  In practice
it's rarely necessary.

>  > Right now, we use stricter policies to prevent problems which cause
>  > breakage.  I think these policies are stifling us.  Loosening them (and
>  > maybe adding a formal patch reversion policy) would let more people fix
>  > problems more easily, as they arise, without slowing development.
>  > 
> 
> I really think that having the bug database track patches is a very
> big improvement. We should try to push its use a bit more before
> giving up on the current rules. 

I don't like it, but that's just my general anti-GNATS rancor.  Maybe
Bugzilla will be better.

>  > If there are people on our Global Maintainer list that we don't think should
>  > be trusted with the extra responsibility of the above, then perhaps we need
>  > to rethink who belongs in that list.  I'm not pointing any fingers - I don't
>  > have anyone in mind, and I've been quite happy working with absolutely all
>  > of the current team.  Just putting the idea out.
>  > 
>  > 
>  > I've discussed this general situation with a (small) sampling of other GDB
>  > developers and contributors - enough to know that I'm not alone in my
>  > concerns.  These aren't entirely my own words, either.  I'll let other
>  > people take credit/blame for them if they want to, and if I've represented
>  > their opinions accurately.
> 
> Hmm, I am not too happy with this behind the scenes discontent, if
> there is something wrong, let's hear it.

That's what I'm doing!  I assure you, I don't enjoy generating this
much rancor.

On Mon, Feb 17, 2003 at 11:28:46PM -0500, Andrew Cagney wrote:
> 
> >  What does it mean to be a Global Maintainer, in practice?
> >   - A certain amount of autonomy in the areas of GDB that no one wants to
> >     take specific responsibility for.  There's no specific maintainer for
> >     things like the frame support or the type/value systems.
> >   - A little more freedom in approving patches to other people's areas of
> >     GDB - not a lot, but it's definitely there.
> >     [In practice, this depends on:
> >      o How much risk you're willing to take of annoying people.
> >      o How likely other maintainers are to shout at you about it.]
> >   - Authority to approve patches covering general debugger issues.
> > 
> >  What does it mean to be a maintainer for a specific host/target/subsystem,
> >  in practice?
> >   - The authority to approve patches and apply your own patches to that area
> >     of the debugger.
> > 
> > I'd like everyone to notice one thing missing from the above list.  No one
> > has the _responsibility_ for approving patches.  This is a volunteer
> > project, and anyone who's watched it in action for a little while will see
> > that the volunteers are often busy and distracted.  There's no one who
> > can handle or should have to handle the responsibilities of patch approval.
> 
> This isn't correct.  The MAINTAINERS file reads:
> 
>                         Various Maintainers
> 
>   Note individuals who maintain parts of the debugger need approval to
>   check in changes outside of the immediate domain that they maintain.
> 
>   If there is no maintainer for a given domain then the responsibility
>   falls to a global maintainer.
> 
>   If there are several maintainers for a given domain then
>   responsibility falls to the first maintainer.  The first maintainer is
>   free to devolve that responsibility among the other maintainers.
> 
> so it makes definite references to `responsibility'.  Put simply, `the
> buck stops'.  Perhaps you should ask a different question `is the buck
> ignored?' and what should be done if/when it is.
> 
> Even though GDB is developed by volunteers, it is entirely reasonable
> to expect the volunteers to take their duties seriously.  Monitoring
> the bug data base; approving patches; where helpful, contributing to
> discussions.  Now that I typically file patches into the bug database,
> this should be much easier.
> 
> However, what is unreasonable is to place un-realistic expectations on
> the said maintainers.  The assumption that a volunteer is working 24x7
> and living in north america isn't, for instance, reasonable.  A
> significant number of GDB maintainers have real jobs and real
> timezones.

[North America doesn't have real jobs and real timezones?  They're just
different...]  OK, I withdraw my comment about no one having
responsibility.  Let me try from another side.

The volunteers do take their duty seriously, but with the density of
responsibility involved [i.e., when a particular person or people are
responsible for a decision, there tend to be just one or two of them]
it's very hard to make forward progress when people are busy.  And
we're all busy people.  Significant patches almost always take three
weeks or more if they can't be approved by their author; some take
months, even with repeated prodding.

We don't just have a bottleneck, we have a bottleneck per maintainer.

> > Some other GNU projects have a similar setup and don't have this problem. 
> > GCC and binutils are my usual examples.  How do they avoid it?  They have a
> > different definition of global maintainer.  That's what ours used to be
> > called - Blanket Write Privileges.  The system works a little differently:
> >   - Maintainers for specific areas of the compiler can commit/approve
> >     patches to the areas they maintain without buy-in from a blanket
> >     maintainer.
> >   - Blanket maintainers can commit/approve patches anywhere without buy-in
> >     from a specific area maintainer.
> 
> `write' was a simple typo.  From the outset (i.e., from when I got
> handed this job) it was made very clear that neither I, nor any other
> global maintainer, could override the decision of a specific
> maintainer.
> 
> This was, and still is, very important.  It ensures that the area has
> a certain level of continuity, and it ensures that I (or another
> global maintainer) doesn't ride gunshot over a specific maintainer.

Which raises two thoughts in my mind.
 - What are the benefits of this continuity, and do they really
outweigh the disadvantages (slow progress, primarily)?  My position is
that they do _not_.
 - Why do we need rules to prevent global maintainers riding gunshot
over area maintainers?  If people are willing (AND ABLE - people's
circumstances, however regrettable, aren't relevant to the GDB project
as a whole) to maintain their area responsively, and if the global
maintainers are reasonable people, there are few problems and problems
can be easily resolved.

> > [I hope Richard will forgive me for using him as an example and for putting
> > words in his mouth...] This doesn't replace common sense - you generally
> > won't find Richard Henderson approving patches to the C++ frontend, because:
> >   - He knows he isn't familiar with it
> >   - He knows it has an active set of maintainers at all times
> > 
> > Similarly, just because he can check in patches to any target backend, that
> > doesn't mean he won't ask a target maintainer to look over it first.  If
> > someone objects to a patch in their area, he would generally not just check
> > it in anyway.  If they object to it after he checks it in, the two will
> > discuss the problem like reasonable people and come to some agreement.
> 
> The interaction dynamics of a committed patch are very different to
> those of an unapproved patch.  Once a patch is committed, the person
> with objections is, unequestionably, put in the back seat.

I disagree.  Especially in the presence of a reversion policy.

> > Some noticeable differences between these two models:
> >   - In the GCC model, more people are able/likely to check in patches which
> >     break things.
> >   - But in the GCC model, more people are able/likely to check in patches to
> >     fix it afterwards.
> 
> (ROFL.)
> 
> The GCC model involves a number of development phases and the above
> comments would only relate to one of those phases.  At other times
> increasingly strict controls are placed on what can be
> committed/approved.  The GCC group spend a significant (out of
> control?)  amount of their time trying to re-stablize GCC for their
> releases.

That's not accurate.  First of all, the comments relate to all three of
the phases in the GCC development process, although the effect is
mostly damped down in the third phase by the release manager's hand. 
Even in stage 3, other maintainers are free to exercise their
judgement.

Also, the GCC tree spends most of its time in a more useful state than
the above really portrays.  Things tend to work.  The restabilization
is 75% a function of less-frequently-tested platforms - and you can't
realistically claim that the stability of GDB development keeps THEM
working, since we have evidence to the contrary for each of the last
several releases.

> For GDB, on the other hand, interesting development can and does get
> approved/committed at any time.  GDB snaps are of such quality that we
> can confidently refer someone to current sources for fixes (except
> when I have a bad day like today :-).  Further, instead of using
> official releases (and as you yourself have done) arbitrary snaps can
> even make their way into a distro.

[Red Hat does this too, by the way.]  Having done it, I've resolved to
avoid it in the future.  Costs outweighed benefits.

> >   - Because more people have the privilege of approving a given patch,
> >     and fewer people's approvals are needed for any particular patch,
> >     patches (usually) get approved more quickly.
> >   - Development can happen more quickly, and does not get slowed to a
> >     standstill when (say) one of us is pulled off of community GDB work for
> >     an urgent customer project.  This happens all the time - I've never seen
> >     all the GDB maintainers with time for GDB at the same time.
> 
> The only way I've got my patches into GCC is by acting as a champion
> for their cause.  Exactly like Elena described.  Having worked on
> other open source projects I'd have to conclude the same also applied
> to them.

Let me put it this way.  When I initially became a GDB developer, I was
in the process of merging back MontaVista patches to the community for
all of the toolchain packages.  For GCC and binutils, I ended up with
(eventually) write-after-approval, which means exactly nothing. 
Depending on the patch, I have to nag people about them for a day or
several.  The only way I could get patches into GDB efficiently enough
for it to be worth my time was to invest a great deal of effort into
the GDB project; to become one of the maintainers in question.  Should
that really be necessary?

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Joel Brobecker]

Daniel said:
> I want to share a piece of perspective on why I raise this issue now. 
> I'm finding the GDB development process to be too slow to be workable -
> patches take a month or more routinely,

> Maybe that means that I just don't have the time and the patience to be
> a useful contributor to GDB.  Me, I think that it means that we need to
> make the process less painful for contributors.

So far, I have only really started making some contributions maybe a
year and a half ago, and my level of involvment is very far from a lot
of the developpers I see on this list. But I tend to agree with Daniel
that patches _sometimes_ take a very long time to be included, and that
it is easy to be discouraged. 

I think there are also a lot of patches floating around that are waiting
for at least a review. One of the ACT developpers actually stopped
submitting his patches, just because he did not receive any feedback.
And yet he spent the effort of creating a PR and attaching the patch to
this PR! So I am submitting the changes on his behalf now, and took on
the job of trying to push for these patches to be reviewed.

I agree that we are all very busy, and that it's natural that reviews
do not always happen in a timely manor. In fact, I am generally happy
with the delay-to-review, but my feeling is that the GDB community is
losing a lot of valuable work because it never gets looked at.

With my modest experience on the GDB project, and without any experience
on any other GNU project like this, it's difficult to make any
recommendation. My feeling is that we could try relaxing a bit the
rules, and allow global maintainers to approves changes if the
associated maintainer is unable to review them in say, a few (couple?)
of weeks.

Look at http://sources.redhat.com/ml/gdb-patches/2002-10/msg00586.html
for intance. This patch addresses a GDB crash, and the fix has been
sitting since Oct 30th, despite 2 pings, 1 recommendation for approval,
and a message to the maintainers. I do not want to blame the maintainers
here, but in my opinion the global maintainers should be able to act as
surrogates when they see that a patch has been sitting for more than a
certain duration, and they have the knowledge to review it.

-- 
Joel

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> With my modest experience on the GDB project, and without any experience
> on any other GNU project like this, it's difficult to make any
> recommendation. My feeling is that we could try relaxing a bit the
> rules, and allow global maintainers to approves changes if the
> associated maintainer is unable to review them in say, a few (couple?)
> of weeks.

Fernando does this already.  He also sends out e-mail saying he is off 
line for a bit (he is down the hall from me which is an advantage). 
Other maintainers do not, sigh!  More of a concern is that repeated 
pings often result in radio silence.

I don't think having a global maintainer step in after N weeks will 
really help.  It just butters over what may be a more serious underlying 
problem.

Andrew

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Wed, Feb 19, 2003 at 10:47:46AM -0500, Andrew Cagney wrote:
> 
> >With my modest experience on the GDB project, and without any experience
> >on any other GNU project like this, it's difficult to make any
> >recommendation. My feeling is that we could try relaxing a bit the
> >rules, and allow global maintainers to approves changes if the
> >associated maintainer is unable to review them in say, a few (couple?)
> >of weeks.
> 
> Fernando does this already.  He also sends out e-mail saying he is off 
> line for a bit (he is down the hall from me which is an advantage). 
> Other maintainers do not, sigh!  More of a concern is that repeated 
> pings often result in radio silence.

Yes, Fernando does this.  He also tends to disappear for long periods
of time without doing it.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Andrew Cagney]

>> Fernando does this already.  He also sends out e-mail saying he is off 
>> line for a bit (he is down the hall from me which is an advantage). 
>> Other maintainers do not, sigh!  More of a concern is that repeated 
>> pings often result in radio silence.
> 
> 
> Yes, Fernando does this.  He also tends to disappear for long periods
> of time without doing it.

Fernando and I finding that my e-mail pings to him got black holed for 6 
months didn't help (this is where being able to walk down the hall is a 
disadvantage :-).   I don't know that others have the luxury of that excuse.

Andrew

Re: [maint] The GDB maintenance process

[Daniel Berlin]

On Tuesday, February 18, 2003, at 08:49  PM, Daniel Jacobowitz wrote:

> Thanks to everyone who responded; I'll try to address everything.
>
> I want to share a piece of perspective on why I raise this issue now.
> I'm finding the GDB development process to be too slow to be workable -
> patches take a month or more routinely, and I don't have the time to
> juggle them for that long.  I'm facing that I may need to back down my
> level of involvement as a consequence of how awkward the development
> model is for getting any forward progress.
>

A few days ago, I actually ran statistics on how long it takes for a 
patch to get first review in gcc vs gdb over the past year, and for 
gcc, it's 2.some odd days.  Believe it or not, for gdb, it was well 
over two weeks.
That's not good.
Outliers are the norm in gdb, there was no middle ground.
Either something takes 1 day to get a review, or it takes months.
The only reason the average is actually only a little over two weeks is 
because of a string of 0 day turnarounds that occasionally happen.
> Maybe that means that I just don't have the time and the patience to be
> a useful contributor to GDB.  Me, I think that it means that we need to
> make the process less painful for contributors.
FWIW, I agree, but i hold no hope it will happen.
I'm not a cynic, i'm just experienced.
I'd be a cynic if it wasn't based on experience, of course.
>
>> I have seen sometimes very quick turnarounds on patches during
>> holidays, and maybe some of such patches should have been thought
>> through more carefully. If you don't give time to the appropriate
>> maintainers to chime in, the entropy can become way too high, with
>> patches, and reverted patches going around.
>
> I guess I just don't see this to be as much of a problem as others do.
> For one thing, with the higher entropy level, more development actually
> happens.
Bingo.
I don't think we should stall development (and in the extreme,  even if 
it means we can't make quality releases any day of the year) because 
mistakes occasionally happen in patches, or because not every 
maintainer in existence has said something about a patch.  That's a 
recipe for no progress.  The old "adding more developers to a late 
project makes it later because of communication overhead" problem.

>
>>> Another thing to think about: because of the layout of the above, 
>>> there is
>>> frequently no one who has the _right_ to approve a patch.  They 
>>> require
>>> buy-in from a number of additional maintainers.  In addition our 
>>> volunteers
>>> are often too busy to find time to respond to patches.  This impacts 
>>> patches
>>> from other maintainers (frequently, but generally a small impact) 
>>> and from
>>> outside contributors (happens less frequently, but larger impact - 
>>> most of
>>> these never get approved at all, from what I've seen).
>>>
>>
>> Wait, lots of external contributor's patches never make it in because
>> of the copyright assignment problems.

Bull.
This counts for *maybe* 10% of patches that don't make it in, if that.
>> Also I see external people
>> dropping patches, not because the are not reviewed, but because they
>> *are reviewed*. I.e. a patch is submitted, I ask for some changes, and
>> the person never comes back with a new patch.

Or maybe it's because frequently the review took >1 month, and they 
just don't want to spend that much more time waiting again after making 
changes, or they moved on to other projects.

Have you noticed GDB *very rarely* keeps minor contributors coming back?
OTOH, most opensource projects (including gcc) i am a part of 
frequently have the same minor contributors, again and again.

Maybe we need community exit interviews to back this point up.

>
> Are we talking about the same thing?  I don't think I understand you.
>
> First of all, the idea of a small, fully funded elite taking control of
> the project doesn't make any sense to me.  Either their changes are
> worthwhile - the existing maintainers will cooperate with them - or
> they overstep their boundaries - they are asked to cease.  We don't
> hand out maintainership to all comers.

Well, they gave it to me (and Stan Shebs :P) at one point, so ...
:)


>
> That's not accurate.  First of all, the comments relate to all three of
> the phases in the GCC development process, although the effect is
> mostly damped down in the third phase by the release manager's hand.
> Even in stage 3, other maintainers are free to exercise their
> judgement.
>
> Also, the GCC tree spends most of its time in a more useful state than
> the above really portrays.
Yup.
Andrew's just plain wrong on this one.
All patches are bootstrapped and reg-tested on at least one platform, 
and those that are larger or could affect multiple platforms are 
bootstrapped and regtested on multiple platforms.

Any needed stabilization is generally because of latent bugs in 
*other*, less tested platforms, that improvements expose.
*Not* problems in the patches themselves.


>
>> For GDB, on the other hand, interesting development can and does get
>> approved/committed at any time.  GDB snaps are of such quality that we
>> can confidently refer someone to current sources for fixes (except
>> when I have a bad day like today :-).  Further, instead of using
>> official releases (and as you yourself have done) arbitrary snaps can
>> even make their way into a distro.
>
> [Red Hat does this too, by the way.]  Having done it, I've resolved to
> avoid it in the future.  Costs outweighed benefits.
>
I agree that it should be avoided, too.
It's just painful and not necessary here.
Our users don't clamor for a release on a certain date, nor have we 
ever had a need to make one.
The only advantage this adds is to companies using gdb sources and 
wanting to do merges to an internal tree.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> A few days ago, I actually ran statistics on how long it takes for a patch to get first review in gcc vs gdb over the past year, and for gcc, it's 2.some odd days.  Believe it or not, for gdb, it was well over two weeks.
> That's not good.

Ah, statistics.  Ah metrics.

Given most outliers are now in the bug database can you perhaphs break 
it down by area and, hopefully, more usefuly, look at each area's change 
rate.

It is the change/progress that is important, not the absolute numbers. 
A lack of change that is of concern.  For instance, I worry about build 
and remote problems.  The former definitly makes progress, the latter 
less so :-(

Andrew

PS: And metrics are made to be abused.  As soon as people know the 
metric that they are being measured by, the quickly addapt their 
behavour to make that specific metric look better.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Sure.  But my portrayal isn't entirely inaccurate, either.  One example
> - inter-compilation-unit references in DWARF-2.  It's been posted at
> least twice, by different contributors.  When asked for changes,
> they've come back with changes.  It's still not supported, over a year
> later.

Yes, I think it is fair to say that we've recently dropped baton on that 
one :-(  JimB, what's the current status?

Anyway a mid-mortem is in order.

GDB has a serious problem with _old_ forks.  cf:

- hp fork
- apple fork
- act's fork
- caldera's fork

You'll note that, in each of these cases, they really are forks(1).  The 
relevent developers created their own repository and then went off into 
the wilderness only to mysteriously appear later with a jumbo patch.  I 
believe that decision for forking was largly driven by the desire to, 
due to short term commercial pressure, avoid doing the right thing.

Not unreasonably, the GDB developers asked that such jumbo patches be 
broken down into something more managable, more up-to-date, and (too 
often) `lint' free(2).

Also, not unreasonably, the GDB developers asked/questioned 
architectural decisions made on those branches.

Less realistic, perhaps, is an over expectation of how much a 
contributor can reasonably be asked.  Even ignoring those `comercial 
pressures', many contributors figure `take it or leave it'.  Maintainers 
need to be willing to not only spend time adding glamerous new features, 
but also when an important patch appears, pick it up and run with it(3).

One thing GCC(4) and GDB are now is encouraging exprementation on 
branches development to always occure on branches cut from the the 
relevant repository.  For GDB we've had both success stories but also 
disasters.  With that in mind, and looking at the GCC / GDB success 
stories, I'd suggest the following guidelines:

- branches shall post all commits
They don't need approval but can be commented on.

- branches shall to be focused
The interps branch started out too large with too many changes - look at 
the size of the final commit compared to that branch at its peak.  Much 
time was lost because the branch started with too much lint :-(

- branches shall track mainline.
This keeps the level of divergence under control.  It also keeps the 
pressure on developers to push cleanups and other stuff into the mainline.

Andrew

--

(1) I was discussing this with a GCC developer and they pointed out, for 
the reasons I note, that these are not branches.

(2) As in the stuff you find on your clothing (not the program).  Forks 
and branches, like navel's, are shockers for breeding lint - entropy 
that serves no useful purpose.

(4) Yes, as I mentioned, I discuss this with GCC developers to see what 
lessons can be learnt.

(3) Note that if I pick up a really old patch, I'll often review, tweak 
and commit it.  I figure that if a patch is more than a month old, it 
isn't realistic to be asking a contributor to make minor changes.

Re: [maint] The GDB maintenance process

[David Carlton]

On Wed, 19 Feb 2003 09:54:50 -0500, Andrew Cagney <ac131313@redhat.com> said:

> One thing GCC(4) and GDB are now is encouraging exprementation on
> branches development to always occure on branches cut from the the
> relevant repository.

Would Red Hat be happy hosting significant branches for other
companies?  Would those other companies be happy depending on Red
Hat's CVS servers?  (Maybe a reasonable answer here is "yes, but money
should change hands".)  It might be nice if GDB were using a source
code management tool that didn't depend on having a single repository,
making it easier for people to maintain public branches elsewhere but
to still sync them with an official branch.

Anyways, I'd been planning for a while to write up some documentation
about creating branches; I'll try to do that sooner than later.  I was
somewhat resistant to the idea myself, but in retrospect it was
unquestionably the right way to go for my project.

David Carlton
carlton@math.stanford.edu

Re: [maint] The GDB maintenance process

[Kevin Buettner]

On Feb 19,  9:33am, David Carlton wrote:

> On Wed, 19 Feb 2003 09:54:50 -0500, Andrew Cagney <ac131313@redhat.com> said:
> 
> > One thing GCC(4) and GDB are now is encouraging exprementation on
> > branches development to always occure on branches cut from the the
> > relevant repository.
> 
> Would Red Hat be happy hosting significant branches for other
> companies?

I assume you're referring to the sources.redhat.com repository, right?

I think the answer is "yes", so long as the branch in question is
intended to (eventually) further mainline gdb development.  If the
branch is just some custom bit of work that is likely to never make
it to the mainline, probably not.

> Would those other companies be happy depending on Red
> Hat's CVS servers?

I doubt it.  But...

For the contract work that Red Hat itself does, Red Hat does not depend
solely on the external (sources.redhat.com) respository.  Red Hat has
an internal repository that it uses for customer work.  From time to time,
the public repository is merged to Red Hat's internal repository.  Also,
from time to time, work that was performed for a customer which couldn't
immediately go into the public repository is pushed out to the public.

> (Maybe a reasonable answer here is "yes, but money
> should change hands".)  It might be nice if GDB were using a source
> code management tool that didn't depend on having a single repository,
> making it easier for people to maintain public branches elsewhere but
> to still sync them with an official branch.

Hmm.  Something to think about.

Kevin

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> On Feb 19,  9:33am, David Carlton wrote:
> 
> 
>> On Wed, 19 Feb 2003 09:54:50 -0500, Andrew Cagney <ac131313@redhat.com> said:
>> 
> 
>> > One thing GCC(4) and GDB are now is encouraging exprementation on
>> > branches development to always occure on branches cut from the the
>> > relevant repository.
> 
>> 
>> Would Red Hat be happy hosting significant branches for other
>> companies?

FYI, it isn't a Red Hat decision.

> I assume you're referring to the sources.redhat.com repository, right?
> 
> I think the answer is "yes", so long as the branch in question is
> intended to (eventually) further mainline gdb development.  If the
> branch is just some custom bit of work that is likely to never make
> it to the mainline, probably not.

And that decision is up to this group.

>> Would those other companies be happy depending on Red
>> Hat's CVS servers?

Should money making corporate types be willing to bet the crown jewls on 
those same resources (sware, subversion, sourceforge)?  Interesting 
question.

Red Hat makes sourceware available as a public service.  It funds it 
rather than owns it.  The machine is adminstered by a group of overseers 
(chrisf and jasonm are two).

Andrew

Re: [maint] The GDB maintenance process

[Christopher Faylor]

On Wed, Feb 19, 2003 at 02:01:14PM -0500, Andrew Cagney wrote:
>Red Hat makes sourceware available as a public service.  It funds it 
>rather than owns it.  The machine is adminstered by a group of overseers 
>(chrisf and jasonm are two).

I know that Jason likes svn but I haven't used it.  I'm not sure that it
would be easy to convince me to spend any time trying to convert
everything to a new source control system.  CVS is nice because it lives
everywhere and is a known commodity.  Do we really want to think about
the headache of having people set up svn (or arch, or whatever) if they
want to develop with gdb?  Do we want to make gdb's repository separate
from, say, cygwin's or newlib's?  Seems like a lot of tricky work to me.

cgf

Re: [maint] The GDB maintenance process

[Jason Molenda]

On Wed, Feb 19, 2003 at 02:01:14PM -0500, Andrew Cagney wrote:
>Red Hat makes sourceware available as a public service.  It funds it 
>rather than owns it.  The machine is adminstered by a group of overseers 
>(chrisf and jasonm are two).

Minor correction - RH owns the physical sources.redhat.com system.
It pays for the network bandwidth, electricity, and the necessary
network IS resources.  The time spent actually administering the
box (almost entirely Chris Faylor these days) is outside the scope
of cgf's Day Job, so most administration is on a volunteer basis.

On Wed, Feb 19, 2003 at 03:39:50PM -0500, Christopher Faylor wrote:

> I know that Jason likes svn but I haven't used it.  I'm not sure that it
> would be easy to convince me to spend any time trying to convert
> everything to a new source control system.  CVS is nice because it lives
> everywhere and is a known commodity.  

I don't think I saw anyone suggesting alternative SCMs... cagney
wrote "sware, subversion, sourceforge", but he probably meant
to write "subversions", as in "subversions.gnu.org".


(svn is really interesting, and I expect I'll be playing with it
more this year, but it'll be a while before anyone wants to switch
their project to using it.  If nothing else, each developer in that
project has to install/set up svn before they can access the
repository.)

Jason

Re: [maint] The GDB maintenance process

[Christopher Faylor]

On Wed, Feb 19, 2003 at 03:17:09PM -0800, Jason Molenda wrote:
>On Wed, Feb 19, 2003 at 03:39:50PM -0500, Christopher Faylor wrote:
>>I know that Jason likes svn but I haven't used it.  I'm not sure that
>>it would be easy to convince me to spend any time trying to convert
>>everything to a new source control system.  CVS is nice because it
>>lives everywhere and is a known commodity.
>
>I don't think I saw anyone suggesting alternative SCMs...  cagney wrote
>"sware, subversion, sourceforge", but he probably meant to write
>"subversions", as in "subversions.gnu.org".

I probably responded one message too late.  David Carlton said this:

>It might be nice if GDB were using a source code management tool that
>didn't depend on having a single repository, making it easier for
>people to maintain public branches elsewhere but to still sync them
>with an official branch.

So when I saw Andrew mention subversion, I thought we were still talking
about source control systems rather than systems which run source
control systems.

cgf

Re: [maint] The GDB maintenance process

[David Carlton]

On Wed, 19 Feb 2003 10:57:20 -0700, Kevin Buettner <kevinb@redhat.com> said:

> For the contract work that Red Hat itself does, Red Hat does not
> depend solely on the external (sources.redhat.com) respository.  Red
> Hat has an internal repository that it uses for customer work.  From
> time to time, the public repository is merged to Red Hat's internal
> repository.  Also, from time to time, work that was performed for a
> customer which couldn't immediately go into the public repository is
> pushed out to the public.

Thanks for the info; that sounds sensible.

It would be nice if there were a scheme that allowed such development
to go on smoothly while still making it as easy as possible for
changes in separate repositories to make it into the mainline.  I
don't really know to do that, though.

David Carlton
carlton@math.stanford.edu

Re: [maint] The GDB maintenance process

[Richard Earnshaw]

> One thing GCC(4) and GDB are now is encouraging exprementation on 
> branches development to always occure on branches cut from the the 
> relevant repository.  For GDB we've had both success stories but also 
> disasters.  With that in mind, and looking at the GCC / GDB success 
> stories, I'd suggest the following guidelines:
> 
> - branches shall post all commits
> They don't need approval but can be commented on.
> 
> - branches shall to be focused
> The interps branch started out too large with too many changes - look at 
> the size of the final commit compared to that branch at its peak.  Much 
> time was lost because the branch started with too much lint :-(
> 
> - branches shall track mainline.
> This keeps the level of divergence under control.  It also keeps the 
> pressure on developers to push cleanups and other stuff into the mainline.

I think there's a few other ground-rules that ought to apply, particularly 
with respect to the first point:

-Branches should have an owner.
The owner can set further policy for a branch, but may not change the 
ground rules.  In particular, they can set a policy for commits (be it 
adding more reviewers or deciding who can commit).

-All commits to a branch must be covered by an assignment
This saves us from the situation where a branch might become contaminated.

R.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> -Branches should have an owner.
> The owner can set further policy for a branch, but may not change the 
> ground rules.  In particular, they can set a policy for commits (be it 
> adding more reviewers or deciding who can commit).

Yep.

> -All commits to a branch must be covered by an assignment
> This saves us from the situation where a branch might become contaminated.

Um, er, yes, `obviously' (How did I forget that one :-( :-).

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

>>  > Right now, we use stricter policies to prevent problems which cause
>>  > breakage.  I think these policies are stifling us.  Loosening them (and
>>  > maybe adding a formal patch reversion policy) would let more people fix
>>  > problems more easily, as they arise, without slowing development.
>>  > 
>> 
>> I really think that having the bug database track patches is a very
>> big improvement. We should try to push its use a bit more before
>> giving up on the current rules. 
> 
> 
> I don't like it, but that's just my general anti-GNATS rancor.  Maybe
> Bugzilla will be better.

The idea of tracking things as bugs or GNATS?

Tracking this stuff in a public database is, I think, clearly better 
than the status quo (my mailbox).  It's already been mentioned that 
there is possibly a lack of response with some maintainers.  At least 
with me putting unreviewed patches in the database, we can figure out 
what patches there are, and if there is a problem.  (my todo list 
includes checking that all the key developers are on the bug tracking 
list, I suspect that some are not.)

Anyway, have a look in the database, you'll notice all sorts of 
interesting things.

For instance search for tdep & h8300.  There is an obvious backlog and 
someone (a global maintainer like yourself or I) needs to step up and 
work through them with the contributors.  That task, while no where near 
as glamerous as a new feature, is absolutly needed and maintainers do 
need to be willing to chip in (in fact I think that task and 
documentation should be given a higher profile when handing out 
`credit').  Hopefuly these patches will even yield a new developer.

Andrew

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Wed, Feb 19, 2003 at 10:17:12AM -0500, Andrew Cagney wrote:
> 
> >> > Right now, we use stricter policies to prevent problems which cause
> >> > breakage.  I think these policies are stifling us.  Loosening them (and
> >> > maybe adding a formal patch reversion policy) would let more people fix
> >> > problems more easily, as they arise, without slowing development.
> >> > 
> >>
> >>I really think that having the bug database track patches is a very
> >>big improvement. We should try to push its use a bit more before
> >>giving up on the current rules. 
> >
> >
> >I don't like it, but that's just my general anti-GNATS rancor.  Maybe
> >Bugzilla will be better.
> 
> The idea of tracking things as bugs or GNATS?

Mostly, GNATS.

> Tracking this stuff in a public database is, I think, clearly better 
> than the status quo (my mailbox).  It's already been mentioned that 
> there is possibly a lack of response with some maintainers.  At least 
> with me putting unreviewed patches in the database, we can figure out 
> what patches there are, and if there is a problem.  (my todo list 
> includes checking that all the key developers are on the bug tracking 
> list, I suspect that some are not.)
> 
> Anyway, have a look in the database, you'll notice all sorts of 
> interesting things.

Which reminds me.  We've got two GNATS databases set up for GDB: 'gdb'
and 'gdb-patches'.  Should we use the gdb-patches GNATS database to
separate them from bug reports?

> For instance search for tdep & h8300.  There is an obvious backlog and 
> someone (a global maintainer like yourself or I) needs to step up and 
> work through them with the contributors.  That task, while no where near 
> as glamerous as a new feature, is absolutly needed and maintainers do 
> need to be willing to chip in (in fact I think that task and 
> documentation should be given a higher profile when handing out 
> `credit').  Hopefuly these patches will even yield a new developer.

TBH, I've been avoiding it because I don't know the slightest thing
about the h8300 and it seemed like someone (I don't remember who -
MichaelS maybe?) did.  If I'm wrong then one of us is just going to
have to suck it up and learn about the h8300... yay, another project.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Which reminds me.  We've got two GNATS databases set up for GDB: 'gdb'
> and 'gdb-patches'.  Should we use the gdb-patches GNATS database to
> separate them from bug reports?

No!!!!  That gdb-patches database should be deleted.  It's dead.

People already have to track:

- gdb@
- gdb-patches@
- gnats@

and that is to complicated for some.  At least, by having both patches 
and bugs in a single database, we've a one stop-shop.  A better change 
is to dump gdb-patches@

> TBH, I've been avoiding it because I don't know the slightest thing
> about the h8300 and it seemed like someone (I don't remember who -
> MichaelS maybe?) did.  If I'm wrong then one of us is just going to
> have to suck it up and learn about the h8300... yay, another project.

No worse than having me review them.  All that can be done is review 
them at a coding / broad architectural level and assume that the 
contributor, given that they are using the target, have it working. 
Knowing that they run the testsuite also helps.

Andrew

Re: [maint] The GDB maintenance process

[Christopher Faylor]

On Wed, Feb 19, 2003 at 11:29:13AM -0500, Andrew Cagney wrote:
>That gdb-patches database should be deleted.  It's dead.

Done.

cgf

Re: [maint] The GDB maintenance process

[Jason Molenda]

On Wed, Feb 19, 2003 at 11:29:13AM -0500, Andrew Cagney wrote:
> 
> > Which reminds me.  We've got two GNATS databases set up for GDB: 'gdb'
> > and 'gdb-patches'.  Should we use the gdb-patches GNATS database to
> > separate them from bug reports?
> 
> No!!!!  That gdb-patches database should be deleted.  It's dead.
> 
> People already have to track:
> 
> - gdb@
> - gdb-patches@
> - gnats@
> 
> and that is to complicated for some.  At least, by having both patches 
> and bugs in a single database, we've a one stop-shop.  A better change 
> is to dump gdb-patches@


We've got two separate discussions here:  The social conventions
and procedures for how patches are discussed/applied, and the
infrastructure to support those things.

Using GNATS as the infrastructure to track patches is pathetic.
Using mailing lists to track patches is annoying.

For a minute, though, imagine a magical patch tracking database.
It would be at the core of the patch workflow, not something glommed
on to the side--anything that says "you send your note to gdb-patches
and then you enter it into the patch database" is missing the point.
It wouldn't be a bug database, it would be a patch database--it's
got to track updates to the patch, it's got to relate groups of
patches or patches dependant on other patches.  It'll track the
discussion about patches.  It could track various maintainer's
opinions on a patch (I like the -1/+0/+1 Apache scheme); it could
track when the patch is blocked on a particular person (submitter,
a maintainer), or maybe when the patch is waiting approval by any
one of several people.  It could even relate GNATS bugs and patches
that are attempting to fix those bugs, so when someone sees a bug
in the GNATS database that hasn't yet been fixed, they can click
over to the magical patch tracking database and see why that patch
isn't in yet.

All of these transactions are still visible on a gdb-patches list,
of course, but the mailing list is a read-only view on to what's
happening.  All additional comments or follow-ups are routed through
the magical patch tracking database, and reflected back out to the
gdb-patches list.


I've heard Sourceforge has such a tool.  JimI was telling me at
lunch how they track the tcl patches this way.  And there is a
separate mechanism that tcl and Python both use for larger-scope
proposals ("PEP" for Python, a "Python Enhancement Proposal").
For instance, here's a PEP to add a boolean type to Python:
	http://www.python.org/peps/pep-0285.html
And it has a link into the Patch Tracker sourceforget database 
for the patch to implement the proposal:
	http://python.org/sf/528022



Don't fall into the trap of saying "the only two ways patches can
be tracked are the gdb-patches mailing list or gnats".  Let's start
by imagining a patch database designed around our actual workflow,
one which would require a minor change in how we work and add a
tiny bit of extra patch-management overhead, and yield benefits of
not losing track of patches.  

If such a magical patch tracking database exists, would it be worth
changing our workflow to use it?  I know that patch submitters would
eagerly jump on such a system, but unless all the actual maintainers
think it's a net gain it's not going anywhere.

Once that's all decided - what we want, and if we'd even use it if
we had it - then there's the minor matter of making it exist. :-)
Maybe bugzilla or gnats can be adapted, maybe another program can
be found from the net or a free bit of code from Sourceforge's
PatchTracker can be appropriated, or maybe someone gets the joyus
chance to write it from scratch.  


Before we get bogged down in implementation details and discussions,
it's critical to discuss what such a system would do, and if we're
all willing to push gdb patch work through it.  If we can't dream
up such a system, then let's be happy with gdb-patches and URLs
for patch references.

J

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> 
> 
> We've got two separate discussions here:  The social conventions
> and procedures for how patches are discussed/applied, and the
> infrastructure to support those things.
> 
> Using GNATS as the infrastructure to track patches is pathetic.

Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
failing hard disk'.

> Using mailing lists to track patches is annoying.

Er, you can't track patches using a mailing list.  A mailing list can be 
used to submit/discuss patches.  It can't be used to track their state. 
  that needs a database.

Time to install aegis, ay?

Andrew

Re: [maint] The GDB maintenance process

[Jason Molenda]

On Wed, Feb 19, 2003 at 06:57:01PM -0500, Andrew Cagney wrote:

> > Using GNATS as the infrastructure to track patches is pathetic.
> 
> Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
> failing hard disk'.

Well, yes. :-)  I didn't mean "you, the fellow who has put patches
into gnats, are a fool" -- I meant that the overhead over putting
patches in gnats is too high compared with just sending them to
gdb-patches.  IMHO this is a method that will fail, which is why
I dragged my feet when Elena originally requested the gdb-patches
gnats database be set up.  Ignoring the fact that gnats is a bug
tracker--not a magical patch tracking database--as long as it isn't
at the center of every developer/maintainer's patch workflow, it
will be doomed to irrelevance.

It's got to be easy, it's got to be relevant, and it's gotta be the
way everything is done.

> > Using mailing lists to track patches is annoying.
> 
> Er, you can't track patches using a mailing list.  A mailing list can be 
> used to submit/discuss patches.  It can't be used to track their state. 
>   that needs a database.

I was speaking loosely - I meant the combination of the mailing
list and the web archives of that mailing list.  The mailing list
web archives are a being used as the patch repository right
now--people use URLs into the archives to refer to old patches,
they use google or the htdig search engine to find old patches,
and they grope around blindly to figure out what ever happened with
a given patch.

> Time to install aegis, ay?

I've never looked at Aegis, so I can't say.  First the gdb maintainers
and developers need to decide what they want and will use, then
make it exist; not look at what exists and settle for it.  Maybe
Aegis is exactly what we'd all love in a magical patch tracking
database and we can use it as-is, but IMHO it's too early in that
discussion to care one way or another.


J

Re: [maint] The GDB maintenance process

[Elena Zannoni]

Jason Molenda writes:
 > On Wed, Feb 19, 2003 at 06:57:01PM -0500, Andrew Cagney wrote:
 > 
 > > > Using GNATS as the infrastructure to track patches is pathetic.
 > > 
 > > Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
 > > failing hard disk'.
 > 
 > Well, yes. :-)  I didn't mean "you, the fellow who has put patches
 > into gnats, are a fool" -- I meant that the overhead over putting
 > patches in gnats is too high compared with just sending them to
 > gdb-patches.  IMHO this is a method that will fail, which is why
 > I dragged my feet when Elena originally requested the gdb-patches
 > gnats database be set up.  Ignoring the fact that gnats is a bug

It must have been my evil twin, because I don't remember asking for
this (I am in favor of something like it, though). To be honest, it
was a project that Jim Blandy started but wasn't finished. It was
abandoned because there were problems with including a patch
preserving spaces, or something like that.

 > tracker--not a magical patch tracking database--as long as it isn't
 > at the center of every developer/maintainer's patch workflow, it
 > will be doomed to irrelevance.
 > 
 > It's got to be easy, it's got to be relevant, and it's gotta be the
 > way everything is done.
 > 
 > > > Using mailing lists to track patches is annoying.
 > > 
 > > Er, you can't track patches using a mailing list.  A mailing list can be 
 > > used to submit/discuss patches.  It can't be used to track their state. 
 > >   that needs a database.
 > 
 > I was speaking loosely - I meant the combination of the mailing
 > list and the web archives of that mailing list.  The mailing list
 > web archives are a being used as the patch repository right
 > now--people use URLs into the archives to refer to old patches,
 > they use google or the htdig search engine to find old patches,
 > and they grope around blindly to figure out what ever happened with
 > a given patch.
 > 
 > > Time to install aegis, ay?
 > 
 > I've never looked at Aegis, so I can't say.  First the gdb maintainers
 > and developers need to decide what they want and will use, then
 > make it exist; not look at what exists and settle for it.  Maybe
 > Aegis is exactly what we'd all love in a magical patch tracking
 > database and we can use it as-is, but IMHO it's too early in that
 > discussion to care one way or another.
 > 
 > 
 > J

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> On Wed, Feb 19, 2003 at 06:57:01PM -0500, Andrew Cagney wrote:
> 
> 
>> > Using GNATS as the infrastructure to track patches is pathetic.
> 
>> 
>> Not as pathetic as `cagney's mailbox sitting on a lapbrick with a 
>> failing hard disk'.
> 
> 
> Well, yes. :-)  I didn't mean "you, the fellow who has put patches
> into gnats, are a fool" -- I meant that the overhead over putting
> patches in gnats is too high compared with just sending them to
> gdb-patches.  IMHO this is a method that will fail, which is why
> I dragged my feet when Elena originally requested the gdb-patches
> gnats database be set up.  Ignoring the fact that gnats is a bug
> tracker--not a magical patch tracking database--as long as it isn't
> at the center of every developer/maintainer's patch workflow, it
> will be doomed to irrelevance.

Actually the overhead is effectively zero.  Any patch not touched for a 
week gets run through a script that turns it into a gnats entry.

Once there, I and everyone else can search it.  I know of non-mainstream 
developers that perfer this as it is easier to pick up a task (and 
easier to track than either gdb@ or gdb-patches@).

It sux, but sux less than not doing it.  It is interum measure that will 
hopefully be replaced by bugzilla.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I've discussed this general situation with a (small) sampling of other GDB
> developers and contributors - enough to know that I'm not alone in my
> concerns.  These aren't entirely my own words, either.  I'll let other
> people take credit/blame for them if they want to, and if I've represented
> their opinions accurately.

I recall you rasing this previously with me but I don't recall
agreeing with your opinions (my concerns run along similar lines to
what Elena posted).

Normally political issues get discussed with the global maintainers
(so that the mailing lists can focus on technical discussion).
However, if you feel that it is better discussed here, then so be it.

enjoy,
Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

>  What does it mean to be a Global Maintainer, in practice?
>   - A certain amount of autonomy in the areas of GDB that no one wants to
>     take specific responsibility for.  There's no specific maintainer for
>     things like the frame support or the type/value systems.
>   - A little more freedom in approving patches to other people's areas of
>     GDB - not a lot, but it's definitely there.
>     [In practice, this depends on:
>      o How much risk you're willing to take of annoying people.
>      o How likely other maintainers are to shout at you about it.]
>   - Authority to approve patches covering general debugger issues.
> 
>  What does it mean to be a maintainer for a specific host/target/subsystem,
>  in practice?
>   - The authority to approve patches and apply your own patches to that area
>     of the debugger.
> 
> I'd like everyone to notice one thing missing from the above list.  No one
> has the _responsibility_ for approving patches.  This is a volunteer
> project, and anyone who's watched it in action for a little while will see
> that the volunteers are often busy and distracted.  There's no one who
> can handle or should have to handle the responsibilities of patch approval.

This isn't correct.  The MAINTAINERS file reads:

                        Various Maintainers

  Note individuals who maintain parts of the debugger need approval to
  check in changes outside of the immediate domain that they maintain.

  If there is no maintainer for a given domain then the responsibility
  falls to a global maintainer.

  If there are several maintainers for a given domain then
  responsibility falls to the first maintainer.  The first maintainer is
  free to devolve that responsibility among the other maintainers.

so it makes definite references to `responsibility'.  Put simply, `the
buck stops'.  Perhaps you should ask a different question `is the buck
ignored?' and what should be done if/when it is.

Even though GDB is developed by volunteers, it is entirely reasonable
to expect the volunteers to take their duties seriously.  Monitoring
the bug data base; approving patches; where helpful, contributing to
discussions.  Now that I typically file patches into the bug database,
this should be much easier.

However, what is unreasonable is to place un-realistic expectations on
the said maintainers.  The assumption that a volunteer is working 24x7
and living in north america isn't, for instance, reasonable.  A
significant number of GDB maintainers have real jobs and real
timezones.

> Some other GNU projects have a similar setup and don't have this problem. 
> GCC and binutils are my usual examples.  How do they avoid it?  They have a
> different definition of global maintainer.  That's what ours used to be
> called - Blanket Write Privileges.  The system works a little differently:
>   - Maintainers for specific areas of the compiler can commit/approve
>     patches to the areas they maintain without buy-in from a blanket
>     maintainer.
>   - Blanket maintainers can commit/approve patches anywhere without buy-in
>     from a specific area maintainer.

`write' was a simple typo.  From the outset (i.e., from when I got
handed this job) it was made very clear that neither I, nor any other
global maintainer, could override the decision of a specific
maintainer.

This was, and still is, very important.  It ensures that the area has
a certain level of continuity, and it ensures that I (or another
global maintainer) doesn't ride gunshot over a specific maintainer.

> [I hope Richard will forgive me for using him as an example and for putting
> words in his mouth...] This doesn't replace common sense - you generally
> won't find Richard Henderson approving patches to the C++ frontend, because:
>   - He knows he isn't familiar with it
>   - He knows it has an active set of maintainers at all times
> 
> Similarly, just because he can check in patches to any target backend, that
> doesn't mean he won't ask a target maintainer to look over it first.  If
> someone objects to a patch in their area, he would generally not just check
> it in anyway.  If they object to it after he checks it in, the two will
> discuss the problem like reasonable people and come to some agreement.

The interaction dynamics of a committed patch are very different to
those of an unapproved patch.  Once a patch is committed, the person
with objections is, unequestionably, put in the back seat.

> Some noticeable differences between these two models:
>   - In the GCC model, more people are able/likely to check in patches which
>     break things.
>   - But in the GCC model, more people are able/likely to check in patches to
>     fix it afterwards.

(ROFL.)

The GCC model involves a number of development phases and the above
comments would only relate to one of those phases.  At other times
increasingly strict controls are placed on what can be
committed/approved.  The GCC group spend a significant (out of
control?)  amount of their time trying to re-stablize GCC for their
releases.

For GDB, on the other hand, interesting development can and does get
approved/committed at any time.  GDB snaps are of such quality that we
can confidently refer someone to current sources for fixes (except
when I have a bad day like today :-).  Further, instead of using
official releases (and as you yourself have done) arbitrary snaps can
even make their way into a distro.

>   - Because more people have the privilege of approving a given patch,
>     and fewer people's approvals are needed for any particular patch,
>     patches (usually) get approved more quickly.
>   - Development can happen more quickly, and does not get slowed to a
>     standstill when (say) one of us is pulled off of community GDB work for
>     an urgent customer project.  This happens all the time - I've never seen
>     all the GDB maintainers with time for GDB at the same time.

The only way I've got my patches into GCC is by acting as a champion
for their cause.  Exactly like Elena described.  Having worked on
other open source projects I'd have to conclude the same also applied
to them.

Sounds like it really is time to drag out "aegis".

Andrew

Re: [maint] The GDB maintenance process

[Jim Blandy]

ac131313@redhat.com (Andrew Cagney) writes:
> > Some noticeable differences between these two models:
> >   - In the GCC model, more people are able/likely to check in patches which
> >     break things.
> >   - But in the GCC model, more people are able/likely to check in patches to
> >     fix it afterwards.
> 
> (ROFL.)
> 
> The GCC model involves a number of development phases and the above
> comments would only relate to one of those phases.  At other times
> increasingly strict controls are placed on what can be
> committed/approved.  The GCC group spend a significant (out of
> control?)  amount of their time trying to re-stablize GCC for their
> releases.
> 
> For GDB, on the other hand, interesting development can and does get
> approved/committed at any time.  GDB snaps are of such quality that we
> can confidently refer someone to current sources for fixes (except
> when I have a bad day like today :-).  Further, instead of using
> official releases (and as you yourself have done) arbitrary snaps can
> even make their way into a distro.

The problem is, being that stable has a cost associated with it.  GCC
pays that cost at certain parts in their cycle; we pay that cost all
the time, every day.

Sure, it's nice being able to grab an arbitrary GDB snapshot and
distribute it.  But the rules which give us that property also slow
GDB's development into a more capable system.  The decision to pay
that particular price for that particular benefit is what we want to
open up for discussion.

Stability needs to be balanced with progress in features; although I'd
hate to see it go too far towards the latter, I think the balance we
have now is tipped too far in favor of the former.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> ac131313@redhat.com (Andrew Cagney) writes:
> 
>> > Some noticeable differences between these two models:
>> >   - In the GCC model, more people are able/likely to check in patches which
>> >     break things.
>> >   - But in the GCC model, more people are able/likely to check in patches to
>> >     fix it afterwards.
> 
>> 
>> (ROFL.)
>> 
>> The GCC model involves a number of development phases and the above
>> comments would only relate to one of those phases.  At other times
>> increasingly strict controls are placed on what can be
>> committed/approved.  The GCC group spend a significant (out of
>> control?)  amount of their time trying to re-stablize GCC for their
>> releases.
>> 
>> For GDB, on the other hand, interesting development can and does get
>> approved/committed at any time.  GDB snaps are of such quality that we
>> can confidently refer someone to current sources for fixes (except
>> when I have a bad day like today :-).  Further, instead of using
>> official releases (and as you yourself have done) arbitrary snaps can
>> even make their way into a distro.
> 
> 
> The problem is, being that stable has a cost associated with it.  GCC
> pays that cost at certain parts in their cycle; we pay that cost all
> the time, every day.

GDB is less stable then you might think.  Right now while both:

- interps
- frame

are causing problems they are not getting in the way of DavidC's dwarf2 
stuff (gee wiz, both my doing :-/).  GDB always builds, gdb always 
`break main; run'.  Is that too much to ask?

The problem with GDB's stability is that allows people to quickly forget:

- what it is like with out it
- how much gain there is from it
- how relatively small the pain
- how much more expensive it is to have to re-do something later
- how, with a bit of peer revew, problematic code could have been done 
right the first time (and how much that fallout costs).

Andrew

Re: [maint] The GDB maintenance process

[Daniel Jacobowitz]

On Wed, Feb 19, 2003 at 11:19:07AM -0500, Andrew Cagney wrote:
> >ac131313@redhat.com (Andrew Cagney) writes:
> >
> >>> Some noticeable differences between these two models:
> >>>   - In the GCC model, more people are able/likely to check in patches 
> >>which
> >>>     break things.
> >>>   - But in the GCC model, more people are able/likely to check in 
> >>patches to
> >>>     fix it afterwards.
> >
> >>
> >>(ROFL.)
> >>
> >>The GCC model involves a number of development phases and the above
> >>comments would only relate to one of those phases.  At other times
> >>increasingly strict controls are placed on what can be
> >>committed/approved.  The GCC group spend a significant (out of
> >>control?)  amount of their time trying to re-stablize GCC for their
> >>releases.
> >>
> >>For GDB, on the other hand, interesting development can and does get
> >>approved/committed at any time.  GDB snaps are of such quality that we
> >>can confidently refer someone to current sources for fixes (except
> >>when I have a bad day like today :-).  Further, instead of using
> >>official releases (and as you yourself have done) arbitrary snaps can
> >>even make their way into a distro.
> >
> >
> >The problem is, being that stable has a cost associated with it.  GCC
> >pays that cost at certain parts in their cycle; we pay that cost all
> >the time, every day.
> 
> GDB is less stable then you might think.  Right now while both:
> 
> - interps
> - frame
> 
> are causing problems they are not getting in the way of DavidC's dwarf2 
> stuff (gee wiz, both my doing :-/).  GDB always builds, gdb always 
> `break main; run'.  Is that too much to ask?

Of course not.  If someone breaks that, they (or we) fix it quickly. 
GCC's no different.

> The problem with GDB's stability is that allows people to quickly forget:
> 
> - what it is like with out it
> - how much gain there is from it
> - how relatively small the pain
> - how much more expensive it is to have to re-do something later
> - how, with a bit of peer revew, problematic code could have been done 
> right the first time (and how much that fallout costs).

I don't see where any of this is coming from.  As you point out above,
in a lot of respects GDB isn't all that stable.  What are we risking
here?

It also seems that Jim and I don't agree that the gain outweighs the
pain.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: [maint] The GDB maintenance process

[Jim Blandy]

I agree with what Daniel has said here.  I'm concerned that some
people have misunderstood his points, so I'll put it differently.

I think GDB could get better use of the contributors it has now by
adjusting the rules for patch approval.

- Slow patch review is a real problem on GDB --- even acknowledging
  the legitimate reasons for some delays that Elena has mentioned.

- It's true that "... some maintainers should try to review patches in
  their areas of responsibility more often", but merely saying so
  doesn't have any effect.  Folks have been saying that ever since
  Cygnus loosened its grip on GDB and the process opened to the
  public.  That statement seems to express a hope that the maintainers
  will somehow "wake up" and everything will get better.  It's been
  years, now, and we need to stop waiting for this to happen.  Let's
  work with the people we've got, rather than hoping they'll transform
  themselves somehow.

- If we accept that the maintainers' behavior is stable, then the next
  alternative is to adjust the organization that they operate under.
  Is there some way to re-organize the people we have, accepting their
  job committments and personal limitations (I have myself in mind
  here as much as anyone else, so don't be affronted), so that things
  progress better?  Is the current organization optimal?

Set that line of thinking aside, and consider another problem:

As far as I can tell, the Global Maintainers don't really have the
privileges that one would expect.  A Global Maintainer ought to be
someone who can make or approve a change anywhere in GDB.  This means,
in addition to having good technical judgement, that we trust them to
ask others as appropriate, and that we consider them good people to
debate with when a disagreement does come up.

This is different from the current definition --- as far as I can
tell, Global Maintainers aren't really that much different from anyone
else.  I've seen them criticized for approving changes, not because of
any specific problem with the changes, but because it was seen as
outside their purview.

I would personally be happy to see the current list of Global
Maintainers, of which I am a member, emptied out, and have people
re-establish their reputations to be added to the list, if it would
make folks more comfortable about straightening out the definition
this way.

So, to bring the two threads together: I think GDB could get better
use of the contributors it has now by relaxing the rules about who is
allowed to approve a patch.  With the redefinition of Global
Maintainer we're suggesting, more people would have the right to
review and approve a patch, so there are more chances someone will.

When I was maintaining Guile, I ended up replacing my dictatorship
with a group of four maintainers --- Mikael Djurfeldt, Maciej
Stachowiak, and Marius Vollmer.  Any one of those guys I trusted as
much as I trust myself.  There was no risk in making them my peers,
since I was just as likely to make a poor decision as they were (if
not more so), and if they didn't like something, it would certainly
benefit from a re-examination.  Surely we have people in the GDB
community in addition to Andrew who have earned that level of trust.


Just to add another idea to the mix:

I was talking to a friend of mine about the way GDB is run, and he was
amazed that we give individual people complete power, and complete
responsibility, for sections of code.  Everyone is going to be wrong
sometimes, he said, and it's easy to protect against, without being
too bureaucratic.

In the Apache system, which is used now in a lot of projects, they
have a pretty big group of global maintainers.  Any non-obvious core
change needs to be described on the list, and get three "+1" votes
from other global maintainers.  But if anyone posts a "-1", the change
is vetoed: it gets discussed for a longer period, and then put up for
a straight vote.

The nice thing here is that *nobody* has absolute power --- come on,
even the most talented among us is going to do something questionable
every so often, and would benefit from some more discussion --- but
it's still easy enough to get buy-in for most changes that it's
efficient.

But whether or not you like that idea, GDB has been operating under
basically the same system, and suffering the same problems, long
enough to confidently conclude that we, at least, are always going to
operate the way we do now under the system we have now.  This
delegated dictatiorship (with all due respect to our hard-working and
conscientious dictator) is more of an inherited legacy from ancient
days than a well-tuned process.  I think it's time to try to bring in
good ideas from other projects, like GCC and Apache.

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> - It's true that "... some maintainers should try to review patches in
>   their areas of responsibility more often", but merely saying so
>   doesn't have any effect.

For the record your name is top of the list of that `some maintainers'.

Andrew

Re: [maint] The GDB maintenance process

[Jim Blandy]

Andrew Cagney <ac131313@redhat.com> writes:

> > - It's true that "... some maintainers should try to review patches in
> >   their areas of responsibility more often", but merely saying so
> >   doesn't have any effect.
> 
> For the record your name is top of the list of that `some maintainers'.

If you like.  I pick on myself often enough elsewhere in that message
that it should be clear I'm not trying to make others feel bad for
shortcomings I have myself.

In fact, this is exactly what we need to put behind us: we can spend
another few years feeling bad for being insufficiently responsive, and
treating it as a personal failure.  But that approach hasn't improved
things noticeably, so, is there something else we can do?  Given the
people we have, the community we have, and their known strengths and
weaknesses, what is the best way to organize them?  Can we improve on
what we're doing now?

I think the explicit hierarchy we have now, outlined in MAINTAINERS,
is a real problem in this sense.  It's a big, public, political deal
to rearrange that hierarchy.  There are other systems where the
processes of promoting promising contributors and clearing dead wood
happen smoothly and automatically, without confrontation.  People
contribute as they are able, and leaders emerge and recede in a
natural way, not by fiat.  The Apache system, for example, encourages
newcomers to acquire expertise in different areas, and allows less
responsive people to simply fall to the wayside as irrelevant.

These systems are in widespread use, and I think some are even
well-documented, like: http://httpd.apache.org/dev/guidelines.html

Re: [maint] The GDB maintenance process

[Christopher Faylor]

Has anyone taken the time to go through the list of outstanding patches
and categorize them by maintainership?

I'm wondering if this is just a problem with a couple of maintainers
being unresponsive.

For instance, as a not-so-wild guess, I'd say that the symtab part of
gdb frequently falls behind.  That may be because there are more
symtab-related patches being supplied or it could be that I'm just
imagining things.

If there are some maintainers who just aren't responsive enough, for
whatever reason, then the simple expedient of adding another maintainer
for that specific part of gdb might go a long way towards fixing the
backlog.

However, if the patches are all over the board then this technique
probably won't work.

cgf

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> Has anyone taken the time to go through the list of outstanding patches
> and categorize them by maintainership?

Already done, see the bug database.   Mind you only very recently but I 
think we're already seeing results.

> I'm wondering if this is just a problem with a couple of maintainers
> being unresponsive.
> 
> For instance, as a not-so-wild guess, I'd say that the symtab part of
> gdb frequently falls behind.  That may be because there are more
> symtab-related patches being supplied or it could be that I'm just
> imagining things.

There are more symtab patches but, thanks to one maintainer, they are 
moving.  Not sure about other categories, some do appear stalled :-(

> If there are some maintainers who just aren't responsive enough, for
> whatever reason, then the simple expedient of adding another maintainer
> for that specific part of gdb might go a long way towards fixing the
> backlog.

There is responsive and then there is reliable :-/

Andrew

Re: [maint] The GDB maintenance process

[Elena Zannoni]

Jim Blandy writes:
 > Andrew Cagney <ac131313@redhat.com> writes:
 > 
 > > > - It's true that "... some maintainers should try to review patches in
 > > >   their areas of responsibility more often", but merely saying so
 > > >   doesn't have any effect.
 > > 
 > > For the record your name is top of the list of that `some maintainers'.
 > 
 > 
 > I think the explicit hierarchy we have now, outlined in MAINTAINERS,
 > is a real problem in this sense.  It's a big, public, political deal
 > to rearrange that hierarchy.  There are other systems where the
 > processes of promoting promising contributors and clearing dead wood
 > happen smoothly and automatically, without confrontation.  People
 > contribute as they are able, and leaders emerge and recede in a
 > natural way, not by fiat.  The Apache system, for example, encourages
 > newcomers to acquire expertise in different areas, and allows less
 > responsive people to simply fall to the wayside as irrelevant.
 > 
 > These systems are in widespread use, and I think some are even
 > well-documented, like: http://httpd.apache.org/dev/guidelines.html

"Membership as a Committer is by invitation only and must be approved
 by consensus of the active Apache PMC members. A Committer is
 considered inactive by their own declaration or by not contributing
 in any form to the project for over six months. An inactive member
 can become active again by reversing whichever condition made them
 inactive (i.e., by reversing their earlier declaration or by once
 again contributing toward the project's work). Membership can be
 revoked by a unanimous vote of all the active PMC members (except the
 member in question if they are a PMC member)."


How is being a member of the 'Committers' by "invitation only" not
going to be another political deal?

How is the 6 months inactivity period going to help? None of the
current gdb maintainers perticipating in this discussion has ever
disappeared for that long, really.  If some person has disappeared
from radar, like relocated, changed e-mail, etc, the person has been
removed from the list. I can think of one such case.


I don't like this one:

"Ideas must be review-then-commit; patches can be
 commit-then-review. With a commit-then-review process, we trust that
 the developer doing the commit has a high degree of confidence in the
 change."

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> How is the 6 months inactivity period going to help? None of the
> current gdb maintainers perticipating in this discussion has ever
> disappeared for that long, really.  If some person has disappeared
> from radar, like relocated, changed e-mail, etc, the person has been
> removed from the list. I can think of one such case.

There was one such case.  There have also been two dummy spits; and, 
after lobying, a number of people `stepping down'.

One thing I've noticed is that people struggle to appreciate the 
difference between `no maintainer' and `inactive maintainer'.  There is 
a definite resistance to `letting go'.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I agree with what Daniel has said here.  I'm concerned that some
> people have misunderstood his points, so I'll put it differently.
> 
> I think GDB could get better use of the contributors it has now by
> adjusting the rules for patch approval.
> 
> - Slow patch review is a real problem on GDB --- even acknowledging
>   the legitimate reasons for some delays that Elena has mentioned.

I found myself babysitting a short list of largely dormant maintainers 
using regular pings.  That was grosely inefficient.  I've now given that 
one up and have moved on to filing unreviewed patches in the bug 
database - worse for the contributor waiting on a dormant maintainer but 
better for me and the other active maintainers.

It has been in place for ~a month and, in my opinion, has definitly 
improved things.  There is a one-stop-shop where active [global] 
maintainers can either find a backlog of patches or areas that need work.

I just wish that the tool being used was less primative :-(

> - It's true that "... some maintainers should try to review patches in
>   their areas of responsibility more often", but merely saying so
>   doesn't have any effect.  Folks have been saying that ever since
>   Cygnus loosened its grip on GDB and the process opened to the
>   public.  That statement seems to express a hope that the maintainers
>   will somehow "wake up" and everything will get better.  It's been
>   years, now, and we need to stop waiting for this to happen.  Let's
>   work with the people we've got, rather than hoping they'll transform
>   themselves somehow.

And the vast majority of maintainers do a pretty good job.

> - If we accept that the maintainers' behavior is stable, then the next
>   alternative is to adjust the organization that they operate under.
>   Is there some way to re-organize the people we have, accepting their
>   job committments and personal limitations (I have myself in mind
>   here as much as anyone else, so don't be affronted), so that things
>   progress better?  Is the current organization optimal?

Stable or reliable?  Most maintainers are reliable, a small few, though, 
seem to keep falling asleep at the wheel.

Having such people as developers isn't a problem.  Having such people 
hold key responsabilities (such as being in the loop to review patches) is.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I was talking to a friend of mine about the way GDB is run, and he was
> amazed that we give individual people complete power, and complete
> responsibility, for sections of code.  Everyone is going to be wrong
> sometimes, he said, and it's easy to protect against, without being
> too bureaucratic.

Interesting spin.

I think the concept such as `the Buck stops' and having the final 
decision, are very different to `complete power'.   The former is about 
responsibility.

A maintainer, who is a member of a larger team, takes on certain 
responsibilities, such making the final decision when reviewing a patch. 
  The maintainer is assumed to have sufficiently good judgment, however, 
they are also assumed to be taking on board the comments and suggestions 
being made by their peers.

Andrew

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> When I was maintaining Guile, I ended up replacing my dictatorship
> with a group of four maintainers --- Mikael Djurfeldt, Maciej
> Stachowiak, and Marius Vollmer.  Any one of those guys I trusted as
> much as I trust myself.  There was no risk in making them my peers,
> since I was just as likely to make a poor decision as they were (if
> not more so), and if they didn't like something, it would certainly
> benefit from a re-examination.  Surely we have people in the GDB
> community in addition to Andrew who have earned that level of trust.

The global write maintainers already have this responsability.  Every so 
often there needs to be a `judgment call'(1) which falls to the gdb head 
(again `the buck stops').  On the whole, though, the group should be 
discusses technical issues selecting designs based solely on their 
relative merits.

One thing I do do though, is nip debates that are re-flogging head 
issues, in the bud.  Proposals for making GDB LGPL, for instance.

Andrew

(1) The last one I remember making was adopting gdb/cli/cli-*.[hc] for 
cli file names and for that one I tossed a coin.

Re: [maint] The GDB maintenance process

[David Carlton]

Here is some free-associating that I have about GDB's approval
process.  I'm not sure yet what I think about various concrete
proposals out there; I would like to see more existing models
discussed, because it seems unlikely to me that GDB's current model is
optimal (though, of course, what is?), and because recent history
seems to be full of interesting and potentially effective software
development models, models that I wish I knew more about.

* Overall, I'm quite happy with my experience working with GDB.  On
  the other hand, I suspect that various idiosyncracies in my
  situation make me a bit of an outlier in terms of ability and
  willingness to stick through the process.  It does seem to me that
  there have been other people's patches that would be valuable but
  that have died through lack of timely follow-through (whether by the
  approvers, by the submitters, or both, not to mention issues like
  copyright approval that we're unlikely to change).  The more people
  who are willing and allowed to help newcomers through this process,
  the better GDB will be.

* Sometimes the division of responsibility seems to me a bit strange.
  If I were a C++ maintainer, for example, it might drive me a bit
  bonkers having to wait for other people's approval for patches that
  only affect C++ but that are located in code that is in somebody
  else's domain.  (I was impressed with the way Daniel got the
  'demangling partial symbols' patch through.)

* The most serious strangeness of division of responsibility is having
  separate maintainers for the testsuite whose approval is required.
  We should do whatever we can to encourage people to write tests,
  whether to test new features or to fill in gaps in existing
  features; the more people who can approve tests, the better.  (I
  might be imagining things, but it seems to me like people have been
  frequently adding tests without waiting for approval recently; this
  seems like a good thing to me.)  It seems to me that, at the very
  least, anybody who can approve patches in a specific area should
  also be able to commit patches for tests in that area.  (Though of
  course it's good to, in addition, allow people to approve testsuite
  patches even if they can't approve non-testsuite patches: like I
  said, the more tests, the better.)

* Speaking of tests, the better the testsuite is, the less likely that
  patches will break things.  (Some currently popular development
  methodologies are rooted on this idea.)  On the other hand, there
  are good reasons (the history of GDB, its architecture, the test
  suite's architecture, and the nature of debuggers) that the
  testsuite's coverage will never be perfect, setting aside the issue
  that "not breaking behavior" isn't the same as "good".

* I have been impressed with GDB's stability.  Of course, it doesn't
  hurt that I'm using GDB on i686-pc-gnu-linux.  (Or that, frankly,
  I'm not a very heavy GDB user.)

* By temperament and programming style, I prefer to break up my
  patches into lots of small changes whose correctness should be
  relatively easy to verify.  Unfortunately, this means that it takes
  a long time for the sum of my changes to be approved.  For example,
  it's been more than 3 months since I posted a version of linespec.c
  that I think most people would agree was a significant improvement
  to its predecessor; I've submitted 14 patches to that file so far,
  with approximately an equal number to come.  I actually rather enjoy
  that process when it's going well - I log in each day, read my
  email, get an approval message for my latest patch, take a bit of
  time to generate and test my next patch, and then move on to
  something else - but it gets annoying when I have to wait for
  approval for simple, obviously correct patches.

  I've had a similar experience with cleaning up lookup_symbol_aux:
  there are bugs in that code, and I have patches to fix them, but I'm
  not sure when they'll make it into GDB.  I sigh with envy as I watch
  Andrew committing simple frame patch after simple frame patch
  without needing anybody else's approval.  A series of small patches
  like that can be a wonderful way to cleanse code.

* Continuing my last bullet point, however, it's not obvious to me
  that the changes in the approval process that Daniel suggests would
  help.  After all, there are already 3 people who can approve patches
  to linespec.c (so I'm certainly not mad at Elena - she's the only
  one taking time to actually approve the patches in question!), and I
  don't see why increasing that number would decrease the approval
  time.  So the obvious place to look for improvement is for me to
  change my behavior; I have some thoughts along those lines that I'm
  going to try to implement over the next few weeks.

* Having patches in the bug database is probably a good idea, but it
  means that somebody has to take the effort to put patches in the bug
  database.  It was great when Andrew cleaned out his mailbox and
  dumped it into the bug database; I'm not optimistic about the bug
  database working well for this in the longer term.

* Increasing the number of people who are familiar with a given
  segment of GDB's code can only improve GDB's health.  The current
  approval process seems to me to work against this, to some extent.

David Carlton
carlton@math.stanford.edu

Re: [maint] The GDB maintenance process

[Mark Kettenis]

I mostly agree with Daniel here.  We have too many single points of
failure.  I still have testsuite patches sitting in my tree, dating
months back since they were never approved.  Similarly, Daniel
probably has been held back more than once since I wasn't able to
review threads-related patches in a timely fashion.  I think we should
allow our global maintainers to approve patches even for parts of GDB
where we have a specific maintainer, if they feel they have the
necessary expertise.

Mark

Re: [maint] The GDB maintenance process

[Andrew Cagney]

> I mostly agree with Daniel here.  We have too many single points of
> failure.  I still have testsuite patches sitting in my tree, dating
> months back since they were never approved.  Similarly, Daniel
> probably has been held back more than once since I wasn't able to
> review threads-related patches in a timely fashion.  I think we should
> allow our global maintainers to approve patches even for parts of GDB
> where we have a specific maintainer, if they feel they have the
> necessary expertise.

Depends on what `single point failure' means.  If you look through the 
MAINTAINERS file you'll notice that all but one of the problem areas is 
double, if not triple maintained.  Even with all that dedundency, 
patches in certain areas continues to stall.  While there might be three 
maintainers, the reality is only one or none are reliable.

The first, and most obvious thing is for the global maintainers to 
review their current number of responsibilities and compare that against 
their actual level of commmitment.  I'm down to just target/arch, 
remote, mips and sim/ppc.

The second, is for maintainers to always be on the lookout for potential 
new maintainers (global or not) and, where possible, step aside to let 
new commers in.

Finally, the GDB `tradition' (that appears to have been forgotten) is to 
up front reach consensus on stuff so that the final patches are trivial 
to review.  cf, the block.[hc] change.

enjoy,
Andrew

Re: [maint] The GDB maintenance process

[Kevin Buettner]

On Feb 17,  1:07pm, Daniel Jacobowitz wrote:

> Right now, we use stricter policies to prevent problems which cause
> breakage.  I think these policies are stifling us.  Loosening them (and
> maybe adding a formal patch reversion policy) would let more people fix
> problems more easily, as they arise, without slowing development.

I agree.

Kevin

Unambiguously specifying source locations

[Daniel Jacobowitz]

This isn't a proposal - I haven't clearly thought out the details - just
some ramblings on a problem.

I would like to have a way to clearly identify a location, not by address. 
Primarily this would be for breakpoints; when we re-read an objfile's
symbols, we need to replace breakpoints somehow, and the more of the user's
intent we can preserve the better.  Obviously this is not a perfectly
solvable problem, but we can do pretty well.

It's an important problem, too.  Not only is recompile/reload a pretty
common thing to do, but IDEs which save breakpoints across sessions would
use this also.  Et cetera.

I think it's only source locations that I need to identify in this way, in
the short term, but we should use an extensible syntax.  For a
way-beyond-current-state example, consider an inlined function with multiple
inlined instances.  One of the simpler things we could do is:

  record the number of instances in this objfile before reloading
  check the number of instances in this objfile after reloading
  if same, then we can preserve things like the breakpoint enabled state
    to the new set
  if not, punt - enable all?  warn?

The goal there being to preserve the status of particular breakpoints across
reload, as best as we can, when unrelated changes are made to the source.
Someone fixes emit_foo () and recompiles and we try not to disturb their
eight disabled and one enabled breakpoint on inlined copies of obstack_alloc
where they were tracking the allocation of the bad object.

We could be much more thorough.  Try this on for size:
  Record the source location of the emitted function which contains each
    inlined instance, as unambiguously as we can.
  Record the inlining path.

The result would look something like (I don't like this syntax, just
illustrating):
  [libfoo.so.2][foo-1.cc:foo_func:75][bar-1.h:inline_one:33]\
    [bar-1.h:inline_two:36]
Then after reload if the path still leads to an inlined copy of inline_two
we can re-establish the breakpoint.  Or there are various fuzzy matching
things we could do.  Et cetera.

Generating such paths would be useful for output anyway; and if we can do
them in such a way as to accept them for input too, that would be useful to
our users.  We could also accept as input ambiguous paths and do basically
wildcarding:
  break [libfoo.so.2][bar-1.h]inline_two

Wouldn't that be nice?

The use of braces is not entirely coincidental.  decode_line_1 currently
does not accept anything that starts with a '[' as far as I can see; ObjC
selectors always have +[ or -[.  Using braces simplifies quoting and parsing
quite a bit.  And it could be extended as necessary without too much
trouble.

There are other problems: for instance, we might want to use linker names
for non-inlined functions where possible, for GDB-generated location
descriptions; that would handle keeping track of which constructor we were
stopped on.  Otherwise we'd need some other method for that.

Another nice thing to have might be an element in that list describing
C++ template instantiation.  Not sure if that's necessary or should just be
added to the function name as needed.

Include paths may also be needed.  This happens in C, with static functions
in headers.  Here's an example that bites me all the time:
  [libbfd.so][elf32-i386.c][elflink.h:2300]
  [libbfd.so][elf32-sparc.c][elflink.h:2300]
or maybe it should be:
  [libbfd.so][elf32-i386.c:elflink.h:2300]
  [libbfd.so][elf32-sparc.c:elflink.h:2300]
The latter, I think.

Right now the equivalent to this is handled in build_canonical_line_spec.
It's a little simple-minded; there are plenty of cases it doesn't handle.
It would be nice to do better.

Any specification of this should be explicit about quoting rules, comma,
dammit.

What other thoughts do you all have?  Am I on the right track, should we
draw up a formal specification for these?

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Unambiguously specifying source locations

[David Ayers]

Daniel Jacobowitz wrote:

>The use of braces is not entirely coincidental.  decode_line_1 currently
>does not accept anything that starts with a '[' as far as I can see; ObjC
>selectors always have +[ or -[.  
>
Actually this is not the case.  If the +/- is omitted both are offered 
as breakpoints if they exist:

(gdb) b [NSObject autorelease]
[0] cancel
[1] all
[2] -[NSObject autorelease] at NSObject.m:1553
[3] +[NSObject autorelease] at NSObject.m:1575
 >

But maybe you could rely on the fact that there must be a white space 
between the class and the message.

Cheers,
David Ayers

Re: Unambiguously specifying source locations

[Daniel Jacobowitz]

On Fri, Oct 10, 2003 at 05:39:34PM +0200, David Ayers wrote:
> Daniel Jacobowitz wrote:
> 
> >The use of braces is not entirely coincidental.  decode_line_1 currently
> >does not accept anything that starts with a '[' as far as I can see; ObjC
> >selectors always have +[ or -[.  
> >
> Actually this is not the case.  If the +/- is omitted both are offered 
> as breakpoints if they exist:
> 
> (gdb) b [NSObject autorelease]
> [0] cancel
> [1] all
> [2] -[NSObject autorelease] at NSObject.m:1553
> [3] +[NSObject autorelease] at NSObject.m:1575
> >

Hmm, thank you.  I don't quite see how that happens so I'll have to go
peer at linespec.c again.

> But maybe you could rely on the fact that there must be a white space 
> between the class and the message.

I'd rather not - there can be whitespace in file names, shared object
names, et cetera.  Perhaps something different is in order.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Unambiguously specifying source locations

[Felix Lee]

>  [libfoo.so.2][foo-1.cc:foo_func:75][bar-1.h:inline_one:33]\
>    [bar-1.h:inline_two:36]

One thing I'm not clear about, you're mixing source locators with
object locators.  Is it really a path?  Maybe it should be
an arbitrary qualifier list instead?  Like
    foofunc[obj libfoo.so]
    foofunc[src foo.h in foo1.cc]
the first would be all foofunc in libfoo.so. the second would be
all foofunc the compiler generates from foo.cc/foo.h.  and
perhaps let the user combine qualifiers with boolean operators.
--

Re: Unambiguously specifying source locations

[Jim Ingham]

So one corollary that makes this easier is that instead of holding onto 
the requested shared library (and requested file for static functions) 
in the address string, we should hold them in the breakpoint structure 
itself.  So then we don't have to worry about what the canonical form 
is.  That too becomes simple, it is:

b->requested_filename = "myFile.c"
b->requested_shlib = "myShlib.dylib"

And if there are any others, we can add them.  So we only have to break 
out the specifications when the user instructs us to set the 
breakpoint.  After that all the bits are clear.

I started playing around with this for dylibs for purposes of my own... 
  I called them requested_* because I wanted to make it clear that these 
were not where we found the thing, but where the user asked for the 
thing.  It is also interesting where the breakpoint eventually landed, 
but that bit should be in the "implementation" side of the breakpoint.  
This clearly belongs in the user side...

Jim


On Oct 14, 2003, at 4:06 PM, gdb-digest-help@sources.redhat.com wrote:

>
>
> On Mon, Oct 13, 2003 at 10:25:20AM -0700, Jim Ingham wrote:
>> I think the intent here is great!
>>
>> I have a heartfelt plea, however, from one who while not as
>> battle-scarred as some others, have waded my way through plenty of
>> decode_line_1 bugs...
>>
>> Is there any way we can not have to keep overloading the expression
>> parser with more specifications?  It seems to me this is just a way to
>> obfuscate the user's intent so that we can get it wrong trying to
>> decode it later.  Daniel's proposed syntax - no offense intended - is
>> sufficiently awful that it should give us pause.  Would:
>>
>> break -shlib foo.dylib -file foo.c MyStaticFunction
>>
>> be all that awful?  This is unambiguous, represents the user's intent
>> exactly, is not too hard to type, and trivial to parse.  Then
>> internally, the breakpoint could actually hold all these separate bits
>> separately, rather than munging them into a canonical form which we 
>> can
>> trip over later on...
>>
>> We will probably have to support the specifications that we do now for
>> ever - though adding switches for them would allow unambiguous entry
>> and would probably be taken up by a good number of users cause it is
>> almost impossible to get wrong...
>
> Feel free to propose a better canonical form :)  You basically just
> did, above.  We need a canonical representation, for instance:
>   - We use it internally to re-place breakpoints after rereading an
>     objfile.
>   - We would like to be able to display it so that breakpoints can be
>     saved and reloaded.
>
> I guess the question is whether these are useful for anything other
> than specifying breakpoint (or tracepoint) locations.  If not then
> flags to break might be canonical enough.
>
> -- 
> Daniel Jacobowitz
> MontaVista Software                         Debian GNU/Linux Developer
>
>
--
Jim Ingham                                   jingham@apple.com
Developer Tools
Apple Computer

Re: Unambiguously specifying source locations

[Jim Ingham]

I think the intent here is great!

I have a heartfelt plea, however, from one who while not as 
battle-scarred as some others, have waded my way through plenty of 
decode_line_1 bugs...

Is there any way we can not have to keep overloading the expression 
parser with more specifications?  It seems to me this is just a way to 
obfuscate the user's intent so that we can get it wrong trying to 
decode it later.  Daniel's proposed syntax - no offense intended - is 
sufficiently awful that it should give us pause.  Would:

break -shlib foo.dylib -file foo.c MyStaticFunction

be all that awful?  This is unambiguous, represents the user's intent 
exactly, is not too hard to type, and trivial to parse.  Then 
internally, the breakpoint could actually hold all these separate bits 
separately, rather than munging them into a canonical form which we can 
trip over later on...

We will probably have to support the specifications that we do now for 
ever - though adding switches for them would allow unambiguous entry 
and would probably be taken up by a good number of users cause it is 
almost impossible to get wrong...

Jim

On Oct 11, 2003, at 5:32 AM, gdb-digest-help@sources.redhat.com wrote:

> This isn't a proposal - I haven't clearly thought out the details - 
> just
> some ramblings on a problem.
>
> I would like to have a way to clearly identify a location, not by 
> address.
> Primarily this would be for breakpoints; when we re-read an objfile's
> symbols, we need to replace breakpoints somehow, and the more of the 
> user's
> intent we can preserve the better.  Obviously this is not a perfectly
> solvable problem, but we can do pretty well.
>
> It's an important problem, too.  Not only is recompile/reload a pretty
> common thing to do, but IDEs which save breakpoints across sessions 
> would
> use this also.  Et cetera.
>
> I think it's only source locations that I need to identify in this 
> way, in
> the short term, but we should use an extensible syntax.  For a
> way-beyond-current-state example, consider an inlined function with 
> multiple
> inlined instances.  One of the simpler things we could do is:
>
>   record the number of instances in this objfile before reloading
>   check the number of instances in this objfile after reloading
>   if same, then we can preserve things like the breakpoint enabled 
> state
>     to the new set
>   if not, punt - enable all?  warn?
>
> The goal there being to preserve the status of particular breakpoints 
> across
> reload, as best as we can, when unrelated changes are made to the 
> source.
> Someone fixes emit_foo () and recompiles and we try not to disturb 
> their
> eight disabled and one enabled breakpoint on inlined copies of 
> obstack_alloc
> where they were tracking the allocation of the bad object.
>
> We could be much more thorough.  Try this on for size:
>   Record the source location of the emitted function which contains 
> each
>     inlined instance, as unambiguously as we can.
>   Record the inlining path.
>
> The result would look something like (I don't like this syntax, just
> illustrating):
>   [libfoo.so.2][foo-1.cc:foo_func:75][bar-1.h:inline_one:33]\
>     [bar-1.h:inline_two:36]
> Then after reload if the path still leads to an inlined copy of 
> inline_two
> we can re-establish the breakpoint.  Or there are various fuzzy 
> matching
> things we could do.  Et cetera.
>
> Generating such paths would be useful for output anyway; and if we can 
> do
> them in such a way as to accept them for input too, that would be 
> useful to
> our users.  We could also accept as input ambiguous paths and do 
> basically
> wildcarding:
>   break [libfoo.so.2][bar-1.h]inline_two
>
> Wouldn't that be nice?
>
> The use of braces is not entirely coincidental.  decode_line_1 
> currently
> does not accept anything that starts with a '[' as far as I can see; 
> ObjC
> selectors always have +[ or -[.  Using braces simplifies quoting and 
> parsing
> quite a bit.  And it could be extended as necessary without too much
> trouble.
>
> There are other problems: for instance, we might want to use linker 
> names
> for non-inlined functions where possible, for GDB-generated location
> descriptions; that would handle keeping track of which constructor we 
> were
> stopped on.  Otherwise we'd need some other method for that.
>
> Another nice thing to have might be an element in that list describing
> C++ template instantiation.  Not sure if that's necessary or should 
> just be
> added to the function name as needed.
>
> Include paths may also be needed.  This happens in C, with static 
> functions
> in headers.  Here's an example that bites me all the time:
>   [libbfd.so][elf32-i386.c][elflink.h:2300]
>   [libbfd.so][elf32-sparc.c][elflink.h:2300]
> or maybe it should be:
>   [libbfd.so][elf32-i386.c:elflink.h:2300]
>   [libbfd.so][elf32-sparc.c:elflink.h:2300]
> The latter, I think.
>
> Right now the equivalent to this is handled in 
> build_canonical_line_spec.
> It's a little simple-minded; there are plenty of cases it doesn't 
> handle.
> It would be nice to do better.
>
> Any specification of this should be explicit about quoting rules, 
> comma,
> dammit.
>
> What other thoughts do you all have?  Am I on the right track, should 
> we
> draw up a formal specification for these?
>
> -- 
> Daniel Jacobowitz
> MontaVista Software                         Debian GNU/Linux Developer
>
--
Jim Ingham                                   jingham@apple.com
Developer Tools
Apple Computer

Re: Unambiguously specifying source locations

[Daniel Jacobowitz]

On Mon, Oct 13, 2003 at 10:25:20AM -0700, Jim Ingham wrote:
> I think the intent here is great!
> 
> I have a heartfelt plea, however, from one who while not as 
> battle-scarred as some others, have waded my way through plenty of 
> decode_line_1 bugs...
> 
> Is there any way we can not have to keep overloading the expression 
> parser with more specifications?  It seems to me this is just a way to 
> obfuscate the user's intent so that we can get it wrong trying to 
> decode it later.  Daniel's proposed syntax - no offense intended - is 
> sufficiently awful that it should give us pause.  Would:
> 
> break -shlib foo.dylib -file foo.c MyStaticFunction
> 
> be all that awful?  This is unambiguous, represents the user's intent 
> exactly, is not too hard to type, and trivial to parse.  Then 
> internally, the breakpoint could actually hold all these separate bits 
> separately, rather than munging them into a canonical form which we can 
> trip over later on...
> 
> We will probably have to support the specifications that we do now for 
> ever - though adding switches for them would allow unambiguous entry 
> and would probably be taken up by a good number of users cause it is 
> almost impossible to get wrong...

Feel free to propose a better canonical form :)  You basically just
did, above.  We need a canonical representation, for instance:
  - We use it internally to re-place breakpoints after rereading an
    objfile.
  - We would like to be able to display it so that breakpoints can be
    saved and reloaded.

I guess the question is whether these are useful for anything other
than specifying breakpoint (or tracepoint) locations.  If not then
flags to break might be canonical enough.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[James Cownie]

> > But what is stopping us picking up that code, compiling it on the host 
> > (not target), and then using it in GDB?
> 
> Version skew primarily. 

Can you not choose the right version of libthread_db at runtime based
on the version of linuxthreads and dlopen it in gdb ?

Ideally there'd be some version information in the thread library code
which gdb could extract in a specified manner, and then a convention
about how that maps to a the name of a suitable .so version of
libthread_db.

This lets you

1) use libthread_db inside gdb on the host machine
2) keep gdb insulated from the version dependency on the thread
   library.

-- Jim 

James Cownie	<jcownie@etnus.com>
Etnus, LLC.     +44 117 9071438
http://www.etnus.com

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Tue, Jun 25, 2002 at 09:48:12AM +0100, James Cownie wrote:
> > > But what is stopping us picking up that code, compiling it on the host 
> > > (not target), and then using it in GDB?
> > 
> > Version skew primarily. 
> 
> Can you not choose the right version of libthread_db at runtime based
> on the version of linuxthreads and dlopen it in gdb ?
> 
> Ideally there'd be some version information in the thread library code
> which gdb could extract in a specified manner, and then a convention
> about how that maps to a the name of a suitable .so version of
> libthread_db.

But there isn't this version information.  We could probably detect the
glibc version number, but these are internal structures, not
interfaces; detecting when they have changed is quite hard.

> This lets you
> 
> 1) use libthread_db inside gdb on the host machine
> 2) keep gdb insulated from the version dependency on the thread
>    library.

Nope.  We'd have to be the ones writing such a thread_db; there is no
version that can run on the host machine at present.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[James Cownie]

Daniel Jacobowitz wrote :-

> But there isn't this version information.  We could probably detect the
> glibc version number, but these are internal structures, not
> interfaces; detecting when they have changed is quite hard.

So, ask the thread library folks to provide a suitable versioning
symbol. As you rightly point out trying to guess the version is
unlikely to succeed !

> Nope.  We'd have to be the ones writing such a thread_db; there is no
> version that can run on the host machine at present.

However, having a version on the host had already been posited by
someone else. I didn't make that part of the proposal.

I'm merely suggesting a way around the version skew which you cited as
the primary problem.

Since libthread_db is always dlopened anyway, 

( http://sources.redhat.com/ml/libc-alpha/2000-06/msg00048.html )

it seems to me that you already need a good version symbol, or else
how can you handle programs which are using different versions on libc
and associated different versions of the threads library ?

-- Jim 

James Cownie	<jcownie@etnus.com>
Etnus, LLC.     +44 117 9071438
http://www.etnus.com

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Tue, Jun 25, 2002 at 04:31:15PM +0100, James Cownie wrote:
> 
> Daniel Jacobowitz wrote :-
> 
> > But there isn't this version information.  We could probably detect the
> > glibc version number, but these are internal structures, not
> > interfaces; detecting when they have changed is quite hard.
> 
> So, ask the thread library folks to provide a suitable versioning
> symbol. As you rightly point out trying to guess the version is
> unlikely to succeed !

Anything that requires changes to the library isn't really viable; too
many libraries already exist.

> > Nope.  We'd have to be the ones writing such a thread_db; there is no
> > version that can run on the host machine at present.
> 
> However, having a version on the host had already been posited by
> someone else. I didn't make that part of the proposal.
> 
> I'm merely suggesting a way around the version skew which you cited as
> the primary problem.
> 
> Since libthread_db is always dlopened anyway, 
> 
> ( http://sources.redhat.com/ml/libc-alpha/2000-06/msg00048.html )
> 
> it seems to me that you already need a good version symbol, or else
> how can you handle programs which are using different versions on libc
> and associated different versions of the threads library ?

We can't.  It is always dlopened on the host when debugging natively
and on the target when debugging remotely, but that's it.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[James Cownie]

> > So, ask the thread library folks to provide a suitable versioning
> > symbol. As you rightly point out trying to guess the version is
> > unlikely to succeed !
> 
> Anything that requires changes to the library isn't really viable; too
> many libraries already exist.

But that seems like an argument for packing up and going for a beer
which could be applied to _any_ improvement in anything associated
with libc :-)

The exact same argument could have been employed against introducing
libthread_db itself in the first place !

Surely one has to move forwards.

Adding a suitable version symbol can only help, it can't break
anything. If it's there the debugger can use it. If not you can keep
doing what you already do, and it seems you really need it even for
the normal native cases.

-- Jim 

James Cownie	<jcownie@etnus.com>
Etnus, LLC.     +44 117 9071438
http://www.etnus.com

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Tue, Jun 25, 2002 at 04:53:37PM +0100, James Cownie wrote:
> > > So, ask the thread library folks to provide a suitable versioning
> > > symbol. As you rightly point out trying to guess the version is
> > > unlikely to succeed !
> > 
> > Anything that requires changes to the library isn't really viable; too
> > many libraries already exist.
> 
> But that seems like an argument for packing up and going for a beer
> which could be applied to _any_ improvement in anything associated
> with libc :-)
> 
> The exact same argument could have been employed against introducing
> libthread_db itself in the first place !
> 
> Surely one has to move forwards.
> 
> Adding a suitable version symbol can only help, it can't break
> anything. If it's there the debugger can use it. If not you can keep
> doing what you already do, and it seems you really need it even for
> the normal native cases.

You'll never get them to document interface changes in pthreads'
internal data structures.  You're welcome to try if you like pain more
than I do.  The only layer the glibc folks support for accessing this
data is thread_db.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[James Cownie]

> You'll never get them to document interface changes in pthreads'
> internal data structures.  You're welcome to try if you like pain
> more than I do.  The only layer the glibc folks support for
> accessing this data is thread_db.

I'm not sugggesting that they document it, rather that they provide a
version symbol which tells you which version of the pthreads code you
have, and then name libthread_db with a name which includes the
version number.

All I'm after is that one can tell from a debugger which version of
the pthread library one is looking at and then deduce which
libthread_db.so to dlopen.

Something as trivial as

static volatile int __pthread_version_number = 123;

somwehere would probably do. 

The documentation requirement would be

  The static symbol __pthread_version_number gives the version number
 of the pthread library to allow a debugger to locate the appropriate
 version of libthread_db.

and that's it. It doesn't change...

Thought they need to remember to change the version number when they
make changes which affect libthread_db.

-- Jim 

James Cownie	<jcownie@etnus.com>
Etnus, LLC.     +44 117 9071438
http://www.etnus.com

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Tue, Jun 25, 2002 at 05:11:39PM +0100, James Cownie wrote:
> 
> > You'll never get them to document interface changes in pthreads'
> > internal data structures.  You're welcome to try if you like pain
> > more than I do.  The only layer the glibc folks support for
> > accessing this data is thread_db.
> 
> I'm not sugggesting that they document it, rather that they provide a
> version symbol which tells you which version of the pthreads code you
> have, and then name libthread_db with a name which includes the
> version number.
> 
> All I'm after is that one can tell from a debugger which version of
> the pthread library one is looking at and then deduce which
> libthread_db.so to dlopen.
> 
> Something as trivial as
> 
> static volatile int __pthread_version_number = 123;
> 
> somwehere would probably do. 
> 
> The documentation requirement would be
> 
>   The static symbol __pthread_version_number gives the version number
>  of the pthread library to allow a debugger to locate the appropriate
>  version of libthread_db.
> 
> and that's it. It doesn't change...
> 
> Thought they need to remember to change the version number when they
> make changes which affect libthread_db.

That's what I meant by document.  If you look at libthread_db, you'll
see that it includes the internal header which describes the layout of
a thread descriptor.  Changes to that structure are not counted and I
don't think they'll like the idea.  They're also hard to identify,
since it draws types from other headers.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Andrew Cagney]

> Adding a suitable version symbol can only help, it can't break
>> anything. If it's there the debugger can use it. If not you can keep
>> doing what you already do, and it seems you really need it even for
>> the normal native cases.
> 
> 
> You'll never get them to document interface changes in pthreads'
> internal data structures.  You're welcome to try if you like pain more
> than I do.  The only layer the glibc folks support for accessing this
> data is thread_db.

Does the current thread-db code at least sanity check version match 
internally?

Andrew

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Tue, Jun 25, 2002 at 12:18:34PM -0400, Andrew Cagney wrote:
> >Adding a suitable version symbol can only help, it can't break
> >>anything. If it's there the debugger can use it. If not you can keep
> >>doing what you already do, and it seems you really need it even for
> >>the normal native cases.
> >
> >
> >You'll never get them to document interface changes in pthreads'
> >internal data structures.  You're welcome to try if you like pain more
> >than I do.  The only layer the glibc folks support for accessing this
> >data is thread_db.
> 
> Does the current thread-db code at least sanity check version match 
> internally?

I don't believe it has any mechanism for this, no.

-- 
Daniel Jacobowitz                           Carnegie Mellon University
MontaVista Software                         Debian GNU/Linux Developer

Re: GDB support for thread-local storage

[Jim Blandy]

Daniel Jacobowitz <drow@mvista.com> writes:
> On Tue, Jun 25, 2002 at 12:18:34PM -0400, Andrew Cagney wrote:
> > >Adding a suitable version symbol can only help, it can't break
> > >>anything. If it's there the debugger can use it. If not you can keep
> > >>doing what you already do, and it seems you really need it even for
> > >>the normal native cases.
> > >
> > >
> > >You'll never get them to document interface changes in pthreads'
> > >internal data structures.  You're welcome to try if you like pain more
> > >than I do.  The only layer the glibc folks support for accessing this
> > >data is thread_db.
> > 
> > Does the current thread-db code at least sanity check version match 
> > internally?
> 
> I don't believe it has any mechanism for this, no.

Doesn't libthread_db read `struct _pthread_descr_struct' from the
linuxthreads library in the inferior, rather than knowing the layout
itself?  So it's actually the inferior's linuxthreads library that
describes its own structures' layout.

It's got *some* defense against version skew.

Re: GDB support for thread-local storage

[Andrew Cagney]

> Daniel Jacobowitz <drow@mvista.com> writes:
> 
>> On Tue, Jun 25, 2002 at 12:18:34PM -0400, Andrew Cagney wrote:
> 
>> > >Adding a suitable version symbol can only help, it can't break
> 
>> > >>anything. If it's there the debugger can use it. If not you can keep
>> > >>doing what you already do, and it seems you really need it even for
>> > >>the normal native cases.
> 
>> > >
>> > >
>> > >You'll never get them to document interface changes in pthreads'
>> > >internal data structures.  You're welcome to try if you like pain more
>> > >than I do.  The only layer the glibc folks support for accessing this
>> > >data is thread_db.
> 
>> > 
>> > Does the current thread-db code at least sanity check version match 
>> > internally?
> 
>> 
>> I don't believe it has any mechanism for this, no.
> 
> 
> Doesn't libthread_db read `struct _pthread_descr_struct' from the
> linuxthreads library in the inferior, rather than knowing the layout
> itself?  So it's actually the inferior's linuxthreads library that
> describes its own structures' layout.

The structure layout will have been compiled into libthread-db.a.  It 
implicitly knows the layout itself.

> It's got *some* defense against version skew.

To defend against version skew, libthread-db needs to check that its 
internals are consistent with the target.  A simple variable value check 
was mentioned.

Andrew

Re: GDB support for thread-local storage

[Jim Blandy]

Andrew Cagney <ac131313@ges.redhat.com> writes:
> > Doesn't libthread_db read `struct _pthread_descr_struct' from the
> > linuxthreads library in the inferior, rather than knowing the layout
> > itself?  So it's actually the inferior's linuxthreads library that
> > describes its own structures' layout.
> 
> The structure layout will have been compiled into libthread-db.a.  It
> implicitly knows the layout itself.

I think you're wrong here.  Could you show me the code?

(Not that the question is especially relevant to the discussion --- of
course it could do a simple checksum check.)

Re: GDB support for thread-local storage

[Andrew Cagney]

> Andrew Cagney <ac131313@ges.redhat.com> writes:
> 
>> > Doesn't libthread_db read `struct _pthread_descr_struct' from the
>> > linuxthreads library in the inferior, rather than knowing the layout
>> > itself?  So it's actually the inferior's linuxthreads library that
>> > describes its own structures' layout.
> 
>> 
>> The structure layout will have been compiled into libthread-db.a.  It
>> implicitly knows the layout itself.
> 
> 
> I think you're wrong here.  Could you show me the code?

Sorry, you've lost me.

http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/linuxthreads/descr.h?rev=1.3&content-type=text/x-cvsweb-markup&cvsroot=glibc
> struct _pthread_descr_struct {
>   /* XXX Remove this union for IA-64 style TLS module */
>   union {
>     struct {
>       void *tcb;                /* Pointer to the TCB.  This is not always
>                                    the address of this thread descriptor.  */
>       union dtv *dtvp;
>       pthread_descr self;       /* Pointer to this structure */
>     } data;
>     void *__padding[16];
>   } p_header;
>   pthread_descr p_nextlive, p_prevlive;
>                                 /* Double chaining of active threads */
>   pthread_descr p_nextwaiting;  /* Next element in the queue holding the thr */
>   pthread_descr p_nextlock;     /* can be on a queue and waiting on a lock */
>   pthread_t p_tid;              /* Thread identifier */
>   int p_pid;                    /* PID of Unix process */

Note the use of ISA/ABI dependant data types such as ``int'', ``void 
*'', ...  As such they are implicitly included in and compiled into 
libhread-db.a

enjoy,
Andrew

Re: GDB support for thread-local storage

[Daniel Jacobowitz]

On Wed, Jun 26, 2002 at 11:56:58PM -0500, Jim Blandy wrote:
> 
> Andrew Cagney <ac131313@ges.redhat.com> writes:
> > > Doesn't libthread_db read `struct _pthread_descr_struct' from the
> > > linuxthreads library in the inferior, rather than knowing the layout
> > > itself?  So it's actually the inferior's linuxthreads library that
> > > describes its own structures' layout.
> > 
> > The structure layout will have been compiled into libthread-db.a.  It
> > implicitly knows the layout itself.
> 
> I think you're wrong here.  Could you show me the code?
> 
> (Not that the question is especially relevant to the discussion --- of
> course it could do a simple checksum check.)

You're right, and you're wrong.  For instance, td_ta_map_id2thr.c:

#include <linuxthreads/internals.h>
  struct pthread_handle_struct phc;
  if (ps_pdread (ta->ph, ta->handles + pt % pthread_threads_max, &phc,
                 sizeof (struct pthread_handle_struct)) != PS_OK)
  if (phc.h_descr == NULL)

etc.

which references:
struct pthread_handle_struct {
  struct _pthread_fastlock h_lock; /* Fast lock for sychronized access */
  pthread_descr h_descr;        /* Thread descriptor or NULL if invalid */
  char * h_bottom;              /* Lowest address in the stack thread */
};


Worse, it reads the _pthread_descr_struct from the inferior. 
_pthread_descr_struct does not describe the layout of anything; it's
the actual descriptor.  It has things like a sigjmp_buf in it.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer