Tracepoint support in Cygnus GDB ?

Tracepoint support in Cygnus GDB ?

[Saravanan]

Hi All,

     I was reading the GDB manual and realized that
  there is,practically, no tracepoint support provided
  by GDB since this support is available only for
  remote targets and for the latter , GDB doesn't provide any
  stub (like gdbserver) which has tracepoint support. I
  have a few doubts in mind regarding this :
  
  1. Why does GDB not provide this support for local
  host ?
  
  2. Is it not the case that the features (supposed to
  be) provided by tracepoints can be done using
  breakpoint features only (for instance , we can
  specify COMMANDS for  a breakpoint just as we can
  specify ACTIONS for tracepoints) ?
  
  3. Does tracepoint use the same TRAP (or ILLEGAL )
  instruction mechanism for tracepoints as it does for
  breakpoints? if not, what does it do?
  
  4. Does this trace support available in any other GDB
     supported by other vendors?
  
Regards,
Saravanan

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Wed, 24 Sep 2003 16:09:49 +0530
> From: Saravanan <pes@india.hp.com>
>   
>   1. Why does GDB not provide this support for local
>   host ?

Simple: because no one wrote tracepoints support for native debugging.

>   2. Is it not the case that the features (supposed to
>   be) provided by tracepoints can be done using
>   breakpoint features only (for instance , we can
>   specify COMMANDS for  a breakpoint just as we can
>   specify ACTIONS for tracepoints) ?

Yes.  But tracepoints are supposed to be much more lightweight
(because only raw data is accumulated during the program's run, while
complicated and expensive computations like evaluation of arbitrary
expressions, are left for later).  So, theoretically, if tracepoints
were available for native debugging, one could perhaps debug programs
without disrupting their time scale too much, maybe even debug
real-time programs (well, one's allowed to dream once in a while,
right? ;-).

>   3. Does tracepoint use the same TRAP (or ILLEGAL )
>   instruction mechanism for tracepoints as it does for
>   breakpoints?

IIRC, yes.

Re: Tracepoint support in Cygnus GDB ?

[Jim Blandy]

"Eli Zaretskii" <eliz@elta.co.il> writes:
> > Date: Wed, 24 Sep 2003 16:09:49 +0530
> > From: Saravanan <pes@india.hp.com>
> >   
> >   1. Why does GDB not provide this support for local
> >   host ?
> 
> Simple: because no one wrote tracepoints support for native
> debugging.

Right -- please contribute support for native tracepoints!

> >   2. Is it not the case that the features (supposed to
> >   be) provided by tracepoints can be done using
> >   breakpoint features only (for instance , we can
> >   specify COMMANDS for  a breakpoint just as we can
> >   specify ACTIONS for tracepoints) ?
> 
> Yes.  But tracepoints are supposed to be much more lightweight
> (because only raw data is accumulated during the program's run, while
> complicated and expensive computations like evaluation of arbitrary
> expressions, are left for later).  So, theoretically, if tracepoints
> were available for native debugging, one could perhaps debug programs
> without disrupting their time scale too much, maybe even debug
> real-time programs (well, one's allowed to dream once in a while,
> right? ;-).

Actually, the tracepoint system *does* evaluate arbitrary expressions
each time a tracepoint is hit.  The expressions are compiled to a
bytecode which is pretty simple to interpret.  Here's why it's
lightweight:
- The bytecode has had all the symbolic work squeezed out of it ---
  it's just a bunch of pretty standard machine-level operations on
  machine words.  No name lookups, no type checking, etc.  So it can
  run in much less memory and much less time than GDB's evaluator.
- This means that it can be evaluated entirely on the target.  No
  communication with the host is needed.  You can do a lot of
  computation in the time it takes a byte to go across a serial line.

There are a bunch of ways we could support this on a native system:

- We could implement it as an extension to ordinary breakpoints:
  hitting a tracepoint would stop the inferior; GDB would collect the
  data, and then continue the inferior.  I think this might not be so
  lightweight.

- We could implement a tracing agent that would actually live in the
  inferior and catch some sort of trap.  This would be more complex,
  but it would disturb the inferior less.

> >   3. Does tracepoint use the same TRAP (or ILLEGAL )
> >   instruction mechanism for tracepoints as it does for
> >   breakpoints?
> 
> IIRC, yes.

Yep.

Re: Tracepoint support in Cygnus GDB ?

[Daniel Jacobowitz]

On Wed, Sep 24, 2003 at 05:38:08PM -0500, Jim Blandy wrote:
> 
> "Eli Zaretskii" <eliz@elta.co.il> writes:
> > > Date: Wed, 24 Sep 2003 16:09:49 +0530
> > > From: Saravanan <pes@india.hp.com>
> > >   
> > >   1. Why does GDB not provide this support for local
> > >   host ?
> > 
> > Simple: because no one wrote tracepoints support for native
> > debugging.
> 
> Right -- please contribute support for native tracepoints!
> 
> > >   2. Is it not the case that the features (supposed to
> > >   be) provided by tracepoints can be done using
> > >   breakpoint features only (for instance , we can
> > >   specify COMMANDS for  a breakpoint just as we can
> > >   specify ACTIONS for tracepoints) ?
> > 
> > Yes.  But tracepoints are supposed to be much more lightweight
> > (because only raw data is accumulated during the program's run, while
> > complicated and expensive computations like evaluation of arbitrary
> > expressions, are left for later).  So, theoretically, if tracepoints
> > were available for native debugging, one could perhaps debug programs
> > without disrupting their time scale too much, maybe even debug
> > real-time programs (well, one's allowed to dream once in a while,
> > right? ;-).
> 
> Actually, the tracepoint system *does* evaluate arbitrary expressions
> each time a tracepoint is hit.  The expressions are compiled to a
> bytecode which is pretty simple to interpret.  Here's why it's
> lightweight:
> - The bytecode has had all the symbolic work squeezed out of it ---
>   it's just a bunch of pretty standard machine-level operations on
>   machine words.  No name lookups, no type checking, etc.  So it can
>   run in much less memory and much less time than GDB's evaluator.
> - This means that it can be evaluated entirely on the target.  No
>   communication with the host is needed.  You can do a lot of
>   computation in the time it takes a byte to go across a serial line.
> 
> There are a bunch of ways we could support this on a native system:
> 
> - We could implement it as an extension to ordinary breakpoints:
>   hitting a tracepoint would stop the inferior; GDB would collect the
>   data, and then continue the inferior.  I think this might not be so
>   lightweight.
> 
> - We could implement a tracing agent that would actually live in the
>   inferior and catch some sort of trap.  This would be more complex,
>   but it would disturb the inferior less.

Or do it via dynamic code replacement at the start of functions. 
That's even lighter, though a bit hard to get right.

I hope to work on this - some day.  Gotta get through dwarf
improvements first :)

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

> "Eli Zaretskii" <eliz@elta.co.il> writes:
> 
>> > Date: Wed, 24 Sep 2003 16:09:49 +0530
>> > From: Saravanan <pes@india.hp.com>
>> >   
>> >   1. Why does GDB not provide this support for local
>> >   host ?
> 
>> 
>> Simple: because no one wrote tracepoints support for native
>> debugging.
> 
> 
> Right -- please contribute support for native tracepoints!

Just don't take Jim's request at face value :-)

To get tracepoint support into GDB both new code and the refinement of 
old code will be needed.  Doing this means helping improve mainline GDB 
and, to do that, an assignment will be needed (form sent).

The thing to keep in mind is that GDB's development model is one of 
constant incremental change - fix a structure here, move a method there, 
oops just accidently fixed a 15 year old bug of not being able to return 
small structs, ...

At a more technical level, over comming months I hope to be tweaking 
GDB's target vector.  While the work doesn't directly relate to 
tracepoints, it will definitly help with a trace point implementation.

so, I guess you'll need to speak with HP lawyers :-)
Andrew

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Thu, 25 Sep 2003 10:45:32 -0400
> From: Andrew Cagney <ac131313@redhat.com>
> 
> > "Eli Zaretskii" <eliz@elta.co.il> writes:
> > 
> >> > Date: Wed, 24 Sep 2003 16:09:49 +0530
> >> > From: Saravanan <pes@india.hp.com>
> >> >   
> >> >   1. Why does GDB not provide this support for local
> >> >   host ?
> > 
> >> 
> >> Simple: because no one wrote tracepoints support for native
> >> debugging.
> > 
> > 
> > Right -- please contribute support for native tracepoints!
> 
> Just don't take Jim's request at face value :-)

On the contrary: please do ;-)

Seriously: as someone who earns his paycheck by writing and debugging
real-time software and teaching others to do so efficiently, a
debugger that can be used to run a real-time program without
completely disrupting it is something to kill for.  (Currently, use of
a debugger is limited to coredumps.)  If GDB were to support such a
feature, it would be a debugger of choice for many of my co-workers.

If done well, native tracepoints might make this possible.

> The thing to keep in mind is that GDB's development model is one of 
> constant incremental change - fix a structure here, move a method there, 
> oops just accidently fixed a 15 year old bug of not being able to return 
> small structs, ...

If this is supposed to be a gripe on the fact that no significant
user-level changes were added to GDB (except, perhaps, on GNU/Linux
systems), then I, too, am worried by that.

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

 > Right -- please contribute support for native tracepoints!

The literal interpretation of this suggestion is to go away and not come 
back until you've come up with an unmergable jumbo patch.  For GDB, 
while its possible to contribute a new architecture (we've got that 
pretty much cleaned up), it isn't possible to simply contribute a 
feature like native tracepoints (we've got much work to do).

>> The thing to keep in mind is that GDB's development model is one of 
>> constant incremental change - fix a structure here, move a method there, 
>> oops just accidently fixed a 15 year old bug of not being able to return 
>> small structs, ...
> 
> 
> If this is supposed to be a gripe on the fact that no significant
> user-level changes were added to GDB (except, perhaps, on GNU/Linux
> systems), then I, too, am worried by that.

The oposite.

Thanks to a change of emphasis away from badly integrated and incomplete 
jumbo patches and more towards change that improves/simplifies gdb's 
structural design, it's become possible to fix very very long standing 
bugs and enable work on those desired new features.

Andrew

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Sat, 27 Sep 2003 10:54:09 -0400
> From: Andrew Cagney <ac131313@redhat.com>
> > 
> > If this is supposed to be a gripe on the fact that no significant
> > user-level changes were added to GDB (except, perhaps, on GNU/Linux
> > systems), then I, too, am worried by that.
> 
> The oposite.
> 
> Thanks to a change of emphasis away from badly integrated and incomplete 
> jumbo patches and more towards change that improves/simplifies gdb's 
> structural design, it's become possible to fix very very long standing 
> bugs and enable work on those desired new features.

Still, it's disturbing, to put it mildly, that I hadn't seen any
significant new features in a long while.  Perhaps we should decide on
a list of new features that the next release should have, and start
working on them.

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

> 
> Still, it's disturbing, to put it mildly, that I hadn't seen any
> significant new features in a long while.

That isn't correct.  GDB 6.0 does contain a significant number of user 
visible features: hosted file I/O (which is embedded), TLS, NPTL, 
separate debug info (which will help embedded), useable java, follow 
fork, ...

On the horizon are Ada, gcov/gprof, strace, ...

What is missing is the good old days [sarcasm] of the eye catching 
glamor jumbo patch - incomplete hacks that were piled, one by one by 
one, on top of GDB until it nearly collapsed.  While we're still digging 
ourselves out of that one, much progress has been made.

 >  Perhaps we should decide on
 > a list of new features that the next release should have, and start
 > working on them.

We've tried that, most recently with 6.0 and some MI features, and 
failed.  As a group we found it necessary to largely disconnect release 
cycles from feature cycles.  Instead releases based are based more on 
the calendar (yes this one is badly late) than some arbitrary feature list.

enjoy,
Andrew

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Sat, 27 Sep 2003 14:37:07 -0400
> From: Andrew Cagney <ac131313@redhat.com>
> 
> > Still, it's disturbing, to put it mildly, that I hadn't seen any
> > significant new features in a long while.
> 
> That isn't correct.  GDB 6.0 does contain a significant number of user 
> visible features: hosted file I/O (which is embedded), TLS, NPTL, 
> separate debug info (which will help embedded), useable java, follow 
> fork, ...

Sorry, this doesn't refute what I said: TLS, NPTL, separate debug
info, and follow-fork features work on GNU/Linux only.  Hosted I/O is
only useful for embedded targets, and Java is only useful for Java
programmers.

I did say in my original message that GNU/Linux was an exception: most
of the new features work only on that system.  Your list supports what
I said.

What I was after was significant new features for native debugging
that would work not only on GNU/Linux, and not only for some specific
programming language.

If the main maintenance effort until now was supposed to make addition
of such features easier, then I applaud that effort and am sympathetic
to it, but I still am impatient to see the features themselves.

>  >  Perhaps we should decide on
>  > a list of new features that the next release should have, and start
>  > working on them.
> 
> We've tried that, most recently with 6.0 and some MI features, and 
> failed.

How did we fail, exactly?  What were the reasons for the failure?
Perhaps we could learn from past mistakes and do better next time?

> As a group we found it necessary to largely disconnect release 
> cycles from feature cycles.  Instead releases based are based more on 
> the calendar (yes this one is badly late) than some arbitrary feature list.

These two goals not necessarily contradict.  We could set up a list of
features that are to be included in the next release, and if some of
the features are not ready in time, make a release without them.

IMHO, having a relatively short list of user-level features that are
first priority would be a good aid for maintainers, in setting their
priority to review patches, if for nothing else.

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

>> Date: Sat, 27 Sep 2003 14:37:07 -0400
>> From: Andrew Cagney <ac131313@redhat.com>
>> 
> 
>> > Still, it's disturbing, to put it mildly, that I hadn't seen any
>> > significant new features in a long while.
> 
>> 
>> That isn't correct.  GDB 6.0 does contain a significant number of user 
>> visible features: hosted file I/O (which is embedded), TLS, NPTL, 
>> separate debug info (which will help embedded), useable java, follow 
>> fork, ...
> 
> 
> Sorry, this doesn't refute what I said: TLS, NPTL, separate debug
> info, and follow-fork features work on GNU/Linux only.  Hosted I/O is
> only useful for embedded targets, and Java is only useful for Java
> programmers.

Keep in mind that GNU systems are a priority.

TLS and NPTL are native GNU/Linux only for reasons out side of GDB's 
control.  Tweaking TLS for other systems should be straight forward.

Follow fork isn't GNU/Linux specific (it originated in HP/UX) but does 
need per-native target changes.

Separate debug info is very much an embedded feature - it lets embedded 
distros make optional all the debug info for all those embedded C and 
C++ libraries.

CFI and location expresions benefit all programmers and are a 
significant new feature (but are not exactly user visible so lack the 
"gee wiz" factor, which is why I didn't mention them).

 From memory, the most recent embedded feature (related to watchpoints 
and the target vector) stalled, the original patch need maintenance 
work, and that hasn't yet happened.

>>  >  Perhaps we should decide on
>>  > a list of new features that the next release should have, and start
>>  > working on them.
>> 
>> We've tried that, most recently with 6.0 and some MI features, and 
>> failed.
> 
> 
> How did we fail, exactly?  What were the reasons for the failure?
> Perhaps we could learn from past mistakes and do better next time?

The usual problems.  Too many "must-have" features, and the few features 
that people do work on running late.

>> As a group we found it necessary to largely disconnect release 
>> cycles from feature cycles.  Instead releases based are based more on 
>> the calendar (yes this one is badly late) than some arbitrary feature list.
> 
> 
> These two goals not necessarily contradict.  We could set up a list of
> features that are to be included in the next release, and if some of
> the features are not ready in time, make a release without them.

That is exactly what we did (the TODO file, 5.0) and, in the end, I 
abandoned every single feature on that list.  Many have since been 
added, but not thanks to attempts to tie them to specific releases.

Having largely disconnected the release schedule from features, we're 
doing much better.  We make regular GDB updates that do contain new 
features and, more importantly, fixes.  At the same time people adding 
features aren't pressured into bad shortcuts for the sake of an 
unrealistic release schedule.

> IMHO, having a relatively short list of user-level features that are
> first priority would be a good aid for maintainers, in setting their
> priority to review patches, if for nothing else.

I believe everyone here knows the #1 priority.  Make:

(gdb) break main
(gdb) run
(gdb) bt
(gdb) print foo

work.  At present it doesn't.  Breaking it down, "print foo" and "break 
main" both require name spaces, "bt" requires improved frame code. 
Things that are being actively worked on.

Andrew

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Sun, 28 Sep 2003 10:13:38 -0400
> From: Andrew Cagney <ac131313@redhat.com>
> 
> Keep in mind that GNU systems are a priority.

I'm aware of that, but IMHO GDB development favors GNU/Linux too much.
And since the Linux kernel changes so frequently, the symbiotic
relationship between GDB and GNU/Linux tends to consume a lot of
resources, as we are shooting a moving target.

In other words, I'd be happier if a larger portion of GDB maintenance
resources were to go into more platform-independent features.

> TLS and NPTL are native GNU/Linux only for reasons out side of GDB's 
> control.  Tweaking TLS for other systems should be straight forward.
> 
> Follow fork isn't GNU/Linux specific (it originated in HP/UX) but does 
> need per-native target changes.

The facts are that these features are currently available only for
GNU/Linux.

> Separate debug info is very much an embedded feature - it lets embedded 
> distros make optional all the debug info for all those embedded C and 
> C++ libraries.

Still, it requires a feature in Binutils that only exists on
GNU/Linux.

> CFI and location expresions benefit all programmers and are a 
> significant new feature (but are not exactly user visible so lack the 
> "gee wiz" factor, which is why I didn't mention them).

Exactly: this is not a user-level feature, but more like a bugfix.

> The usual problems.  Too many "must-have" features, and the few features 
> that people do work on running late.

Then let's have less must-have features next time.

> > These two goals not necessarily contradict.  We could set up a list of
> > features that are to be included in the next release, and if some of
> > the features are not ready in time, make a release without them.
> 
> That is exactly what we did (the TODO file, 5.0) and, in the end, I 
> abandoned every single feature on that list.

Are you saying that there's no way we could set up practical goals for
GDB development?  I'd be surprised if you actually meant that, but
that's how it sounds.

> Having largely disconnected the release schedule from features, we're 
> doing much better.  We make regular GDB updates that do contain new 
> features and, more importantly, fixes.

As I said at the beginning of this discussion, I'm worried by the fact
that most of the recent GDB development effort is consumed by fixing
bugs and by building infrastructure, and very little goes into
user-level features.  So I applaud the progress being made in the
directions you describe, but I'd like to see our progress defined more
by user-level features, not only by bugfixes and infrastructure-related
work.

> > IMHO, having a relatively short list of user-level features that are
> > first priority would be a good aid for maintainers, in setting their
> > priority to review patches, if for nothing else.
> 
> I believe everyone here knows the #1 priority.  Make:
> 
> (gdb) break main
> (gdb) run
> (gdb) bt
> (gdb) print foo
> 
> work.  At present it doesn't.

Something that should work but doesn't is a bug.  So you again are
talking about fixing a bug; I don't see any user-level feature in the
above.

> Breaking it down, "print foo" and "break 
> main" both require name spaces, "bt" requires improved frame code. 

That's infrastructure again.  While I understand the need for building
good infrastructure, I hope that infrastructure will be used for
adding user-level features, hopefully sooner rather than later.

Re: Tracepoint support in Cygnus GDB ?

[Daniel Jacobowitz]

On Sun, Sep 28, 2003 at 11:41:20PM +0200, Eli Zaretskii wrote:
> > Separate debug info is very much an embedded feature - it lets embedded 
> > distros make optional all the debug info for all those embedded C and 
> > C++ libraries.
> 
> Still, it requires a feature in Binutils that only exists on
> GNU/Linux.

That is not correct.  It works for any ELF target.

> > CFI and location expresions benefit all programmers and are a 
> > significant new feature (but are not exactly user visible so lack the 
> > "gee wiz" factor, which is why I didn't mention them).
> 
> Exactly: this is not a user-level feature, but more like a bugfix.

Hardly.  Don't you consider improved debugging of optimized code to be
a feature?  Especially considering how abysmal support is now.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

>> Date: Sun, 28 Sep 2003 10:13:38 -0400
>> From: Andrew Cagney <ac131313@redhat.com>
>> 
>> Keep in mind that GNU systems are a priority.
> 
> 
> I'm aware of that, but IMHO GDB development favors GNU/Linux too much.
> And since the Linux kernel changes so frequently, the symbiotic
> relationship between GDB and GNU/Linux tends to consume a lot of
> resources, as we are shooting a moving target.

There's a certain irony here.  People also complain that GDB favours 
embedded systems over GNU/Linux native systems; and spends too much time 
adding new architectures.

The symbiotic relationship is between GDB and the compiler; not 
GNU/Linux.  It's the compiler that GDB's trying to keep pace with (well 
actually catch up).

As for GNU systems (such as GNU/Linux).  If existing functionality 
breaks, we'd better do something about it.

TLS, for instance, is an ABI extension that is implemented by GCC (with 
a little bit of co-operation from the [linux] kernel and [g]libc).  Non 
GNU/Linux systems will definitly pick up and exploit the feature!

>> TLS and NPTL are native GNU/Linux only for reasons out side of GDB's 
>> control.  Tweaking TLS for other systems should be straight forward.
>> 
>> Follow fork isn't GNU/Linux specific (it originated in HP/UX) but does 
>> need per-native target changes.
> 
> 
> The facts are that these features are currently available only for
> GNU/Linux.

This simply reflects the reality that most developers use GNU/Linux systems.

Unlike the past, the emphasis is on doing the internals in a portable 
way so that they do apply across all systems.  This definitly wasn't the 
case when the original HP/UX only follow-fork implementation was committed.

The missing bit is an interest, by non GNU/Linux users, in getting the 
features added to their system.

>> Separate debug info is very much an embedded feature - it lets embedded 
>> distros make optional all the debug info for all those embedded C and 
>> C++ libraries.
> 
> 
> Still, it requires a feature in Binutils that only exists on
> GNU/Linux.

Eli, here you're simply wrong.  It requires a compatible toolchain 
(binutils, elfutils).  It does not require GNU/Linux.

> Are you saying that there's no way we could set up practical goals for
> GDB development?  I'd be surprised if you actually meant that, but
> that's how it sounds.

I'm saying trying to tie specific features to specific releases is 
unrealistic.

>> Having largely disconnected the release schedule from features, we're 
>> doing much better.  We make regular GDB updates that do contain new 
>> features and, more importantly, fixes.
> 
> 
> As I said at the beginning of this discussion, I'm worried by the fact
> that most of the recent GDB development effort is consumed by fixing
> bugs and by building infrastructure, and very little goes into
> user-level features.  So I applaud the progress being made in the
> directions you describe, but I'd like to see our progress defined more
> by user-level features, not only by bugfixes and infrastructure-related
> work.

>> > IMHO, having a relatively short list of user-level features that are
>> > first priority would be a good aid for maintainers, in setting their
>> > priority to review patches, if for nothing else.
> 
>> 
>> I believe everyone here knows the #1 priority.  Make:
>> 
>> (gdb) break main
>> (gdb) run
>> (gdb) bt
>> (gdb) print foo
>> 
>> work.  At present it doesn't.
> 
> 
> Something that should work but doesn't is a bug.  So you again are
> talking about fixing a bug; I don't see any user-level feature in the
> above.

The above is subtle.

Having it work correctly requires the constant addition of new features.

>> Breaking it down, "print foo" and "break 
>> main" both require name spaces, "bt" requires improved frame code. 
> 
> 
> That's infrastructure again.  While I understand the need for building
> good infrastructure, I hope that infrastructure will be used for
> adding user-level features, hopefully sooner rather than later.

Is the request to support debugging i386 binaries on x86-64 a bug, 
infrastructure, or a user visible feature?  I'd argue its a feature, 
only, once working it just gives the user a better "break main; run" 
experience.

Is namespace support a bug, infrastructure, or a new feature.  Again, 
I'd argue that its a new feature, but again at the end of the day it 
just improves "break main; run".

Is the lack of lazy breakpoints a bug, infrastructure or new feature 
(it's what every Java developer I know is asking for)?  In the end, 
though, it too only results in a better "break main; run" experience so 
I guess its a bug.

The inability to set a breakpoint on inline functions?  Bug? 
Infrastructure?  Feature?  Anyone using anything inline complains about 
that one.

I can't think of anyone here that is working on infrastructure for 
infrastructures sake (If I were doing that I'd be spending my time 
re-writing architecture specific code instead of deprecating / deleting 
them :-).  The objective is new features.  However, due to past neglect, 
we're now paying dearly spending more on cleanup and [arrrrg] finish up 
and less on user features.

Andrew

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Sun, 28 Sep 2003 17:07:01 -0400
> From: Daniel Jacobowitz <drow@mvista.com>
> 
> On Sun, Sep 28, 2003 at 11:41:20PM +0200, Eli Zaretskii wrote:
> > > Separate debug info is very much an embedded feature - it lets embedded 
> > > distros make optional all the debug info for all those embedded C and 
> > > C++ libraries.
> > 
> > Still, it requires a feature in Binutils that only exists on
> > GNU/Linux.
> 
> That is not correct.  It works for any ELF target.

Perhaps I don't know something: I thought that separate debug info
required support from Binutils, to be able to produce the files with
the debug info.  I'm sure at some point someone said here that this
option is not available on all systems.  Isn't that right?  What is
the exact procedure of producing the separate debug info?

> Don't you consider improved debugging of optimized code to be
> a feature?  Especially considering how abysmal support is now.

I consider this a bugfix.

Re: Tracepoint support in Cygnus GDB ?

[Eli Zaretskii]

> Date: Sun, 28 Sep 2003 18:23:40 -0400
> From: Andrew Cagney <ac131313@redhat.com>
> 
> The symbiotic relationship is between GDB and the compiler; not 
> GNU/Linux.  It's the compiler that GDB's trying to keep pace with (well 
> actually catch up).

I think, at least in the case of GNU/Linux, this is inaccurate.  We
are trying to catch up with the compiler, the library (glibc), and the
kernel (the system calls and innards that directly affect features
like threading and ptrace).  My (perhaps inaccurate) impression is
that quite a few changes in GDB are due to the need to track those
fast moving targets.

> As for GNU systems (such as GNU/Linux).  If existing functionality 
> breaks, we'd better do something about it.

If the debugger knows less about the initimate details of a particular
OS, it is less likely to break when some central component of the OS
changes.  I'm sure we are all aware of that, so it bewilders me why
should we argue about this trivial piece of knowledge.

> Unlike the past, the emphasis is on doing the internals in a portable 
> way so that they do apply across all systems.  This definitly wasn't the 
> case when the original HP/UX only follow-fork implementation was committed.

Andrew, you misunderstand me.  I did not intend to publish some
criticism on the way you lead GDB development as opposed to what was
before.  I'm just voicing my personal concerns about what I percieve
as a lack of significant progress, user-level feature-wise, in GDB
during the last years.

As a data point, consider the number of changes to the user manual:
the vast majority of changes didn't require any additions to the
manual.  To me, it's a clear sign that most of the efforts do not
produce new features.

> Eli, here you're simply wrong.  It requires a compatible toolchain 
> (binutils, elfutils).  It does not require GNU/Linux.

On what systems, except on GNU/Linux, do we have such a toolchain
available?

> > Are you saying that there's no way we could set up practical goals for
> > GDB development?  I'd be surprised if you actually meant that, but
> > that's how it sounds.
> 
> I'm saying trying to tie specific features to specific releases is 
> unrealistic.

I suggested to point out specific goals, but not to tie them
dogmatically to specific releases.

> Is the request to support debugging i386 binaries on x86-64 a bug, 
> infrastructure, or a user visible feature?

A minor feature, IMHO.

> Is namespace support a bug, infrastructure, or a new feature.  Again, 
> I'd argue that its a new feature, but again at the end of the day it 
> just improves "break main; run".

If it does nothing else than improve "break main; run", then it's not
a feature.

> Is the lack of lazy breakpoints a bug, infrastructure or new feature 

IMHO, it's an improvement in GDB efficiency, but functionally it's
not a new feature.

> The inability to set a breakpoint on inline functions?  Bug? 
> Infrastructure?  Feature?  Anyone using anything inline complains about 
> that one.

A bug or a minor feature.

> The objective is new features.  However, due to past neglect, 
> we're now paying dearly spending more on cleanup and [arrrrg] finish up 
> and less on user features.

I just hope that we don't lose the sight of the forest (user features)
for the trees, that's all.

Re: Tracepoint support in Cygnus GDB ?

[Daniel Jacobowitz]

On Mon, Sep 29, 2003 at 08:15:08AM +0200, Eli Zaretskii wrote:
> > Date: Sun, 28 Sep 2003 17:07:01 -0400
> > From: Daniel Jacobowitz <drow@mvista.com>
> > 
> > On Sun, Sep 28, 2003 at 11:41:20PM +0200, Eli Zaretskii wrote:
> > > > Separate debug info is very much an embedded feature - it lets embedded 
> > > > distros make optional all the debug info for all those embedded C and 
> > > > C++ libraries.
> > > 
> > > Still, it requires a feature in Binutils that only exists on
> > > GNU/Linux.
> > 
> > That is not correct.  It works for any ELF target.
> 
> Perhaps I don't know something: I thought that separate debug info
> required support from Binutils, to be able to produce the files with
> the debug info.  I'm sure at some point someone said here that this
> option is not available on all systems.  Isn't that right?  What is
> the exact procedure of producing the separate debug info?

It's not available on non-ELF targets :)  Although, it could be
implemented, if anyone cared to do so.  It requires use of either
elfutils or of binutils' objcopy program.  See the manual for
--only-keep-debug and --add-gnu-debuglink.

> > Don't you consider improved debugging of optimized code to be
> > a feature?  Especially considering how abysmal support is now.
> 
> I consider this a bugfix.

Having both implemented and used it, I do not.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

>> Date: Sun, 28 Sep 2003 18:23:40 -0400
>> From: Andrew Cagney <ac131313@redhat.com>
>> 
>> The symbiotic relationship is between GDB and the compiler; not 
>> GNU/Linux.  It's the compiler that GDB's trying to keep pace with (well 
>> actually catch up).
> 
> 
> I think, at least in the case of GNU/Linux, this is inaccurate.  We
> are trying to catch up with the compiler, the library (glibc), and the
> kernel (the system calls and innards that directly affect features
> like threading and ptrace).  My (perhaps inaccurate) impression is
> that quite a few changes in GDB are due to the need to track those
> fast moving targets.

For GLIBC and the Linux kernel [lets be rude :-] while they do suck a 
certain percentage of resources, it's largly [and unfortunatly] due to 
us needing to identify problems in the upstream code.  The GDB changes 
that are a direct consequence of the mods are actually minimal.

That leaves GCC and other compilers with their increasingly agressive 
optimizations.

Even there, there is some good news.  I believe that, in the past, the 
single biggest maintenance headache was the prologue analyzer.  With 
CFI, that's largely stopped.  Instead, provided the compiler is working 
correctly, GDB can unwind any stack frame.  Maintainers, and developers, 
can get on with more interesting tasks such as adding new features :-)

Same goes for location expressions (once finished, its going to involve 
more surgery, sigh).

>> As for GNU systems (such as GNU/Linux).  If existing functionality 
>> breaks, we'd better do something about it.
> 
> 
> If the debugger knows less about the initimate details of a particular
> OS, it is less likely to break when some central component of the OS
> changes.  I'm sure we are all aware of that, so it bewilders me why
> should we argue about this trivial piece of knowledge.

>> Unlike the past, the emphasis is on doing the internals in a portable 
>> way so that they do apply across all systems.  This definitly wasn't the 
>> case when the original HP/UX only follow-fork implementation was committed.
> 
> 
> Andrew, you misunderstand me.  I did not intend to publish some
> criticism on the way you lead GDB development as opposed to what was
> before.  I'm just voicing my personal concerns about what I percieve
> as a lack of significant progress, user-level feature-wise, in GDB
> during the last years.

I'm not taking it that way :-)  I'm trying to draw your attention to the 
mistakes we as the GDB community allowed to happen in the past.

> As a data point, consider the number of changes to the user manual:
> the vast majority of changes didn't require any additions to the
> manual.  To me, it's a clear sign that most of the efforts do not
> produce new features.

The apparent lack of new features is a reasonable concern.

The point I'm trying to make is that there is a very real reason for 
this.  GDB suffered [here we go, sob story :-)] from years of neglect. 
The '90s were a period where new architectures and even some new 
features were added, regardless of their level of completness or 
maintenance cost.  As maintainers we're now paying for those years of 
neglect.

>> Eli, here you're simply wrong.  It requires a compatible toolchain 
>> (binutils, elfutils).  It does not require GNU/Linux.
> 
> 
> On what systems, except on GNU/Linux, do we have such a toolchain
> available?

I think Daniel's explained this.

>> > Are you saying that there's no way we could set up practical goals for
>> > GDB development?  I'd be surprised if you actually meant that, but
>> > that's how it sounds.
> 
>> 
>> I'm saying trying to tie specific features to specific releases is 
>> unrealistic.
> 
> 
> I suggested to point out specific goals, but not to tie them
> dogmatically to specific releases.

That is reasonable.  At present they are bug reports (category 
change-request).  I think Joel suggested a page to cross reference the 
bug reports so that new features were easier to find.

Just keep in mind, and going back to my original concern, the emphasis 
here must not be adding a new feature at all cost.  The last thing this 
group needs is for people to get an unrealistic expectation that their 
jumbo feature patch is simply going to be committed .  Instead its a 
case of making the incremental changes needed so that the feature, in 
the end, just slips in.

>> The objective is new features.  However, due to past neglect, 
>> we're now paying dearly spending more on cleanup and [arrrrg] finish up 
>> and less on user features.
> 
> 
> I just hope that we don't lose the sight of the forest (user features)
> for the trees, that's all.

Hmm, good analogy.  In the past exactly that did happen.  People kept 
trying to cram more trees (architectures / features) into an already 
overgrown forest.  Think of the current work as thinning the trees so we 
can once again see the entire forest.

To take this back to the original question (I should note that I speak 
from experience here).  In the deep dark past I wrote RDA (Red Hat's 
debug agent). I then, later looked at _quickly_ grafting a sample 
tracepoint implementation on top of it.  I dropped it having figured out 
the amount of work required.  But did learn several things:

- rda's strongly OO target stack was "in the ballpark"
If GDB's going to get tracepoints, its target stack will need to be 
changed to work in similar ways.  There are a heap of reasons why GDB's 
target stack needs an upgrade, this is one of them.  Others include the 
need to finish async.

- the very non-OO sample tracepoint code, was struggling.
It had a too direct and global a connection to both the architecture and 
target.  Having someone cleanroom the code is a realistic option.

- long term, gdbserver and gdb should merge
(I know daniel questions this :-) If the lower layers of the target 
stack can be stripped back to something like rda/gdbserver then the 
merge will be possible.  By doing this we will have eliminated much 
notable redundency in GDBs code, and in eliminating that redundency it 
will be possible to add target based features just once.

enjoy,
Andrew

Re: Tracepoint support in Cygnus GDB ?

[Daniel Jacobowitz]

On Mon, Sep 29, 2003 at 10:47:52AM -0400, Andrew Cagney wrote:
> - long term, gdbserver and gdb should merge
> (I know daniel questions this :-) If the lower layers of the target 
> stack can be stripped back to something like rda/gdbserver then the 
> merge will be possible.  By doing this we will have eliminated much 
> notable redundency in GDBs code, and in eliminating that redundency it 
> will be possible to add target based features just once.

Nah, I love it :)  We just had some disagreements about methods the
last time it came up, I think.  I barely recall.  It requires
substantial work on both sides; in a release or three, we should sit
down and draw out a way to make it work.

-- 
Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Re: Tracepoint support in Cygnus GDB ?

[Jim Blandy]

Andrew Cagney <ac131313@redhat.com> writes:
>  > Right -- please contribute support for native tracepoints!
> 
> The literal interpretation of this suggestion is to go away and not
> come back until you've come up with an unmergable jumbo patch.

Geez.  It's not as if I said, "--- and please do it badly!!!"

Saravanan, I think there are two points germane to your original post:

- At the moment, I don't know of any company that is investing effort
  in the tracepoint support.  But GDB's feature set is not restricted
  to those things that a small group of corporations decide they want
  in closed meetings.  Anyone with the skills and the patience to work
  through the design with the rest of the group, and the time to code
  it up, can get a feature in.  So if native support for tracepoints
  is of interest to you, and you think you've got the skills and the
  time, I really encourage you to take it on yourself.

  The first step would be to start a discussion on this list and build
  a consensus on how to do it.  Michael and I would be happy to see
  the tracepoint stuff worked on, so we'd help out as best we could.
  If you convinced him you were serious, Andrew would probably explain
  some of the things he hinted at in his previous message enough that
  someone who doesn't actually share his cerebellum could start
  working on them.  Maybe he already has, and could post pointers to
  archived messages.

- In the past, companies in positions of power have chosen to
  implement various features in GDB as quickly and cheaply as
  possible, and put off dealing with the impact on simplicity and
  maintainability until never.  I would say the original C++ and
  thread support would fall in this category, but supporting (at one
  point) forty-some architectures without much attention paid to the
  interface between per-architecture and generic code had its effects,
  too.

  Andrew's spent years doing work on GDB's fundamentals to disentangle
  the architecture description system and the frame system.  I just
  re-did one architecture's frame handling using his new interfaces,
  and it took me a single day to finish.  This used to take me a week,
  and it'd still be wrong.

  David Carlton essentially volunteered a year of his time to work
  with Daniel Jacobowitz to straighten out the C++ support.  I'm not a
  C++ user myself, but I understand it's vastly better than it used to
  be.

  In this context, you can see why one would want to place an emphasis
  on cleaning up some of the supporting infrastructure to make way for
  native tracepoints, and then doing it right, rather than just
  pursuing the shortest path.

Re: Tracepoint support in Cygnus GDB ?

[Andrew Cagney]

>   The first step would be to start a discussion on this list and build
>   a consensus on how to do it.  Michael and I would be happy to see
>   the tracepoint stuff worked on, so we'd help out as best we could.
>   If you convinced him you were serious, Andrew would probably explain
>   some of the things he hinted at in his previous message enough that
>   someone who doesn't actually share his cerebellum could start
>   working on them.  Maybe he already has, and could post pointers to
>   archived messages.

(Snide remarks aside) I would go through the archives and the target 
code (FIXME comments in remote.c, for instance) the the basic theory of 
the target stack has been posted and discussed many times.  The key word 
"sandwich" in gdb@ throws up one such post :-)

Just, please don't look to me for highly detailed and definitive 
interface specifications.  I see no value in providing developers with 
such a level of formalization.  Let the person writing the code figure 
out an interface that just meets the immediate need; and recognize that 
in 6 months new requirements will change the interface anyway.  No need 
for immediate perfection.

> - In the past, companies in positions of power have chosen to
>   implement various features in GDB as quickly and cheaply as
>   possible, and put off dealing with the impact on simplicity and
>   maintainability until never.  I would say the original C++ and
>   thread support would fall in this category, but supporting (at one
>   point) forty-some architectures without much attention paid to the
>   interface between per-architecture and generic code had its effects,
>   too.

The problem will always be there, and can't simply be attributed to 
employee pressure.  As a reasonable generalization, people have a 
tendency to try to avoid the hard but necessary work of peer review, 
incremental design, shortcut avoidance, finishing features, writing 
testsuites, ....  We're all responsible for ensuring these things do occure.

enjoy,
Andrew

Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Stan Shebs]

Andrew Cagney wrote:

>
> The apparent lack of new features is a reasonable concern.
>
> The point I'm trying to make is that there is a very real reason for 
> this.  GDB suffered [here we go, sob story :-)] from years of neglect. 
> The '90s were a period where new architectures and even some new 
> features were added, regardless of their level of completness or 
> maintenance cost.  As maintainers we're now paying for those years of 
> neglect.

Heh, I've been trolled! :-)  I must say, I've been a little envious
watching GDB development over the past couple of years; Cygnus was never
able to afford so many cycles spent on internals. For multi-arch alone
it took over three years from initial proposal to the actual hacking...

>
> Just keep in mind, and going back to my original concern, the emphasis 
> here must not be adding a new feature at all cost.  The last thing 
> this group needs is for people to get an unrealistic expectation that 
> their jumbo feature patch is simply going to be committed .  Instead 
> its a case of making the incremental changes needed so that the 
> feature, in the end, just slips in.

The downside of this approach is that if you discourage contributors too
much up front, you'll never get those new features, cleanly implemented or
otherwise.  This is exactly the problem that Richard Kenner caused for GCC;
he didn't want exception handling from the C++ folks because he was going
to do a much better implementation, and they would use that.  After years
of waiting for him to actually do it, it turned out to be easier for the
rest of the GCC hackers just to fork (thus EGCS). Now GDB today is not at
all like pre-EGCS GCC, but some of the same tension is there.

It's a balancing act; you can't just hold off on new features until the
infrastructure is perfected, because it will always seem flawed in some
way.  At the same time entropy always increases, and new features make it
increase faster, so you need the infrastructure work to compensate.

I just thumbed through the GDB website and sources, and gdb/TODO seemed
like it was mostly a list of desired infrastructure changes.  For instance,
I didn't see a mention of the desirability of tracepoint support for native
debugging, although that is an obvious item for a wishlist.  Perhaps the
todo list should be separated into "user-visible" and "infrastructure"
sections, with appropriate xrefs, so that everybody is aware of what's
desired.

Stan

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Andrew Cagney]

> Heh, I've been trolled! :-)  I must say, I've been a little envious
> watching GDB development over the past couple of years; Cygnus was never
> able to afford so many cycles spent on internals. For multi-arch alone
> it took over three years from initial proposal to the actual hacking...

Interesting view of history.

> I just thumbed through the GDB website and sources, and gdb/TODO seemed
> like it was mostly a list of desired infrastructure changes.  For instance,
> I didn't see a mention of the desirability of tracepoint support for native
> debugging, although that is an obvious item for a wishlist.  Perhaps the
> todo list should be separated into "user-visible" and "infrastructure"
> sections, with appropriate xrefs, so that everybody is aware of what's
> desired.

GDB tracks new features using the bug database.  I see one tracepoint 
change which is me pointing out that an architecture method is broken. 
Must remember to delete the TODO file.

Andrew

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Stan Shebs]

Andrew Cagney wrote:

>> Heh, I've been trolled! :-)  I must say, I've been a little envious
>> watching GDB development over the past couple of years; Cygnus was never
>> able to afford so many cycles spent on internals. For multi-arch alone
>> it took over three years from initial proposal to the actual hacking...
>
>
> Interesting view of history.

You sound dubious... But in 1995 I drew up a document listing a bunch
of directions for Cygnus to pursue with GDB, presented them at every
quarterly meeting, and management would smile and nod and not make
any promises.  Multi-arch even got onto the future work schedule
a couple times, but then contracts came in and bumped it off again.
The 1995 doc is presumably still officially RH confidential, but
perhaps you could get somebody over there to approve posting it; it
would be a useful window into a less-well-known era of GDB history.

Stan

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Andrew Cagney]

> Andrew Cagney wrote:
> 
> Heh, I've been trolled! :-)  I must say, I've been a little envious
> watching GDB development over the past couple of years; Cygnus was never
> able to afford so many cycles spent on internals. For multi-arch alone
> it took over three years from initial proposal to the actual hacking...

...

> You sound dubious... But in 1995 I drew up a document listing a bunch
> of directions for Cygnus to pursue with GDB, presented them at every
> quarterly meeting, and management would smile and nod and not make
> any promises.  Multi-arch even got onto the future work schedule
> a couple times, but then contracts came in and bumped it off again.
> The 1995 doc is presumably still officially RH confidential, but
> perhaps you could get somebody over there to approve posting it; it
> would be a useful window into a less-well-known era of GDB history.

Do you accept that, in hindsite, such an approach was doomed to failure? 
  Mgt could never buy into such large infrastructure investments and, 
hence, were paying you lip service.  Even when you did get multi-arch 
onto the scheduled, the work got cut short.  multi-arch continues to be 
finished by other means.

Rather than looking at features and infrastructure as adversaries, think 
of them as mutual friends.  One works off the other.  A simplification 
of the code here, leads to a simpler/faster/correct implementation of a 
feature there.

During that period, Cygnus failed to recognize such benefits, and as a 
consequence, swandered an oportunity to do infrastructure work for free.

Fortunatly, that has since changed.

Andrew

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Stan Shebs]

Andrew Cagney wrote:

>> I just thumbed through the GDB website and sources, and gdb/TODO seemed
>> like it was mostly a list of desired infrastructure changes.  For 
>> instance,
>> I didn't see a mention of the desirability of tracepoint support for 
>> native
>> debugging, although that is an obvious item for a wishlist.  Perhaps the
>> todo list should be separated into "user-visible" and "infrastructure"
>> sections, with appropriate xrefs, so that everybody is aware of what's
>> desired.
>
>
> GDB tracks new features using the bug database.  I see one tracepoint 
> change which is me pointing out that an architecture method is broken. 
> Must remember to delete the TODO file.

OK, I see plenty there, but most look like infrastructure changes,
and of the 139 items, I still can't tell which are the most
important features that users have been waiting for.  That's
where a real document comes in handy; you can use it to explain
why features A, B, and C are more important than D, E, and F,
or why feature G must wait for infrastructure changes H, I, and J.

Stan

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Andrew Cagney]

> 
> OK, I see plenty there, but most look like infrastructure changes,
> and of the 139 items, I still can't tell which are the most
> important features that users have been waiting for.  That's
> where a real document comes in handy; you can use it to explain
> why features A, B, and C are more important than D, E, and F,
> or why feature G must wait for infrastructure changes H, I, and J.

Bugzilla lets you specify bug dependencies, and lets users vote on bug 
urgency.  Joel? (well they were canadian), from memory, also observed 
that GDBs contribute page should point the new developer in the 
direction of a few easy/mundane tasks.

Andrew

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Stan Shebs]

Andrew Cagney wrote:

>> Andrew Cagney wrote:
>>
>> Heh, I've been trolled! :-)  I must say, I've been a little envious
>> watching GDB development over the past couple of years; Cygnus was never
>> able to afford so many cycles spent on internals. For multi-arch alone
>> it took over three years from initial proposal to the actual hacking...
>
>
> ...
>
>> You sound dubious... But in 1995 I drew up a document listing a bunch
>> of directions for Cygnus to pursue with GDB, presented them at every
>> quarterly meeting, and management would smile and nod and not make
>> any promises.  Multi-arch even got onto the future work schedule
>> a couple times, but then contracts came in and bumped it off again.
>> The 1995 doc is presumably still officially RH confidential, but
>> perhaps you could get somebody over there to approve posting it; it
>> would be a useful window into a less-well-known era of GDB history.
>
>
> Do you accept that, in hindsite, such an approach was doomed to 
> failure?  Mgt could never buy into such large infrastructure 
> investments and, hence, were paying you lip service.  Even when you 
> did get multi-arch onto the scheduled, the work got cut short.  
> multi-arch continues to be finished by other means.

That's an interesting question.  Thinking about that, and comparing with
GCC experience, I'd say that in general it's just extremely difficult to
get infrastructural work accomplished in a small group or small company;
you'd have to have a sufficiently large and/or well-funded group that
the time taken by infrastructure does not affect the group's overall
schedule.

>
> Rather than looking at features and infrastructure as adversaries, 
> think of them as mutual friends.  One works off the other.  A 
> simplification of the code here, leads to a simpler/faster/correct 
> implementation of a feature there.

I think we're in agreement on that.

>
> During that period, Cygnus failed to recognize such benefits, and as a 
> consequence, swandered an oportunity to do infrastructure work for free.
>
> Fortunatly, that has since changed.

I'm very glad that Red Hat can manage it! Cygnus was much more of a
hand-to-mouth operation; if contract work didn't get delivered on time,
people had to be laid off. There was VC money later, but it mostly got
swallowed up by various unprofitable initiatives.

Stan

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Joel Brobecker]

> That's an interesting question.  Thinking about that, and comparing with
> GCC experience, I'd say that in general it's just extremely difficult to
> get infrastructural work accomplished in a small group or small company;
> you'd have to have a sufficiently large and/or well-funded group that
> the time taken by infrastructure does not affect the group's overall
> schedule.

Unfortunately, my own experience with large companies is that they have
very very very controlled budgets and that it's hard to explain to a
manager who only cares about functionality that such and such
infrastructure work is going to save some bucks later if the rework is
going to take more than, say, a few days.

In my opinion, unless you find a talented developper willing to invest
a lot of his own time, the right way to go is incremental improvements
via transition plans.

-- 
Joel

Re: Features vs infrastructure (was Re: Tracepoint support in Cygnus GDB ?)

[Andrew Cagney]

> That's an interesting question.  Thinking about that, and comparing with
>> GCC experience, I'd say that in general it's just extremely difficult to
>> get infrastructural work accomplished in a small group or small company;
>> you'd have to have a sufficiently large and/or well-funded group that
>> the time taken by infrastructure does not affect the group's overall
>> schedule.

That argument only serves as an excuse for poor engineering.

> Unfortunately, my own experience with large companies is that they have
> very very very controlled budgets and that it's hard to explain to a
> manager who only cares about functionality that such and such
> infrastructure work is going to save some bucks later if the rework is
> going to take more than, say, a few days.
> 
> In my opinion, unless you find a talented developper willing to invest
> a lot of his own time, the right way to go is incremental improvements
> via transition plans.

Yes.

Andrew