[ECOS] dependancies on libextras.a extras.o

[ECOS] dependancies on libextras.a extras.o

[Andrew Lunn]

Hi Folks

It seems that the tests have a dependancy on libextras.a. Now the
tests don't link with libextras.a, they link with extras.o. 

I've been using one of the build trees clever facilities. If you put a
copy of a source file into the work tree it will compile that instead
of the one in the repository. I've been playing with an ethernet
device driver. When this is compiled its put into libextras.a. To save
some build time i've being running the make from a couple of levels
down in the work tree, so it only looks at the sources i've changed. I
type make and then make test and as expected it relinks the test
programs. After a lot of head scratching and disassembling object
files, libraries and applications i figured out i was missing
something obvious. When you make from the root of the work tree it
runs the linker on libextras.a to make extras.o. I was missing this
stage when making in the subtree so the tests were linking with the
old object code! Arrg.

Please can you remove the dependancy on libextras.a. If anybody else
makes the same mistake in the future its will be a lot more obvious
whats going on when the tests are not relinked because what the realy
depend on has not changed. 

        Thanks
                Andrew

Re: [ECOS] dependancies on libextras.a extras.o

[Bart Veer]

>>>>> "Andrew" == Andrew Lunn <andrew.lunn@ascom.ch> writes:

    Andrew> It seems that the tests have a dependancy on libextras.a.
    Andrew> Now the tests don't link with libextras.a, they link with
    Andrew> extras.o.

    Andrew> I've been using one of the build trees clever facilities.
    Andrew> If you put a copy of a source file into the work tree it
    Andrew> will compile that instead of the one in the repository.
    Andrew> I've been playing with an ethernet device driver. When
    Andrew> this is compiled its put into libextras.a. To save some
    Andrew> build time i've being running the make from a couple of
    Andrew> levels down in the work tree, so it only looks at the
    Andrew> sources i've changed. I type make and then make test and
    Andrew> as expected it relinks the test programs. After a lot of
    Andrew> head scratching and disassembling object files, libraries
    Andrew> and applications i figured out i was missing something
    Andrew> obvious. When you make from the root of the work tree it
    Andrew> runs the linker on libextras.a to make extras.o. I was
    Andrew> missing this stage when making in the subtree so the tests
    Andrew> were linking with the old object code! Arrg.

    Andrew> Please can you remove the dependancy on libextras.a. If
    Andrew> anybody else makes the same mistake in the future its will
    Andrew> be a lot more obvious whats going on when the tests are
    Andrew> not relinked because what the realy depend on has not
    Andrew> changed.

To explain what is going on here:

At link time the device drivers need to be in an object file, not in a
library. Typically device driver functions are not called directly,
only indirectly after a name lookup, so the linker decides that the
functions are never used and discards them.

But you cannot directly build a single object file from multiple
source files.

So the build system creates a library libextras.a as the various
packages are processed, and at the end it turns the whole library into
a single object file by a strange linker invocation. This can only
happen at the end of a build, hence it is done from the toplevel
makefile. The actual work happens elsewhere, currently in the common
HAL package although conceivably it might have to happen in
architecture-specific HALs in some cases. A custom build step is used,
which means that the configuration tools do not actually have any
built-in knowledge of extras.o

I do agree that not rebuilding the tests at all in these circumstances
would be better than rebuilding them incorrectly, but since a makefile
$(wildcard ) is used that is not easy. Tests should definitely be
rebuilt if libtarget.a changes. A better approach is to have an
explicit rule in each package's makefile (possibly via rules.mak) of
the form:

$(PREFIX)/lib/extras.o: $(PREFIX)/lib/libextras.a
    make -C $(BUILD_TREE) $(PREFIX)/lib/extras.o

That should cause just the right number of rebuilds. However the
actual rule for rebuilding extras.o is not in the toplevel makefile,
it is currently in the common HAL, so it would have to be:

    make -C $(BUILD_TREE)/hal/common/current $(PREFIX)/lib/extras.o

But that does not work either because rules.mak does not know which
package is responsible for building extras.o (in fact right now it
does not even know that extras.o exists), or which version of that
package is being used. So things get yet more convoluted, and I am not
sure that there is a quick solution.

In any specific installation you can add a suitable rule to rules.mak
which will work for the time being, but it may suffer bit rot over
time.

The medium/long term intention is to switch to a single makefile
approach rather than per-package makefiles (you would still have the
latter so that you can rebuild just one package's test cases, but they
would invoke the toplevel makefile with a suitable target). A single
makefile should be faster then the current approach, and provide much
better support for parallel builds. Getting it right will involve
quite a bit of work, and I am not sure when it is going to happen.

Bart