Re: Shared library compilation

Re: Shared library compilation

[Doug Semler]

----- Original Message -----
From: Martin v. Loewis <martin@mira.isdn.cs.tu-berlin.de>
To: <doug@seaspace.com>
Cc: <egcs@egcs.cygnus.com>
Sent: Friday, February 26, 1999 9:25 AM
Subject: Re: Shared library compilation


>> Is there a particular reason why gcc -shared calls ld with all the
>> crt*.o files?  What happens is that when I build a shared library this
>> way, even on a 900 byte .o file, the shared library is 32 K.
>> If I build the shared library with ld -shared (gnu linker, btw), it
doesn't
>> bring in these files.
>
>On ELF, this is needed so that the .ctors and .dtors sections get
>combined and referenced from a suitable .init section. Pretty much the
>same holds for the exception handling support code.
>
>Regards,
>Martin
>

Damn.   That's what I kind of thought, but I was hoping it wasn't the case.

So, if I don't have global statics in the library, it should be ok to build
it using the ld -static rather than the gcc -static?  Or am I missing
something?

The reason I am curious is:

1)  Even though the size increase is only (cough) 32 K, we have quite a few
libraries which we are thinking about going shared.
2)  Multiple instances of the crt.* files.   If the OS gets changed between
revs of one of the libraries, can we guarantee the correct version of the
file
gets linked???   EG:

    I build lib1.so and lib2.so using solaris 2.5, and build my apps
referencing
those libraries.   Later, lib2 undergoes changes, and we rebuild.  however,
in the meantime, solaris 2.5.1 comes out, changing crt1.o.   lib2.so is
built
including this file.   We release the library, and the executable references
both the old version of crt1.o and the new version of crt1.o.   Which one
gets
loaded????   (I am not really savvy on the internals of the dynamic linker
on solaris, or any other box for that matter....I only present this as a
really
basic example :).....)

3)  Again, like I said in the original email, if someone builds (by
accident,
from what you are telling me...ugh) a new version of the library using the
ld command line against an executable which was built using the gcc
version....undefined symbol errors appear when trying to run....

 I wish there was a better way :)
---
Doug Semler | doug@seaspace.com
SeaSpace Corporation | Garbage In -- Gospel Out
Least Senior Software Developer; | Minister of things to do Next Quarter
Low Man on the Totem Pole | (but will Never Be Done) DNRC O-
A closed mind is a terrible thing | Bus Error (passengers dumped)

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Re: Shared library compilation

[Paul Derbyshire]

[Much deleted]

Something is broken here, and since nobody can put their finger on a
particular piece of code being buggy, it must be a design error.

1. Code duplication, massive code duplication of the crt* stuff in
   shared libraries.
2. Inconsistent versions of the same stuff on the same system,
   consequence of problem 1.
3. Ambiguities in some situations and undefined references in others,
   consequence of problem 1.

Now I can put my finger on the design error. The error is that every shared
library is getting a copy of the C runtime and (if necessary) C++ runtime
support code, which is the code dupe that is problem 1. The solution is to
collect all the stuff being duplicated into its own shared object which is
(dynamically) linked to anything that needs it, namely all the other shared
libraries and all the programs. 
A program loads with the shared objects it uses, including the crt shared
object, which then initializes the C runtime for the program and the other
loaded shared objects, then entering the main() function, and everything is
peachy keen.

Requires changes to the linker and possibly to the C runtime objects, and
recompilation of stuff.

Consequences:

1. Code duplication goes away.
2. Painless modular upgrading: replace the crt shared object and
   everything instantly uses the new version. No inconsistent
   versions coexist. 
3. No ambiguities since only one version exists at a time on a box,
   and no undefined references since that one version does exist and
   everything knows exactly where to find it.

Or is this fix unworkable for some reason?

-- 
   .*.  "Clouds are not spheres, mountains are not cones, coastlines are not
-()  <  circles, and bark is not smooth, nor does lightning travel in a
   `*'  straight line."    -------------------------------------------------
        -- B. Mandelbrot  | http://surf.to/pgd.net
_____________________ ____|________     Paul Derbyshire     pderbysh@usa.net
Programmer & Humanist|ICQ: 10423848|

Re: Shared library compilation

[Doug Semler]

----- Original Message -----
From: Martin v. Loewis <martin@mira.isdn.cs.tu-berlin.de>
To: <doug@seaspace.com>
Cc: <egcs@egcs.cygnus.com>
Sent: Friday, February 26, 1999 9:25 AM
Subject: Re: Shared library compilation


>> Is there a particular reason why gcc -shared calls ld with all the
>> crt*.o files?  What happens is that when I build a shared library this
>> way, even on a 900 byte .o file, the shared library is 32 K.
>> If I build the shared library with ld -shared (gnu linker, btw), it
doesn't
>> bring in these files.
>
>On ELF, this is needed so that the .ctors and .dtors sections get
>combined and referenced from a suitable .init section. Pretty much the
>same holds for the exception handling support code.
>
>Regards,
>Martin
>

Damn.   That's what I kind of thought, but I was hoping it wasn't the case.

So, if I don't have global statics in the library, it should be ok to build
it using the ld -static rather than the gcc -static?  Or am I missing
something?

The reason I am curious is:

1)  Even though the size increase is only (cough) 32 K, we have quite a few
libraries which we are thinking about going shared.
2)  Multiple instances of the crt.* files.   If the OS gets changed between
revs of one of the libraries, can we guarantee the correct version of the
file
gets linked???   EG:

    I build lib1.so and lib2.so using solaris 2.5, and build my apps
referencing
those libraries.   Later, lib2 undergoes changes, and we rebuild.  however,
in the meantime, solaris 2.5.1 comes out, changing crt1.o.   lib2.so is
built
including this file.   We release the library, and the executable references
both the old version of crt1.o and the new version of crt1.o.   Which one
gets
loaded????   (I am not really savvy on the internals of the dynamic linker
on solaris, or any other box for that matter....I only present this as a
really
basic example :).....)

3)  Again, like I said in the original email, if someone builds (by
accident,
from what you are telling me...ugh) a new version of the library using the
ld command line against an executable which was built using the gcc
version....undefined symbol errors appear when trying to run....

 I wish there was a better way :)
---
Doug Semler | doug@seaspace.com
SeaSpace Corporation | Garbage In -- Gospel Out
Least Senior Software Developer; | Minister of things to do Next Quarter
Low Man on the Totem Pole | (but will Never Be Done) DNRC O-
A closed mind is a terrible thing | Bus Error (passengers dumped)

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Re: Shared library compilation

[Paul Derbyshire]

[Much deleted]

Something is broken here, and since nobody can put their finger on a
particular piece of code being buggy, it must be a design error.

1. Code duplication, massive code duplication of the crt* stuff in
   shared libraries.
2. Inconsistent versions of the same stuff on the same system,
   consequence of problem 1.
3. Ambiguities in some situations and undefined references in others,
   consequence of problem 1.

Now I can put my finger on the design error. The error is that every shared
library is getting a copy of the C runtime and (if necessary) C++ runtime
support code, which is the code dupe that is problem 1. The solution is to
collect all the stuff being duplicated into its own shared object which is
(dynamically) linked to anything that needs it, namely all the other shared
libraries and all the programs. 
A program loads with the shared objects it uses, including the crt shared
object, which then initializes the C runtime for the program and the other
loaded shared objects, then entering the main() function, and everything is
peachy keen.

Requires changes to the linker and possibly to the C runtime objects, and
recompilation of stuff.

Consequences:

1. Code duplication goes away.
2. Painless modular upgrading: replace the crt shared object and
   everything instantly uses the new version. No inconsistent
   versions coexist. 
3. No ambiguities since only one version exists at a time on a box,
   and no undefined references since that one version does exist and
   everything knows exactly where to find it.

Or is this fix unworkable for some reason?

-- 
   .*.  "Clouds are not spheres, mountains are not cones, coastlines are not
-()  <  circles, and bark is not smooth, nor does lightning travel in a
   `*'  straight line."    -------------------------------------------------
        -- B. Mandelbrot  | http://surf.to/pgd.net
_____________________ ____|________     Paul Derbyshire     pderbysh@usa.net
Programmer & Humanist|ICQ: 10423848|

Shared library compilation

[Doug Semler]

I can't find this in the faq, but maybe someone knows....

Is there a particular reason why gcc -shared calls ld with all the
crt*.o files?  What happens is that when I build a shared library this
way, even on a 900 byte .o file, the shared library is 32 K. 
If I build the shared library with ld -shared (gnu linker, btw), it doesn't
bring in these files.

There is also an (IMHO) undesirable side effect that if I link a program
with a .so built with gcc -shared, then replace it with one built with
ld -shared, the executable no longer can resolve the startup code
from the crt*.o files, since they were in the shared library, the linker
never brought them into the executable!

---
Doug Semler                       | doug@seaspace.com
SeaSpace Corporation              | Garbage In -- Gospel Out
Least Senior Software Developer;  | Minister of things to do Next Quarter
Low Man on the Totem Pole         | (but will Never Be Done) DNRC  O-
A closed mind is a terrible thing | Bus Error (passengers dumped)
  
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Re: Shared library compilation

[Martin v. Loewis]

> Is there a particular reason why gcc -shared calls ld with all the
> crt*.o files?  What happens is that when I build a shared library this
> way, even on a 900 byte .o file, the shared library is 32 K. 
> If I build the shared library with ld -shared (gnu linker, btw), it doesn't
> bring in these files.

On ELF, this is needed so that the .ctors and .dtors sections get
combined and referenced from a suitable .init section. Pretty much the
same holds for the exception handling support code.

Regards,
Martin

Re: Shared library compilation

[Paul Derbyshire]

At 05:35 AM 2/28/99 -0800, you wrote:
>Inconsistencies and Ambiguities are not a consequence of problem 1, but
>of human error (eg creating libs with ld instead of gcc at a future time)

If you can create libs with both, someone will create libs with both;
Finagle's law. The argument you gave above is also used to "justify" making
printer cables that fit equally well the right and wrong way round in their
sockets and sticking a warning note about checking the cable orientation in
the bottom right corner of the page full of legal information that nobody
ever reads :-)

>Creation of a new crt shared object on new os release comes to mind.
>Who's gonna remember to do that????  (Remember, this isn't the
>egcs crt files, but the os's)

Why bother with that? Just change the linker as needed and tell people to
compile a .so version of their os's crt.o. Or make a little script to do it
for them. Assuming they have the crt sources for their os, which they
should. Under DJGPP it's in the djlsr archive; under the various unices it
comes with the kernel and base installation package.


-- 
   .*.  "Clouds are not spheres, mountains are not cones, coastlines are not
-()  <  circles, and bark is not smooth, nor does lightning travel in a
   `*'  straight line."    -------------------------------------------------
        -- B. Mandelbrot  | http://surf.to/pgd.net
_____________________ ____|________     Paul Derbyshire     pderbysh@usa.net
Programmer & Humanist|ICQ: 10423848|

Shared library compilation

[Doug Semler]

Forgot to mention:
sparc-solaris2.5
egcs 1.1.1 release
binutils 2.9.1 (ld)
---
Doug Semler                       | doug@seaspace.com
SeaSpace Corporation              | Garbage In -- Gospel Out
Least Senior Software Developer;  | Minister of things to do Next Quarter
Low Man on the Totem Pole         | (but will Never Be Done) DNRC  O-
A closed mind is a terrible thing | Bus Error (passengers dumped)
  
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Re: Shared library compilation

[Paul Derbyshire]

At 05:35 AM 2/28/99 -0800, you wrote:
>Inconsistencies and Ambiguities are not a consequence of problem 1, but
>of human error (eg creating libs with ld instead of gcc at a future time)

If you can create libs with both, someone will create libs with both;
Finagle's law. The argument you gave above is also used to "justify" making
printer cables that fit equally well the right and wrong way round in their
sockets and sticking a warning note about checking the cable orientation in
the bottom right corner of the page full of legal information that nobody
ever reads :-)

>Creation of a new crt shared object on new os release comes to mind.
>Who's gonna remember to do that????  (Remember, this isn't the
>egcs crt files, but the os's)

Why bother with that? Just change the linker as needed and tell people to
compile a .so version of their os's crt.o. Or make a little script to do it
for them. Assuming they have the crt sources for their os, which they
should. Under DJGPP it's in the djlsr archive; under the various unices it
comes with the kernel and base installation package.


-- 
   .*.  "Clouds are not spheres, mountains are not cones, coastlines are not
-()  <  circles, and bark is not smooth, nor does lightning travel in a
   `*'  straight line."    -------------------------------------------------
        -- B. Mandelbrot  | http://surf.to/pgd.net
_____________________ ____|________     Paul Derbyshire     pderbysh@usa.net
Programmer & Humanist|ICQ: 10423848|

Re: Shared library compilation

[Martin v. Loewis]

> Is there a particular reason why gcc -shared calls ld with all the
> crt*.o files?  What happens is that when I build a shared library this
> way, even on a 900 byte .o file, the shared library is 32 K. 
> If I build the shared library with ld -shared (gnu linker, btw), it doesn't
> bring in these files.

On ELF, this is needed so that the .ctors and .dtors sections get
combined and referenced from a suitable .init section. Pretty much the
same holds for the exception handling support code.

Regards,
Martin

Shared library compilation

[Doug Semler]

Forgot to mention:
sparc-solaris2.5
egcs 1.1.1 release
binutils 2.9.1 (ld)
---
Doug Semler                       | doug@seaspace.com
SeaSpace Corporation              | Garbage In -- Gospel Out
Least Senior Software Developer;  | Minister of things to do Next Quarter
Low Man on the Totem Pole         | (but will Never Be Done) DNRC  O-
A closed mind is a terrible thing | Bus Error (passengers dumped)
  
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------

Shared library compilation

[Doug Semler]

I can't find this in the faq, but maybe someone knows....

Is there a particular reason why gcc -shared calls ld with all the
crt*.o files?  What happens is that when I build a shared library this
way, even on a 900 byte .o file, the shared library is 32 K. 
If I build the shared library with ld -shared (gnu linker, btw), it doesn't
bring in these files.

There is also an (IMHO) undesirable side effect that if I link a program
with a .so built with gcc -shared, then replace it with one built with
ld -shared, the executable no longer can resolve the startup code
from the crt*.o files, since they were in the shared library, the linker
never brought them into the executable!

---
Doug Semler                       | doug@seaspace.com
SeaSpace Corporation              | Garbage In -- Gospel Out
Least Senior Software Developer;  | Minister of things to do Next Quarter
Low Man on the Totem Pole         | (but will Never Be Done) DNRC  O-
A closed mind is a terrible thing | Bus Error (passengers dumped)
  
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS/M d---(pu) s++:- a-- C++ UILSH+++$ P--- L++ E--- W+
N++ o-- K? w--(++$) O- M-- V- PS+ !PE Y PGP t(+) 5+++ X+
R- tv+(-) b+(++) DI++++ D G e++>++++ h!>--- r% y+>+++++**
------END GEEK CODE BLOCK------