difference between 32 bit and 64 bit .so

difference between 32 bit and 64 bit .so

[MK]

Hello,

I am sorry if this is not related to the group but I do know of any
other place where I can find the answer to my question.

I recently saw that while building a .so dynamic library on 32 bit
linux machines (Intel processor), it is ok if you omit the -fpic flag.
You can still link these libraries correctly. However the same is not
true on 64 bit machines ie you do need the -fpic flag.

I see from here -
http://www.gentoo.org/proj/en/hardened/pic-internals.xml?style=printable
why this works on 32 bit machines. (the linker doing some magic to
effectively create "private per process" copies of the libraries with
the correct relocations etc.

I am wondering why this stops working on 64 bit machines?

Thanks a lot :)

Re: difference between 32 bit and 64 bit .so

[Alan Modra]

On Fri, Jul 23, 2010 at 12:59:55AM -0400, MK wrote:
> I recently saw that while building a .so dynamic library on 32 bit
> linux machines (Intel processor), it is ok if you omit the -fpic flag.

This is not true.  Some features of ELF shared libraries will not work
properly if you compile without -fpic or -fPIC, even on 32-bit x86.

> I am wondering why this stops working on 64 bit machines?

You'd sacrifice speed in executables if the compiler was forced to
generate code that could be used in shared libs.

It would be better to wonder why we still support non-PIC libraries on
32-bit x86.

-- 
Alan Modra
Australia Development Lab, IBM

Re: difference between 32 bit and 64 bit .so

[Ian Lance Taylor]

Alan Modra <amodra@gmail.com> writes:

>> I am wondering why this stops working on 64 bit machines?
>
> You'd sacrifice speed in executables if the compiler was forced to
> generate code that could be used in shared libs.

Another way to answer this is to observe that by default x86_64 uses the
small memory model.  This means that non-PIC code uses 32-bit PC
relative references for everything.  This means that in cases where the
main executable overrides a symbol defined in a non-PIC shared library,
the dynamic linker will only be able to resolve the reloc if the
executable and the shared library have addresses within 32 bits.  Since
the address space is 64 bits on x86_64, there is no way to guarantee
that.

I actually think that in general non-PIC shared libraries are useful,
because they shift the PIC performance penalty to startup.  That is a
poor choice in general, but a good choice for a long running program.
Unfortunately there is no way to make it work on x86_64.

Ian

Re: difference between 32 bit and 64 bit .so

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 487 bytes --]

On Friday, July 23, 2010 01:59:04 Alan Modra wrote:
> It would be better to wonder why we still support non-PIC libraries on
> 32-bit x86.

because the performance penalty is larger than many people are willing to 
accept.  there certainly are real world applications where larger memory 
footprint (due to textrels) are completely worth the speed gain.

for example, the php devs have done benchmarking in this area with apache and 
actually default to non-pic on x86.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: difference between 32 bit and 64 bit .so

[Jakub Jelinek]

On Fri, Jul 23, 2010 at 06:26:19AM -0400, Mike Frysinger wrote:
> On Friday, July 23, 2010 01:59:04 Alan Modra wrote:
> > It would be better to wonder why we still support non-PIC libraries on
> > 32-bit x86.
> 
> because the performance penalty is larger than many people are willing to 
> accept.  there certainly are real world applications where larger memory 
> footprint (due to textrels) are completely worth the speed gain.

It isn't just larger memory footprint though (and longer startup), it is
also bad for security reasons, because you need to allow execution of code
from pages that have been writable by the process.  And, that is, IMNSHO,
never worth the speed gain.

Just use x86-64 instead of x86 if speed is a problem.

	Jakub

Re: difference between 32 bit and 64 bit .so

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1266 bytes --]

On Friday, July 23, 2010 06:33:10 Jakub Jelinek wrote:
> On Fri, Jul 23, 2010 at 06:26:19AM -0400, Mike Frysinger wrote:
> > On Friday, July 23, 2010 01:59:04 Alan Modra wrote:
> > > It would be better to wonder why we still support non-PIC libraries on
> > > 32-bit x86.
> > 
> > because the performance penalty is larger than many people are willing to
> > accept.  there certainly are real world applications where larger memory
> > footprint (due to textrels) are completely worth the speed gain.
> 
> It isn't just larger memory footprint though (and longer startup), it is
> also bad for security reasons, because you need to allow execution of code
> from pages that have been writable by the process.  And, that is, IMNSHO,
> never worth the speed gain.

for many people, the security aspect is irrelevant.  if the process has the 
ability to change the protection on its pages, the fact that the ldso made 
them writable long enough to process textrels isnt terribly relevant.  that 
vector is always available for exploitation.

> Just use x86-64 instead of x86 if speed is a problem.

sure, i'll let the people running proven hardware know that they need to 
"upgrade" all of it to address issues they could care less about
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: difference between 32 bit and 64 bit .so

[Jakub Jelinek]

On Fri, Jul 23, 2010 at 06:39:17AM -0400, Mike Frysinger wrote:
> On Friday, July 23, 2010 06:33:10 Jakub Jelinek wrote:
> for many people, the security aspect is irrelevant.  if the process has the 
> ability to change the protection on its pages, the fact that the ldso made 

The textrels means that you need to give such an ability to the process.
Without them you can forbid changing protection from W to X (and mapping
WX).

> > Just use x86-64 instead of x86 if speed is a problem.
> 
> sure, i'll let the people running proven hardware know that they need to 
> "upgrade" all of it to address issues they could care less about

It has been very hard to buy a 32-bit only x86 box which would have reasonable
performance for many years now, so I don't think people need to do any
upgrade.  If they are using Atoms, they probably care about other things
than speed.

	Jakub

Re: difference between 32 bit and 64 bit .so

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1549 bytes --]

On Friday, July 23, 2010 06:45:23 Jakub Jelinek wrote:
> On Fri, Jul 23, 2010 at 06:39:17AM -0400, Mike Frysinger wrote:
> > On Friday, July 23, 2010 06:33:10 Jakub Jelinek wrote:
> > for many people, the security aspect is irrelevant.  if the process has
> > the ability to change the protection on its pages, the fact that the
> > ldso made
> 
> The textrels means that you need to give such an ability to the process.
> Without them you can forbid changing protection from W to X (and mapping
> WX).

true, but still not necessarily important to everyone

> > > Just use x86-64 instead of x86 if speed is a problem.
> > 
> > sure, i'll let the people running proven hardware know that they need to
> > "upgrade" all of it to address issues they could care less about
> 
> It has been very hard to buy a 32-bit only x86 box which would have
> reasonable performance for many years now, so I don't think people need to
> do any upgrade.  If they are using Atoms, they probably care about other
> things than speed.

like most things, it isnt about getting the absolute best.  it's about getting 
"good enough".  running older x86 hardware that can do the job it needs to 
with the faster php means people dont need to spend money & time on rebuilding 
systems.  i'm still running 32bit systems on critical websites that customers 
use everyday.  as the machines break down, they do get replaced by newer shiny 
x86_64 boxes, but if they're still working, then there's no reason at all to 
force the transition.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: difference between 32 bit and 64 bit .so

[Goswin von Brederlow]

Ian Lance Taylor <iant@google.com> writes:

> Alan Modra <amodra@gmail.com> writes:
>
>>> I am wondering why this stops working on 64 bit machines?
>>
>> You'd sacrifice speed in executables if the compiler was forced to
>> generate code that could be used in shared libs.
>
> Another way to answer this is to observe that by default x86_64 uses the
> small memory model.  This means that non-PIC code uses 32-bit PC
> relative references for everything.  This means that in cases where the
> main executable overrides a symbol defined in a non-PIC shared library,
> the dynamic linker will only be able to resolve the reloc if the
> executable and the shared library have addresses within 32 bits.  Since
> the address space is 64 bits on x86_64, there is no way to guarantee
> that.
>
> I actually think that in general non-PIC shared libraries are useful,
> because they shift the PIC performance penalty to startup.  That is a
> poor choice in general, but a good choice for a long running program.
> Unfortunately there is no way to make it work on x86_64.
>
> Ian

2 options to make it work, if non-pic is still worth it on x86_64:

1) Use 64bit PC relative references.

2) Map objects close together and give runtime (load time) errors only
in cases where the relative distance actually does exceed 32bit. How
many binaries exceed 2GB size?

MfG
        Goswin

Re: difference between 32 bit and 64 bit .so

[Ian Lance Taylor]

Goswin von Brederlow <goswin-v-b@web.de> writes:

> 2 options to make it work, if non-pic is still worth it on x86_64:
>
> 1) Use 64bit PC relative references.

This is the large model.  You can get this from gcc with
-mcmodel=large.  Of course, then you are paying a different type of
performance penalty.

> 2) Map objects close together and give runtime (load time) errors only
> in cases where the relative distance actually does exceed 32bit. How
> many binaries exceed 2GB size?

Enough.  And you also need to change the default behaviour of the
dynamic linker, since by default they won't be within 32 bits.

Ian