No .got section in ELF

No .got section in ELF

[yunfeng zhang]

The idea I got is about removing .got section in ELF format totally.

Before we go, let's see the limitation on the idea
1) It must be deployed on aligned segment model, such as Linux, which cs.start
= ds.start.
2) Currently, I only know how to do on x86 ELF.

Here is a typical sample in PIC model (shared library) when library want to
access its global data
    ...
    // Later code snippet template is used by gcc in almost all shared function
    // to imitate `mov %ip, %ebx'.
    call next:
next:
    pop %ebx // << A.
    ...
    movl new_offset(%ebx), %eax // << B. load global variable foo to eax.
    ...
    .global foo // << C.
OK!, to ld, offsetof(C - A) is const, and to gcc offsetof(B - A) is also
const, so to aligned segment model, new_offset = offset(C - A) - offset(B - A),
right?

Here is the new workflow
1) Using new option, such as, new option `--segment-model=aligned' to trigger
the feature.
2) Gcc creates a section (nogot) which stores data pair offsetof(B - A) and
new_offset address for every function.
3) Ld reads nogot section and update new_offset according to above formular.
4) nogot section is discarded later.
Constrast to traditional PIC code, we save an indirect memory load instruction
and shrink memory fee by avoiding to load .got section into memory. And it
seems it's fit with gcc4.5 link-time optimizer feature.

Re: No .got section in ELF

[Andrew Haley]

yunfeng zhang wrote:
> The idea I got is about removing .got section in ELF format totally.
> 
> Before we go, let's see the limitation on the idea
> 1) It must be deployed on aligned segment model, such as Linux, which cs.start
> = ds.start.
> 2) Currently, I only know how to do on x86 ELF.
> 
> Here is a typical sample in PIC model (shared library) when library want to
> access its global data
>     ...
>     // Later code snippet template is used by gcc in almost all shared function
>     // to imitate `mov %ip, %ebx'.
>     call next:
> next:
>     pop %ebx // << A.
>     ...
>     movl new_offset(%ebx), %eax // << B. load global variable foo to eax.
>     ...
>     .global foo // << C.
> OK!, to ld, offsetof(C - A) is const, and to gcc offsetof(B - A) is also
> const, so to aligned segment model, new_offset = offset(C - A) - offset(B - A),
> right?

Surely not, because in a shared library the address of the data varies.  There
are in theory many copies of the library, each one with its r/w data in a different
place.

Andrew.

Re: No .got section in ELF

[Richard Henderson]

On 11/23/2009 05:58 PM, yunfeng zhang wrote:
> next:
>      pop %ebx //<<  A.
>      ...
>      movl new_offset(%ebx), %eax //<<  B. load global variable foo to eax.
>      ...
>      .global foo //<<  C.
> OK!, to ld, offsetof(C - A) is const...

Your premise is wrong right there.  C-A is *not* a constant.  That's why 
we used a load from memory.  If it had been a constant then B would have 
been:

   leal foo@GOTOFF(%ebx), %eax


r~

Re: No .got section in ELF

[yunfeng zhang]

It seems that original limitation isn't clear or sufficient

For a sample:

// f.c
int g;
void foo(void)
{
	g = 1;
}

compile with `gcc -shared -fPIC -Wl,-soname,f.so,-Map,f.map -o f.so f.c',
according to f.map, offsetof(g - foo) in library is 0x1550 - 0x3cc = 0x1184,
however let's load the library to memory, offsetof(g - foo) in memory is
0x2c2550 - 0x2c13cc = 0x1184. That's the key of my idea!

Target OS need support --segment-model=elf-layout. In fact, Linux only simply
maps the library to memory, using segment model *defined* in ELF!

So as the previous mail, offsetof(C - A) is const!

Re: No .got section in ELF

[Richard Henderson]

On 11/25/2009 06:24 PM, yunfeng zhang wrote:
> It seems that original limitation isn't clear or sufficient
>
> For a sample:
>
> // f.c
> int g;
> void foo(void)
> {
> 	g = 1;
> }
>
> compile with `gcc -shared -fPIC -Wl,-soname,f.so,-Map,f.map -o f.so f.c'...

With -fPIC, the variable G may be overridden by another variable of the 
same name from another shared object earlier in the search path.  That 
is, the offset is *not* fixed because the final address of G may reside 
in a different .so file.

Change your program to

   static int g;

or

   extern int g __attribute__((visibility("hidden")));
   int g;

and compare the results.  In either case G is known to resolve to the 
instance present in f.so.  In either case we'll use a constant offset.

You really need to understand how ELF actually works before suggesting 
that it's broken.


r~

Re: No .got section in ELF

[yunfeng zhang]

The result is the same

#include<stdio.h>

extern int g __attribute__((visibility("hidden")));
int g;

int foo(int a, int b)
{
        g = a + b;
        printf("%x, %x", &g, foo);
        return g;
}

load and call `foo' in the library, an outputting (with vdso) is
    cc15bc, cc03fc
and open f.map
    0x15bc, 0x3fc

It shows Linux simply maps the library to memory *using* library segment layout.

Using e.cc to call it

#include <exception>
#include <typeinfo>
#include <cstddef>
#include <dlfcn.h>
#include <stdio.h>

int main(void)
{
                void* handle = dlopen("./f.so", RTLD_NOW);
                typedef int (*gso)(int, int);
                gso f;
                *(void**) (&f) = dlsym(handle, "foo");
                f(1, 2);
        return 0;
}

gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-44).

2009/11/26 Richard Henderson <rth@redhat.com>:
> On 11/25/2009 06:24 PM, yunfeng zhang wrote:
>>
>> It seems that original limitation isn't clear or sufficient
>>
>> For a sample:
>>
>> // f.c
>> int g;
>> void foo(void)
>> {
>>        g = 1;
>> }
>>
>> compile with `gcc -shared -fPIC -Wl,-soname,f.so,-Map,f.map -o f.so
>> f.c'...
>
> With -fPIC, the variable G may be overridden by another variable of the same
> name from another shared object earlier in the search path.  That is, the
> offset is *not* fixed because the final address of G may reside in a
> different .so file.
>
> Change your program to
>
>  static int g;
>
> or
>
>  extern int g __attribute__((visibility("hidden")));
>  int g;
>
> and compare the results.  In either case G is known to resolve to the
> instance present in f.so.  In either case we'll use a constant offset.
>
> You really need to understand how ELF actually works before suggesting that
> it's broken.
>
>
> r~
>

Re: No .got section in ELF

[Alexandre Oliva]

On Nov 26, 2009, yunfeng zhang <zyf.zeroos@gmail.com> wrote:

> The result is the same

But the code isn't.  See how, with hidden, we use the fixed offset.

Now remove the hidden attribute, define g also in the main program, and
see what it prints.

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Re: No .got section in ELF

[yunfeng zhang]

The rsult is also same, you go too far.

Code I want to show a fact in Linux, when a process load a library into memory,
such as 0x1000

---- foo.so 0x1000
ELF header
.text section
.data section
.bss section
...

Here data in 0x1000 and its follower have an *exact* map to foo.so in disk, you
need review my code, so which can bring an optimization is offsetof(C - A) is
const. Further, everything in data/bss section has a fixed offset to text
section.


2009/11/26 Alexandre Oliva <aoliva@redhat.com>:
> On Nov 26, 2009, yunfeng zhang <zyf.zeroos@gmail.com> wrote:
>
>> The result is the same
>
> But the code isn't.  See how, with hidden, we use the fixed offset.
>
> Now remove the hidden attribute, define g also in the main program, and
> see what it prints.
>
> --
> Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
> You must be the change you wish to see in the world. -- Gandhi
> Be Free! -- http://FSFLA.org/   FSF Latin America board member
> Free Software Evangelist      Red Hat Brazil Compiler Engineer
>

Re: No .got section in ELF

[Dave Korn]

yunfeng zhang wrote:

> have an *exact* map to foo.so in disk

  This is where your misunderstanding arises.  Just as 5+3 can add up to the
same result as 4+4, so you cannot ignore that the final mapped addresses you
are seeing add up to the same result via different routes.  Compile your
sources using --save-temps, and look at the differences in the generated
assembly files.  By looking only at the final results, you are losing
information about how those results are arrived at.

    cheers,
      DaveK

Re: No .got section in ELF

[yunfeng zhang]

You can run a.out several times

24>>a5e0<<, 249>>41c<<
11>>15e0<<, 110>>41c<<
c7>>85e0<<, c77>>41c<<
8c>>35e0<<, 8c2>>41c<<
...

Now open f.map
g = 0x15e0, foo = 0x41c

is it 5 + 3 = 4 + 4?


2009/11/27 Dave Korn <dave.korn.cygwin@googlemail.com>:
> yunfeng zhang wrote:
>
>> have an *exact* map to foo.so in disk
>
>  This is where your misunderstanding arises.  Just as 5+3 can add up to the
> same result as 4+4, so you cannot ignore that the final mapped addresses you
> are seeing add up to the same result via different routes.  Compile your
> sources using --save-temps, and look at the differences in the generated
> assembly files.  By looking only at the final results, you are losing
> information about how those results are arrived at.
>
>    cheers,
>      DaveK
>
>
>
>

Re: No .got section in ELF

[Dave Korn]

yunfeng zhang wrote:
> a.out 

> f.map

> is it 5 + 3 = 4 + 4?

  If you refuse to look at the source code that gcc generates, you will never
know what the difference is.

  I cannot make that decision for you.

    cheers,
      DaveK

Re: No .got section in ELF

[Jie Zhang]

On 11/26/2009 02:04 PM, yunfeng zhang wrote:
> The result is the same
>
> #include<stdio.h>
>
> extern int g __attribute__((visibility("hidden")));
> int g;
>
> int foo(int a, int b)
> {
>          g = a + b;
>          printf("%x, %x",&g, foo);
>          return g;
> }
>
> load and call `foo' in the library, an outputting (with vdso) is
>      cc15bc, cc03fc
> and open f.map
>      0x15bc, 0x3fc
>
> It shows Linux simply maps the library to memory *using* library segment layout.
>
> Using e.cc to call it
>
> #include<exception>
> #include<typeinfo>
> #include<cstddef>
> #include<dlfcn.h>
> #include<stdio.h>
>
> int main(void)
> {
>                  void* handle = dlopen("./f.so", RTLD_NOW);
>                  typedef int (*gso)(int, int);
>                  gso f;
>                  *(void**) (&f) = dlsym(handle, "foo");
>                  f(1, 2);
>          return 0;
> }
>
You got the bad test case. Please try the following:

$ cat f.c
#include <stdio.h>
int g;

int foo(int a, int b)
{
   g = a + b;
   printf("&g = 0x%x, foo = 0x%x\n", &g, foo);
   return g;
}

$ cat e.c
int g;
extern int foo(int a, int b);

int main(void)
{
   foo(1, 2);
   return 0;
}

$ gcc -shared -fPIC -Wl,-soname,./libf.so,-Map,f.map -o libf.so f.c
$ gcc -o e e.c -ldl -L. -lf
$ ./e
&g = 0x600a30, foo = 0x294a2614

Then comment out the "int g;" in e.c. and do the same steps as above:

$ ./e
&g = 0x58294948, foo = 0x58094614

You can see that "C-A" is *not* a constant. Your premise is wrong.


Jie

Re: No .got section in ELF

[yunfeng zhang]

Sorry! I've made a mistake! But using LD_PRELOAD to force to reposition a
variable/function from a module is violating software engineer. And the more
important is, as the result, all user *all* pay the bill for this even they
make sure they don't need the feature, such as, glibc itself.

Re: No .got section in ELF

[Alexandre Oliva]

On Nov 27, 2009, yunfeng zhang <zyf.zeroos@gmail.com> wrote:

> The rsult is also same, you go too far.

If the g in the main program didn't preempt the definition in the
library, then something is amiss in your system.

> Here data in 0x1000 and its follower have an *exact* map to foo.so in
> disk, you need review my code, so which can bring an optimization is
> offsetof(C - A) is const. Further, everything in data/bss section has
> a fixed offset to text section.

Yes, what you're proposing is already implemented, using @GOTOFF instead
of @GOT.  But we can only use @GOTOF when the symbol is nonpreemptible
and known to be defined within the same shared object.

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Re: No .got section in ELF

[Ian Lance Taylor]

yunfeng zhang <zyf.zeroos@gmail.com> writes:

> Sorry! I've made a mistake! But using LD_PRELOAD to force to reposition a
> variable/function from a module is violating software engineer. And the more
> important is, as the result, all user *all* pay the bill for this even they
> make sure they don't need the feature, such as, glibc itself.

Correct.  That is how ELF is defined and implemented.

If you want to propose a change to the ELF definition, please do that
in the appropriate forum--which is not here.  You will be told that it
has worked that way for some 20 years now, and will not be changed
today.  There are ways to change the default, which people here have
already explained.

Ian

Re: No .got section in ELF

[Yunfeng ZHANG]

Thank you! I've known to how to create a *compat* PIC library, firstly using
`-fvisibility=hidden' in compile command line to hidden all symbols, then
using `objcopy -R' to remove .got section totally! However, assemble result is
just like this

	call	__i686.get_pc_thunk.bx
	addl	$_GLOBAL_OFFSET_TABLE_, %ebx // << A.
	...
	movl	%eax, g@GOTOFF(%ebx) // << B.

It seems gcc should do better since both A and B are const, so it's safe to
remove line A as my idea shows.

Re: No .got section in ELF

[Dmitry Gorbachev]

[-- Attachment #1: Type: text/plain, Size: 40 bytes --]

Yunfeng Zhang, please try this example.

[-- Attachment #2: elf.tar.gz --]
[-- Type: application/x-gzip, Size: 502 bytes --]