How to reclaim .rela.dyn entries for symbols that become hidden?

How to reclaim .rela.dyn entries for symbols that become hidden?

[Carlos O'Donell]

binutils,

I was cleaning up the hppa plabel emit support, and I noticed that I
have a number of extra R_PARISC_NONE relocs in .rela.dyn. I decided that
I might try to reclaim the wasted space.

An example executable might look like this:

Relocation section '.rela.dyn' at offset 0x2fc contains 11 entries:
 Offset     Info    Type            Sym.Value  Sym. Name + Addend
00000000  00000000 R_PARISC_NONE                                00000000
00000000  00000000 R_PARISC_NONE                                00000000
00000000  00000000 R_PARISC_NONE                                00000000
00000000  00000000 R_PARISC_NONE                                00000000
00010420  00000542 R_PARISC_PLABEL21 0001054c   main + 0
00010424  00000546 R_PARISC_PLABEL14 0001054c   main + 0
00010430  00000442 R_PARISC_PLABEL21 000106e0   __libc_csu_init + 0
00010434  00000446 R_PARISC_PLABEL14 000106e0   __libc_csu_init + 0
00010438  00000242 R_PARISC_PLABEL21 00010770   __libc_csu_fini + 0
0001043c  00000246 R_PARISC_PLABEL14 00010770   __libc_csu_fini + 0
000108b4  00000841 R_PARISC_PLABEL32 00000000   _Jv_RegisterClasses + 0

Lets not worry about wether this is optimal (It's not and the fix is to
change glibc/sysdeps/hppa/start.S to use plabel words so these become 1
reloc each).

Symbol table '.dynsym' contains 11 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
     0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __fini_array_end
     2: 00010770   132 FUNC    GLOBAL DEFAULT   11 __libc_csu_fini
     3: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __fini_array_start
     4: 000106e0   144 FUNC    GLOBAL DEFAULT   11 __libc_csu_init
     5: 0001054c    28 FUNC    GLOBAL DEFAULT   11 main
     6: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __init_array_end
     7: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __init_array_start
     8: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
     9: 00000000   368 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2 (2)
    10: 00010884     4 FUNC    WEAK   DEFAULT   11 __gmon_start__

Let us also not worry why --export-dynamic is the default on hppa.
Though I may raise such a discussion later, under the flag of "function
descriptor comparisons."

The four R_PARISC_NONE entries seem to come from somewhere? There are
infact 4 hidden symbols. In check_relocs they are not hidden, but space
is *not* allocated to them. However, allocate_dynrelocs does eventually
see they are dynamic and records that (bfd_elf_link_record_dynamic_symbol).
Does this call have any side effects that I might not realize?
In relocate_section though the visibility is hidden.

Are these hidden symbols the cuprits of the extra relocs in .rela.dyn,
or should I be looking elsewhere?

c.

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Daniel Jacobowitz]

On Thu, Jun 30, 2005 at 12:59:22PM -0400, Carlos O'Donell wrote:
> The four R_PARISC_NONE entries seem to come from somewhere? There are
> infact 4 hidden symbols. In check_relocs they are not hidden, but space
> is *not* allocated to them. However, allocate_dynrelocs does eventually
> see they are dynamic and records that (bfd_elf_link_record_dynamic_symbol).
> Does this call have any side effects that I might not realize?
> In relocate_section though the visibility is hidden.

No, it shouldn't have extra side effects.

> Are these hidden symbols the cuprits of the extra relocs in .rela.dyn,
> or should I be looking elsewhere?

Somewere, you're allocating the space for them.  Probably this means
that you're counting things incorrectly, or too early.  Is
allocate_dynrelocs allocating space for them despite the fact that they
are hidden?  If not, where's it come from?

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[John David Anglin]

> Lets not worry about wether this is optimal (It's not and the fix is to
> change glibc/sysdeps/hppa/start.S to use plabel words so these become 1
> reloc each).
> 
> Symbol table '.dynsym' contains 11 entries:
>    Num:    Value  Size Type    Bind   Vis      Ndx Name
>      0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
>      1: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __fini_array_end
>      2: 00010770   132 FUNC    GLOBAL DEFAULT   11 __libc_csu_fini
>      3: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __fini_array_start
>      4: 000106e0   144 FUNC    GLOBAL DEFAULT   11 __libc_csu_init
>      5: 0001054c    28 FUNC    GLOBAL DEFAULT   11 main
>      6: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __init_array_end
>      7: 00020b18     0 OBJECT  LOCAL  HIDDEN  ABS __init_array_start
>      8: 00000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
>      9: 00000000   368 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2 (2)
>     10: 00010884     4 FUNC    WEAK   DEFAULT   11 __gmon_start__
> 
> Let us also not worry why --export-dynamic is the default on hppa.
> Though I may raise such a discussion later, under the flag of "function
> descriptor comparisons."

That's interesting.

> Are these hidden symbols the cuprits of the extra relocs in .rela.dyn,
> or should I be looking elsewhere?

That seems the most likely explanation.  I'm not sure but I don't
think we have these in 2.15.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Carlos O'Donell]

On Thu, Jun 30, 2005 at 01:30:57PM -0400, Daniel Jacobowitz wrote:
> On Thu, Jun 30, 2005 at 12:59:22PM -0400, Carlos O'Donell wrote:
> > The four R_PARISC_NONE entries seem to come from somewhere? There are
> > infact 4 hidden symbols. In check_relocs they are not hidden, but space
> > is *not* allocated to them. However, allocate_dynrelocs does eventually
> > see they are dynamic and records that (bfd_elf_link_record_dynamic_symbol).
> > Does this call have any side effects that I might not realize?
> > In relocate_section though the visibility is hidden.
> 
> No, it shouldn't have extra side effects.

Okay.
 
> > Are these hidden symbols the cuprits of the extra relocs in .rela.dyn,
> > or should I be looking elsewhere?
> 
> Somewere, you're allocating the space for them.  Probably this means
> that you're counting things incorrectly, or too early.  Is
> allocate_dynrelocs allocating space for them despite the fact that they
> are hidden?  If not, where's it come from.

I guess I have to go really slowly in the debugger and watch the count.
I wish I knew where I was allocating space for them. 

In check_relocs we do some early allocation:
1591: (elf32_hppa_check_relocs): sym main needs a reloc section.
1612: (elf32_hppa_check_relocs): Allocate reloc section .rela.text
1659: (elf32_hppa_check_relocs): Space allocated.
1591: (elf32_hppa_check_relocs): sym __libc_csu_init needs a reloc section.
1659: (elf32_hppa_check_relocs): Space allocated.
1591: (elf32_hppa_check_relocs): sym __libc_csu_fini needs a reloc section.
1659: (elf32_hppa_check_relocs): Space allocated.
1591: (elf32_hppa_check_relocs): sym $global$ needs a reloc section.
1659: (elf32_hppa_check_relocs): Space allocated.
1591: (elf32_hppa_check_relocs): sym _Jv_RegisterClasses needs a reloc section.
1612: (elf32_hppa_check_relocs): Allocate reloc section .rela.rodata.cst4
1659: (elf32_hppa_check_relocs): Space allocated.

'$global$' is allocated, but killed later.

Then in allocate_dynrelocs:
2184: (allocate_dynrelocs): Record dynamic symbol __fini_array_end
2215: (allocate_dynrelocs): dyn_relocs == NULL, sym __fini_array_end
2284: (allocate_dynrelocs): Keep sym __libc_csu_fini
2301: (allocate_dynrelocs): sym __libc_csu_fini, allocated reloc in .rela.text
2290: (allocate_dynrelocs): Kill sym $global$
2184: (allocate_dynrelocs): Record dynamic symbol __fini_array_start
2215: (allocate_dynrelocs): dyn_relocs == NULL, sym __fini_array_start
2284: (allocate_dynrelocs): Keep sym __libc_csu_init
2301: (allocate_dynrelocs): sym __libc_csu_init, allocated reloc in .rela.text
2284: (allocate_dynrelocs): Keep sym main
2301: (allocate_dynrelocs): sym main, allocated reloc in .rela.text
2184: (allocate_dynrelocs): Record dynamic symbol __init_array_end
2215: (allocate_dynrelocs): dyn_relocs == NULL, sym __init_array_end
2184: (allocate_dynrelocs): Record dynamic symbol _GLOBAL_OFFSET_TABLE_
2215: (allocate_dynrelocs): dyn_relocs == NULL, sym _GLOBAL_OFFSET_TABLE_
2184: (allocate_dynrelocs): Record dynamic symbol __init_array_start
2215: (allocate_dynrelocs): dyn_relocs == NULL, sym __init_array_start
2284: (allocate_dynrelocs): Keep sym _Jv_RegisterClasses
2301: (allocate_dynrelocs): sym _Jv_RegisterClasses, allocated reloc in .rela.rodata.cst4

The hidden symbols are recorded but not allocated space for a reloc.
Time to dig more. Perhaps I've missed an allocation somewhere.

The reason I figure it's these symbols is because in the past we had:

Relocation section '.rela.dyn' at offset 0x328 contains 5 entries:
 Offset     Info    Type            Sym.Value  Sym. Name + Addend
00020a94  00000101 R_PARISC_DIR32    00020940   __fini_array_end + 0
00020a98  00000301 R_PARISC_DIR32    00020940   __fini_array_start + 0
00020a9c  00000601 R_PARISC_DIR32    00020940   __init_array_end + 0
00020aa0  00000701 R_PARISC_DIR32    00020a80   _GLOBAL_OFFSET_TABLE_ + 0
00020aa4  00000801 R_PARISC_DIR32    00020940   __init_array_start + 0

Now they are all hidden or removed (_GOT_).

c.

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Carlos O'Donell]

On Thu, Jun 30, 2005 at 04:28:55PM -0400, Carlos O'Donell wrote:
> > Somewere, you're allocating the space for them.  Probably this means
> > that you're counting things incorrectly, or too early.  Is
> > allocate_dynrelocs allocating space for them despite the fact that they
> > are hidden?  If not, where's it come from.

*shrug*

2312: (allocate_dynrelocs): sym __libc_csu_fini, alloc reloc in .rela.text, size (2 * 12)
2314: (allocate_dynrelocs): sreloc .rela.text now size 24
2312: (allocate_dynrelocs): sym __libc_csu_init, alloc reloc in .rela.text, size (2 * 12)
2314: (allocate_dynrelocs): sreloc .rela.text now size 48
2312: (allocate_dynrelocs): sym main, alloc reloc in .rela.text, size (2 * 12)
2314: (allocate_dynrelocs): sreloc .rela.text now size 72
2312: (allocate_dynrelocs): sym _Jv_RegisterClasses, alloc reloc in .rela.rodata.cst4, size (1 * 12)
2314: (allocate_dynrelocs): sreloc .rela.rodata.cst4 now size 12

2590: (elf32_hppa_size_dynamic_sections): Allocate sec .rela.text size 72 at 0x244190
2590: (elf32_hppa_size_dynamic_sections): Allocate sec .rela.rodata.cst4 size 12 at 0x2441d8
2590: (elf32_hppa_size_dynamic_sections): Allocate sec .plt size 76 at 0x2441e8
2590: (elf32_hppa_size_dynamic_sections): Allocate sec .rela.plt size 72 at 0x244238
2590: (elf32_hppa_size_dynamic_sections): Allocate sec .got size 40 at 0x244280
2590: (elf32_hppa_size_dynamic_sections): Allocate sec .rela.got size 48 at 0x2442a8

3809: (elf32_hppa_relocate_section): Found output .rela.dyn, size 132

elf_link_sort_relocs: .rela.dyn count = 11 (132 / 12)

I count the symbols and I *swear* I only allocated room for 72+12 = 84.
Yet the count at the start of relocate_section is 132.

c.

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Carlos O'Donell]

On Thu, Jun 30, 2005 at 05:41:36PM -0400, Carlos O'Donell wrote:
> On Thu, Jun 30, 2005 at 04:28:55PM -0400, Carlos O'Donell wrote:
> > > Somewere, you're allocating the space for them.  Probably this means
> > > that you're counting things incorrectly, or too early.  Is
> > > allocate_dynrelocs allocating space for them despite the fact that they
> > > are hidden?  If not, where's it come from.
> 
> *shrug*

I forgot .rela.got becomes part of the .rela.dyn section based on the
linker script. This accounts for the other 48 bytes, or 4 relocs that
appear in that section.

The problem is that during allocate_dynrelocs I only know it's
undefined, and it's not yet marked dynamic, and it doesn't have the
visibility set yet either. So I record it as a dynamic symbol, and
allocate it some space.

Later on it turns out to be hidden, and the space isn't needed, the
reloc isn't emitted.

I guess I somehow have to move this logic further back.

c.

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Alan Modra]

On Fri, Jul 01, 2005 at 05:13:13PM -0400, Carlos O'Donell wrote:
> I forgot .rela.got becomes part of the .rela.dyn section based on the
> linker script. This accounts for the other 48 bytes, or 4 relocs that
> appear in that section.

Yeah, it pays to debug this sort of problem with -z nocombreloc

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Carlos O'Donell]

On Mon, Jul 04, 2005 at 11:48:30AM +0930, Alan Modra wrote:
> On Fri, Jul 01, 2005 at 05:13:13PM -0400, Carlos O'Donell wrote:
> > I forgot .rela.got becomes part of the .rela.dyn section based on the
> > linker script. This accounts for the other 48 bytes, or 4 relocs that
> > appear in that section.
> 
> Yeah, it pays to debug this sort of problem with -z nocombreloc

I'm still stuck. At the point in allocate_dynrelocs the undefined
symbol is not marked dynamic, or hidden for matter. I am forced to
allocate space.

Now the question I don't clearly understand is: After having
allocated that space, can I go back and change the section size in
relocate_section, like this:

      sec = htab->sgot;
      sec->size -= GOT_ENTRY_SIZE;

And thus effect the final writeout of the section? It would seem like
the thing to do if the symbol becomes hidden and remains undefined
e.g. change the space allocated for the reloc.

Second question: Where do __init_array_start, __init_array_end,
__fini_array_start, and __fini_array_end resolve to?

Are these provided by the linker's internal script?

Thanks for all the newbie help.

c.

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Daniel Jacobowitz]

On Mon, Jul 04, 2005 at 11:40:36AM -0400, Carlos O'Donell wrote:
> On Mon, Jul 04, 2005 at 11:48:30AM +0930, Alan Modra wrote:
> > On Fri, Jul 01, 2005 at 05:13:13PM -0400, Carlos O'Donell wrote:
> > > I forgot .rela.got becomes part of the .rela.dyn section based on the
> > > linker script. This accounts for the other 48 bytes, or 4 relocs that
> > > appear in that section.
> > 
> > Yeah, it pays to debug this sort of problem with -z nocombreloc
> 
> I'm still stuck. At the point in allocate_dynrelocs the undefined
> symbol is not marked dynamic, or hidden for matter. I am forced to
> allocate space.

How are symbols becoming hidden after allocate_dynrelocs?  Where does
it happen?

> Now the question I don't clearly understand is: After having
> allocated that space, can I go back and change the section size in
> relocate_section, like this:
> 
>       sec = htab->sgot;
>       sec->size -= GOT_ENTRY_SIZE;

No.  We've fixed file offsets after allocation.

> Second question: Where do __init_array_start, __init_array_end,
> __fini_array_start, and __fini_array_end resolve to?
> 
> Are these provided by the linker's internal script?

ld/emultempl/elf32.em:
      gld${EMULATION_NAME}_provide_bound_symbols (".init_array",
                                                  "__init_array_start",
                                                  "__init_array_end");

-- 
Daniel Jacobowitz
CodeSourcery, LLC

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[John David Anglin]

> > Are these provided by the linker's internal script?
> 
> ld/emultempl/elf32.em:
>       gld${EMULATION_NAME}_provide_bound_symbols (".init_array",
>                                                   "__init_array_start",
>                                                   "__init_array_end");

2005-03-22  H.J. Lu  <hongjiu.lu@intel.com>

        * emultempl/elf32.em (gld${EMULATION_NAME}_provide_bound_symbols): New
	(gld${EMULATION_NAME}_finish): Call
	gld${EMULATION_NAME}_provide_bound_symbols to provide
	__preinit_array_start, __preinit_array_end, __init_array_start,
	__init_array_end, __fini_array_start and __fini_array_end.

	* scripttempl/elf.sc: Don't provide __preinit_array_start,
	__preinit_array_end, __init_array_start, __init_array_end,
	__fini_array_start nor __fini_array_end.

static void
gld${EMULATION_NAME}_provide_bound_symbols (const char *sec,
                                            const char *start,
					    const char *end)
										{
  asection *s = bfd_get_section_by_name (output_bfd, sec);
  if (s && bfd_section_removed_from_list (output_bfd, s))
    s = NULL;
  _bfd_elf_provide_section_bound_symbols (&link_info, s, start, end);
}

2005-05-04  H.J. Lu  <hongjiu.lu@intel.com>

        * bfd-in.h (_bfd_elf_provide_section_bound_symbols): New.
	* bfd-in2.h: Regenerated.

	* elflink.c (bfd_elf_set_symbol): New.
	(_bfd_elf_provide_symbol): Call it.
	(_bfd_elf_provide_section_bound_symbols): New.
					`
Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: How to reclaim .rela.dyn entries for symbols that become hidden?

[Carlos O'Donell]

On Mon, Jul 04, 2005 at 11:43:32AM -0400, Daniel Jacobowitz wrote:
> > I'm still stuck. At the point in allocate_dynrelocs the undefined
> > symbol is not marked dynamic, or hidden for matter. I am forced to
> > allocate space.
> 
> How are symbols becoming hidden after allocate_dynrelocs?  Where does
> it happen?

I'm trying to track them down.

In allocate_dynrelocs I see them like this:

2178: (allocate_dynrelocs): Dumping symbol
397: (DBG_DUMP): Dumping...
        info->shared 0
        sym __init_array_start
        hh 0x206798
        hh->non_plabel_ref  0
        hh->plabel 0
        hh->eh.dynindx  0xffffffff
        hh->eh.non_got_ref 0
        hh->eh.def_dynamic 0
        hh->eh.def_regular 0
        hh->eh.root.type == bfd_link_hash_undefined 1
        hh->eh.root.type == bfd_link_hash_undefweak 0
        ELF_ST_VISIBILITY (hh->eh.other) == STV_DEFAULT 1
        SYMBOL_CALLS_LOCAL (info, &hh->eh) 0
2204: (allocate_dynrelocs): Record dynamic symbol __init_array_start

Code in allocate_dynrelocs to accounts for them:

      /* Make sure this symbol is output as a dynamic symbol.
         Undefined weak syms won't yet be marked as dynamic.  */
      if (eh->dynindx == -1
          && !eh->forced_local
          && eh->type != STT_PARISC_MILLI)
        {
          if (! bfd_elf_link_record_dynamic_symbol (info, eh))
            return FALSE;
        }

After record dynamic symbol, it has a dynindx, and the code allocates
space in the srelgot section.

In the executable they appear in the .dynsyms but their visibility is
hidden, and since they are also local the code does a bfd_put_32 to
fixup the relocation, and the reloc space is left unused.

In relocate_section they appear like this:

3966: (elf32_hppa_relocate_section): Dumping r_type 34
397: (DBG_DUMP): Dumping...
        info->shared 0
        sym __init_array_start
        hh 0x206798
        hh->non_plabel_ref  0
        hh->plabel 0
        hh->eh.dynindx  0x7
        hh->eh.non_got_ref 0
        hh->eh.def_dynamic 0
        hh->eh.def_regular 1
        hh->eh.root.type == bfd_link_hash_undefined 0
        hh->eh.root.type == bfd_link_hash_undefweak 0
        ELF_ST_VISIBILITY (hh->eh.other) == STV_DEFAULT 0
        SYMBOL_CALLS_LOCAL (info, &hh->eh) 1
4004: (elf32_hppa_relocate_section): do_got = 1;
4056: (elf32_hppa_relocate_section): bfd_put_32

It's not my job to set the visibility, or to remove the undefined-edness
of the symbol, that's done by the linker. I should probably test this on
i386 to see how the symbol looks to the backend during
allocate_dynrelocs.

I expected that a change in visibility would have called
elf_backend_hide_symbol, but that doesn't seem to be the case for these
symbols.

> > Now the question I don't clearly understand is: After having
> > allocated that space, can I go back and change the section size in
> > relocate_section, like this:
> > 
> >       sec = htab->sgot;
> >       sec->size -= GOT_ENTRY_SIZE;
> 
> No.  We've fixed file offsets after allocation.

I figured as much. I was hoping that the size variables would be used in
the last phase during the object writeout.
 
> > Second question: Where do __init_array_start, __init_array_end,
> > __fini_array_start, and __fini_array_end resolve to?
> > 
> > Are these provided by the linker's internal script?
> 
> ld/emultempl/elf32.em:
>       gld${EMULATION_NAME}_provide_bound_symbols (".init_array",
>                                                   "__init_array_start",
>                                                   "__init_array_end");

Ah, perfect, I missed that. Thanks Daniel.

c.