Changing .dynstr to be more C++ friendly?

Changing .dynstr to be more C++ friendly?

[Chris Kirby]

We have a large C++ embedded project where we have some large (~7MB) .dynstr sections that are taking up a lot of memory.  I used objdump to dump out all of the dynamic strings, and I noticed two C++ features that contributed to the large string size:
1) namespaces and class names are at the front of the mangled symbol name
2) templates with multiple instantiations

Looking at the strings, I noticed that there are possibilities for compression:
- For functions in the same namespace / class (#1), they tended to have a common prefix.
- For instantiated templates (#2), they tended to end with the same suffix since they implement similar functions.

My thinking was that we could change the definition of an Elf symbol to not contain a single name, but rather N distinct name components (3 in this case).  Using the example above, we could use the first for the prefix, the second for the unique portion, and the third for the suffix.  The hope would be that the prefix and suffix strings would be shared across multiple symbols.

For C++ applications I think the compression possible would be fairly high.  How difficult do you think it would it be to add these additional strings to the elf symbol table entries?

- Chris Kirby 

MIPS gas doesn't set SH_FLAG

[Brett Porter]

Here is a rather simple program:

        .section .debug_frame,0x7000001e,0x8000000,0,4
        .byte   0

I'm trying to assemble it on an IRIX64 6.5 system with a recently
released gas and with gas built from snapshot sources dated today.

That section flag bit is for SHF_MIPS_NOSTRIP.  elfdump shows that
it isn't set in the output file.  It is set when assembled with the
IRIX assembler. I bumped into this one -- didn't go hunting for
other flags...

Is this a simple missing piece or part of a bigger, messier puzzle,
or ???

Thanks,
Brett

Re: MIPS gas doesn't set SH_FLAG

[Nick Clifton]

Hi Brett, Hi Eric, Hi Thiemo,

 > Brett Porter wrote:
> Here is a rather simple program:
> 
>         .section .debug_frame,0x7000001e,0x8000000,0,4
>         .byte   0
> 
> I'm trying to assemble it on an IRIX64 6.5 system with a recently
> released gas and with gas built from snapshot sources dated today.
> 
> That section flag bit is for SHF_MIPS_NOSTRIP.  elfdump shows that
> it isn't set in the output file.  It is set when assembled with the
> IRIX assembler. I bumped into this one -- didn't go hunting for
> other flags...
> 
> Is this a simple missing piece or part of a bigger, messier puzzle,
> or ???

It is a bigger, messier puzzle in that no MIPS specific ELF section 
header flags can (currently) be set via the .section directive.

The MIPS port of GAS does have special code to handle setting other 
section header flags via the .section directive, so I am assuming that 
the intent was that MIPS specific flags would be supported too.

Hence I am offering up the attached patch as a possible solution.  Eric, 
Thiemo - what do you think ?  Tested with no regressions on mips-elf, 
mips64-linux-gnu and mipsisa32el-linux-gnu targets.

Note - as an aside I noticed that readelf does not currently decode and 
display the values of MIPS specific section flags - it probably should 
do this, if somebody wanted to write the code...

Cheers
   Nick

bfd/ChangeLog
2005-02-28  Nick Clifton  <nickc@redhat.com>

	* elfxx-mips.c (_bfd_mips_elf_fake_sections): Copy user specified,
	MIPS specific section flags into the fake section's header.

gas/ChangeLog
2005-02-28  Nick Clifton  <nickc@redhat.com>

	* config/tc-mips.c (mips_elf_section_flags): New function.  Ensure
	that user specified MIPS specific section flags are permitted in
	the section header.
	* config/tc-mips.h (md_elf_section_flags): Define.  Use
	mips_elf_section_flags.

Re: MIPS gas doesn't set SH_FLAG

[Eric Christopher]

> Hence I am offering up the attached patch as a possible solution.  Eric, 
> Thiemo - what do you think ?  Tested with no regressions on mips-elf, 
> mips64-linux-gnu and mipsisa32el-linux-gnu targets.

> 
> bfd/ChangeLog
> 2005-02-28  Nick Clifton  <nickc@redhat.com>
> 
> 	* elfxx-mips.c (_bfd_mips_elf_fake_sections): Copy user specified,
> 	MIPS specific section flags into the fake section's header.
> 
> gas/ChangeLog
> 2005-02-28  Nick Clifton  <nickc@redhat.com>
> 
> 	* config/tc-mips.c (mips_elf_section_flags): New function.  Ensure
> 	that user specified MIPS specific section flags are permitted in
> 	the section header.
> 	* config/tc-mips.h (md_elf_section_flags): Define.  Use
> 	mips_elf_section_flags.

ChangeLogs look good, how about that patch? :)

-eric

Re: MIPS gas doesn't set SH_FLAG

[Nick Clifton]

[-- Attachment #1: Type: text/plain, Size: 175 bytes --]

Hi Eric, Hi Brett, Hi Thiemo,

> ChangeLogs look good, how about that patch? :)


I could have sworn that I attached the file.  Oh well... Here it is again.

Cheers
   Nick



[-- Attachment #2: mips.patch --]
[-- Type: text/plain, Size: 2999 bytes --]

Index: bfd/elfxx-mips.c
===================================================================
RCS file: /cvs/src/src/bfd/elfxx-mips.c,v
retrieving revision 1.124
diff -c -3 -p -r1.124 elfxx-mips.c
*** bfd/elfxx-mips.c	23 Feb 2005 11:39:14 -0000	1.124
--- bfd/elfxx-mips.c	28 Feb 2005 17:01:12 -0000
*************** _bfd_mips_elf_fake_sections (bfd *abfd, 
*** 4578,4583 ****
--- 4578,4587 ----
  
    name = bfd_get_section_name (abfd, sec);
  
+   /* If the user has specified any MIPS specific
+      flags then make sure that they are propogated.  */
+   hdr->sh_flags |= (sec->flags & (SHF_MASKOS | SHF_MASKPROC));
+ 
    if (strcmp (name, ".liblist") == 0)
      {
        hdr->sh_type = SHT_MIPS_LIBLIST;
Index: gas/config/tc-mips.c
===================================================================
RCS file: /cvs/src/src/gas/config/tc-mips.c,v
retrieving revision 1.287
diff -c -3 -p -r1.287 tc-mips.c
*** gas/config/tc-mips.c	23 Feb 2005 12:28:04 -0000	1.287
--- gas/config/tc-mips.c	28 Feb 2005 17:01:16 -0000
*************** s_change_section (int ignore ATTRIBUTE_U
*** 11385,11398 ****
--- 11385,11401 ----
      section_type = get_absolute_expression ();
    else
      section_type = 0;
+ 
    if (*input_line_pointer++ == ',')
      section_flag = get_absolute_expression ();
    else
      section_flag = 0;
+ 
    if (*input_line_pointer++ == ',')
      section_entry_size = get_absolute_expression ();
    else
      section_entry_size = 0;
+ 
    if (*input_line_pointer++ == ',')
      section_alignment = get_absolute_expression ();
    else
*************** s_change_section (int ignore ATTRIBUTE_U
*** 11424,11429 ****
--- 11427,11447 ----
  #endif /* OBJ_ELF */
  }
  
+ #ifdef OBJ_ELF
+ /* Ensure that any MIPS specific flags in ATTR are passed
+    on to FLAGS.  */
+ 
+ int
+ mips_elf_section_flags (flagword flags, int attr)
+ {
+   /* Strictly speaking we ought to only select those bits in the OS and PROC
+      specific sections of the flags which we know to have defined meanings
+      for the MIPS.  But using the masks is easier and will prevent stange
+      bugs when new flags are added but this code is not updated.  */
+   return flags | (attr & (SHF_MASKPROC | SHF_MASKOS));
+ }
+ #endif /* OBJ_ELF */
+ 
  void
  mips_enable_auto_align (void)
  {
Index: gas/config/tc-mips.h
===================================================================
RCS file: /cvs/src/src/gas/config/tc-mips.h,v
retrieving revision 1.35
diff -c -3 -p -r1.35 tc-mips.h
*** gas/config/tc-mips.h	29 Apr 2004 05:14:22 -0000	1.35
--- gas/config/tc-mips.h	28 Feb 2005 17:01:16 -0000
*************** extern unsigned long mips_cprmask[4];
*** 150,155 ****
--- 150,158 ----
  #define elf_tc_final_processing mips_elf_final_processing
  extern void mips_elf_final_processing (void);
  
+ #define md_elf_section_flags(FLAGS, ATTR, TYPE)	\
+            mips_elf_section_flags ((FLAGS), (ATTR))
+ extern int mips_elf_section_flags (flagword, int);
  #endif
  
  extern void md_mips_end (void);

Re: MIPS gas doesn't set SH_FLAG

[Brett Porter]

Thanks Nick.  I've applied your patch and done a little testing
-- it looks good.

Brett

Re: MIPS gas doesn't set SH_FLAG

[Eric Christopher]

On Tue, 2005-03-01 at 09:59 +0000, Nick Clifton wrote:
> Hi Eric, Hi Brett, Hi Thiemo,
> 
> > ChangeLogs look good, how about that patch? :)
> 
> 
> I could have sworn that I attached the file.  Oh well... Here it is again.

Sure. This is great, thanks.

-eric

Re: Changing .dynstr to be more C++ friendly?

[Alan Modra]

On Wed, Feb 23, 2005 at 02:22:50PM -0500, Chris Kirby wrote:
> We have a large C++ embedded project

Personally, I don't think you should mention "embedded" and "C++" in the
same breath.

> My thinking was that we could change the definition of an Elf symbol
> to not contain a single name, but rather N distinct name components
> (3 in this case). 

I think you underestimate the scope of the change you are proposing.
This would require changes to all consumers of ELF files, which means
more than just the assembler and linker.  Debuggers, ld.so, etc.

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre

Re: Changing .dynstr to be more C++ friendly?

[Chris Kirby]

At 12:17 PM 2/24/2005 +1030, Alan Modra wrote:
>On Wed, Feb 23, 2005 at 02:22:50PM -0500, Chris Kirby wrote:
>> My thinking was that we could change the definition of an Elf symbol
>> to not contain a single name, but rather N distinct name components
>> (3 in this case). 
>
>I think you underestimate the scope of the change you are proposing.
>This would require changes to all consumers of ELF files, which means
>more than just the assembler and linker.  Debuggers, ld.so, etc.

I was hoping that they all used bfd or liberty so we would only have to change it there.  If everyone has there own code to interact with ELF files, then I agree it would be too large of a change to be feasible right now.  If ELF is ever revised, it would be nice to consider ways to compress the dynamic strings for languages other than just C. 

For our embedded application, I am wondering how feasible it would be to change the symbol names to something shorter.  As long as the application and the shared library are in agreement in what the symbol names are, it should not matter what they are.  This would allow us to come up with a more compact naming scheme than the current C++ name mangler.  I could imagine a post processing step after the application and libraries are built where the current names in use are found, converted to a shorter representation, and then replaced.

- Chris

Re: Changing .dynstr to be more C++ friendly?

[Daniel Jacobowitz]

On Thu, Feb 24, 2005 at 09:52:12AM -0500, Chris Kirby wrote:
> At 12:17 PM 2/24/2005 +1030, Alan Modra wrote:
> >On Wed, Feb 23, 2005 at 02:22:50PM -0500, Chris Kirby wrote:
> >> My thinking was that we could change the definition of an Elf symbol
> >> to not contain a single name, but rather N distinct name components
> >> (3 in this case). 
> >
> >I think you underestimate the scope of the change you are proposing.
> >This would require changes to all consumers of ELF files, which means
> >more than just the assembler and linker.  Debuggers, ld.so, etc.
> 
> I was hoping that they all used bfd or liberty so we would only have
> to change it there.  If everyone has there own code to interact with
> ELF files, then I agree it would be too large of a change to be
> feasible right now.  If ELF is ever revised, it would be nice to
> consider ways to compress the dynamic strings for languages other
> than just C.
> 
> For our embedded application, I am wondering how feasible it would be
> to change the symbol names to something shorter.  As long as the
> application and the shared library are in agreement in what the
> symbol names are, it should not matter what they are.  This would
> allow us to come up with a more compact naming scheme than the
> current C++ name mangler.  I could imagine a post processing step
> after the application and libraries are built where the current names
> in use are found, converted to a shorter representation, and then
> replaced.

I don't know if this will interfere with any of the C++ support library
functions, but you can probably make it work.  It's not hard to rename
symbols in an ELF file.

-- 
Daniel Jacobowitz
CodeSourcery, LLC