[PATCH 1/4] LD: Linker script hidden symbol support

[PATCH 1/4] LD: Linker script hidden symbol support

[Maciej W. Rozycki]

Hi,

 A change to LD seven years ago, as recorded here:

http://sourceware.org/ml/binutils/2005-08/msg00045.html

switched the scope of symbols created in linker scripts from object to 
global.  A later update added support for hidden default symbols (as in 
PROVIDE vs PROVIDE_HIDDEN) here:

http://sourceware.org/ml/binutils/2005-08/msg00054.html

however the lost ability to create hidden symbols unconditionally has 
never been restituted.

 A workaround exists, in the form of version scripts (that may also be 
embedded in regular linker scripts with the use of the VERSION keyword) 
that can force the scope of any global symbols back to object, however 
that approach was not considered back when PROVIDE_HIDDEN was defined and 
therefore I decided to follow the precedent and implement a new keyword 
that will complement the three cases already handled: assignment, PROVIDE 
and PROVIDE_HIDDEN.

 Somewhat unimaginatively I called the new keyword HIDDEN, its syntax is 
exactly like that of PROVIDE and PROVIDE_HIDDEN, and its semantics exactly 
like that of a simple assignment (compound assignments are not supported; 
they don't adjust the scope of an already defined symbol), except that the 
newly defined symbol will have its scope set to object.

 While at it I have cleaned up a hack that reached as far as BFD and that 
caused this change not to be contained within LD -- for unconditional 
symbol definitions current code overloads the "hidden" member of the 
assignment structure used internally to mean the symbol has been defined 
with the --defsym command-line option rather than in a script.

 As a result unrelated code has to handle this special case in a few 
places, including BFD, with a suitable comment attached to explain what is 
really going on there, for a questionable saving of some structure space.  
With this change I have removed the hack and there is now a separate 
"defsym" member that denotes such symbols explicitly.

 The original change also broke all MIPS SVR4 ABI binaries produced as 
with the switch from the object to the global scope the required _gp 
symbol was no longer defined such as the MIPS psABI mandated.  This will 
be handled with a follow-up change, building on functionality added here.

 I have regression-tested it successfully against a superset of Alan's set 
of targets with no problems seen.  OK to apply?

2012-08-03  Maciej W. Rozycki  <macro@codesourcery.com>

	bfd/
	* elflink.c (bfd_elf_record_link_assignment): Remove --defsym
	symbols special case.

	ld/
	* ldexp.h (etree_union): Add defsym member to the assign member
	structure.
	(exp_assign): Add hidden argument to prototype.
	* ldexp.c (exp_fold_tree_1): Use the defsym member to handle
	--defsym symbols.
	(exp_assop): Add defsym argument, initialize the defsym member
	of the assign structure.
	(exp_assign): Handle hidden symbols.
	(exp_defsym): Update to use the defsym argument to exp_assop.
	(exp_provide): Update to handle the defsym argument to exp_assop.
	* ldlex.l (HIDDEN): New token.
	* ldgram.y (HIDDEN): Likewise.
	(assignment, section): Update calls to exp_assign.
	* ldctor.c (ldctor_build_sets): Likewise.
	* mri.c (mri_format): Likewise.
	* ldlang.c (lang_insert_orphan, lang_leave_overlay): Likewise.
	(open_input_bfds): Remove --defsym symbols special case.
	* ld/emultempl/beos.em (gld_${EMULATION_NAME}_set_symbols):
	Update call to exp_assign.
	* ld/emultempl/pe.em (gld_${EMULATION_NAME}_set_symbols):
	Likewise.
	* ld/emultempl/pep.em (gld_${EMULATION_NAME}_set_symbols):
	Likewise.
	* ld/emultempl/spuelf.em (spu_place_special_section): Likewise.
	* ld/emultempl/xtensaelf.em (ld_xtensa_insert_page_offsets):
	Likewise.
	* ld.texinfo (Assigning Values to Symbols): Add HIDDEN.
	(HIDDEN): New subsection.

  Maciej

binutils-ld-hidden.diff
Index: binutils-fsf-trunk-quilt/bfd/elflink.c
===================================================================
--- binutils-fsf-trunk-quilt.orig/bfd/elflink.c	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/bfd/elflink.c	2012-08-02 22:35:36.011761953 +0100
@@ -569,7 +569,7 @@ bfd_elf_record_link_assignment (bfd *out
 
   h->def_regular = 1;
 
-  if (provide && hidden)
+  if (hidden)
     {
       bed = get_elf_backend_data (output_bfd);
       h->other = (h->other & ~ELF_ST_VISIBILITY (-1)) | STV_HIDDEN;
Index: binutils-fsf-trunk-quilt/ld/ld.texinfo
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ld.texinfo	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ld.texinfo	2012-08-02 22:35:36.011761953 +0100
@@ -3410,6 +3410,7 @@ the symbol and place it into the symbol 
 
 @menu
 * Simple Assignments::		Simple Assignments
+* HIDDEN::			HIDDEN
 * PROVIDE::			PROVIDE
 * PROVIDE_HIDDEN::		PROVIDE_HIDDEN
 * Source Code Reference::	How to use a linker script defined symbol in source code
@@ -3473,6 +3474,31 @@ the last @samp{.text} input section.  Th
 defined as the address following the @samp{.text} output section aligned
 upward to a 4 byte boundary.
 
+@node HIDDEN
+@subsection HIDDEN
+@cindex HIDDEN
+For ELF targeted ports, define a symbol that will be hidden and won't be
+exported.  The syntax is @code{HIDDEN(@var{symbol} = @var{expression})}.
+
+Here is the example from @ref{Simple Assignments}, rewritten to use
+@code{HIDDEN}:
+
+@smallexample
+HIDDEN(floating_point = 0);
+SECTIONS
+@{
+  .text :
+    @{
+      *(.text)
+      HIDDEN(_etext = .);
+    @}
+  HIDDEN(_bdata = (. + 3) & ~ 3);
+  .data : @{ *(.data) @}
+@}
+@end smallexample
+@noindent
+In this case none of the three symbols will be visible outside this module.
+
 @node PROVIDE
 @subsection PROVIDE
 @cindex PROVIDE
Index: binutils-fsf-trunk-quilt/ld/ldctor.c
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ldctor.c	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ldctor.c	2012-08-02 22:35:36.011761953 +0100
@@ -321,9 +321,11 @@ ldctor_build_sets (void)
 
       lang_add_assignment (exp_assign (".",
 				       exp_unop (ALIGN_K,
-						 exp_intop (reloc_size))));
+						 exp_intop (reloc_size)),
+				       FALSE));
       lang_add_assignment (exp_assign (p->h->root.string,
-				       exp_nameop (NAME, ".")));
+				       exp_nameop (NAME, "."),
+				       FALSE));
       lang_add_data (size, exp_intop (p->count));
 
       for (e = p->elements; e != NULL; e = e->next)
Index: binutils-fsf-trunk-quilt/ld/ldexp.c
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ldexp.c	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ldexp.c	2012-08-02 22:35:36.041394701 +0100
@@ -892,7 +892,7 @@ exp_fold_tree_1 (etree_type *tree)
 	  if (expld.result.valid_p
 	      || (expld.phase <= lang_mark_phase_enum
 		  && tree->type.node_class == etree_assign
-		  && tree->assign.hidden))
+		  && tree->assign.defsym))
 	    {
 	      if (h == NULL)
 		{
@@ -1054,6 +1054,7 @@ static etree_type *
 exp_assop (const char *dst,
 	   etree_type *src,
 	   enum node_tree_enum class,
+	   bfd_boolean defsym,
 	   bfd_boolean hidden)
 {
   etree_type *n;
@@ -1065,20 +1066,25 @@ exp_assop (const char *dst,
   n->assign.type.node_class = class;
   n->assign.src = src;
   n->assign.dst = dst;
+  n->assign.defsym = defsym;
   n->assign.hidden = hidden;
   return n;
 }
 
+/* Handle linker script assignments and HIDDEN.  */
+
 etree_type *
-exp_assign (const char *dst, etree_type *src)
+exp_assign (const char *dst, etree_type *src, bfd_boolean hidden)
 {
-  return exp_assop (dst, src, etree_assign, FALSE);
+  return exp_assop (dst, src, etree_assign, FALSE, hidden);
 }
 
+/* Handle --defsym command-line option.  */
+
 etree_type *
 exp_defsym (const char *dst, etree_type *src)
 {
-  return exp_assop (dst, src, etree_assign, TRUE);
+  return exp_assop (dst, src, etree_assign, TRUE, FALSE);
 }
 
 /* Handle PROVIDE.  */
@@ -1086,7 +1092,7 @@ exp_defsym (const char *dst, etree_type 
 etree_type *
 exp_provide (const char *dst, etree_type *src, bfd_boolean hidden)
 {
-  return exp_assop (dst, src, etree_provide, hidden);
+  return exp_assop (dst, src, etree_provide, FALSE, hidden);
 }
 
 /* Handle ASSERT.  */
Index: binutils-fsf-trunk-quilt/ld/ldexp.h
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ldexp.h	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ldexp.h	2012-08-02 22:35:36.061761927 +0100
@@ -67,6 +67,7 @@ typedef union etree_union {
     node_type type;
     const char *dst;
     union etree_union *src;
+    bfd_boolean defsym;
     bfd_boolean hidden;
   } assign;
   struct {
@@ -199,7 +200,7 @@ etree_type *exp_unop
 etree_type *exp_nameop
   (int, const char *);
 etree_type *exp_assign
-  (const char *, etree_type *);
+  (const char *, etree_type *, bfd_boolean);
 etree_type *exp_defsym
   (const char *, etree_type *);
 etree_type *exp_provide
Index: binutils-fsf-trunk-quilt/ld/ldgram.y
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ldgram.y	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ldgram.y	2012-08-02 22:35:36.061761927 +0100
@@ -146,7 +146,7 @@ static int error_index;
 %token STARTUP HLL SYSLIB FLOAT NOFLOAT NOCROSSREFS
 %token ORIGIN FILL
 %token LENGTH CREATE_OBJECT_SYMBOLS INPUT GROUP OUTPUT CONSTRUCTORS
-%token ALIGNMOD AT SUBALIGN PROVIDE PROVIDE_HIDDEN AS_NEEDED
+%token ALIGNMOD AT SUBALIGN HIDDEN PROVIDE PROVIDE_HIDDEN AS_NEEDED
 %type <token> assign_op atype attributes_opt sect_constraint
 %type <name>  filename
 %token CHIP LIST SECT ABSOLUTE  LOAD NEWLINE ENDWORD ORDER NAMEWORD ASSERT_K
@@ -759,7 +759,7 @@ end:	';' | ','
 assignment:
 		NAME '=' mustbe_exp
 		{
-		  lang_add_assignment (exp_assign ($1, $3));
+		  lang_add_assignment (exp_assign ($1, $3, FALSE));
 		}
 	|	NAME assign_op mustbe_exp
 		{
@@ -767,7 +767,11 @@ end:	';' | ','
 						   exp_binop ($2,
 							      exp_nameop (NAME,
 									  $1),
-							      $3)));
+							      $3), FALSE));
+		}
+	|	HIDDEN '(' NAME '=' mustbe_exp ')'
+		{
+		  lang_add_assignment (exp_assign ($3, $5, TRUE));
 		}
 	|	PROVIDE '(' NAME '=' mustbe_exp ')'
 		{
@@ -1085,7 +1089,7 @@ section:	NAME 		{ ldlex_expression(); }
 		opt_exp_with_type
 		{
 		  ldlex_popstate ();
-		  lang_add_assignment (exp_assign (".", $3));
+		  lang_add_assignment (exp_assign (".", $3, FALSE));
 		}
 		'{' sec_or_group_p1 '}'
 	|	INCLUDE filename
Index: binutils-fsf-trunk-quilt/ld/ldlang.c
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ldlang.c	2012-08-02 15:17:47.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ldlang.c	2012-08-02 22:35:36.061761927 +0100
@@ -3286,7 +3286,7 @@ open_input_bfds (lang_statement_union_ty
 #endif
 	  break;
 	case lang_assignment_statement_enum:
-	  if (s->assignment_statement.exp->assign.hidden)
+	  if (s->assignment_statement.exp->assign.defsym)
 	    /* This is from a --defsym on the command line.  */
 	    exp_fold_tree_no_dot (s->assignment_statement.exp);
 	  break;
@@ -7437,7 +7437,7 @@ lang_leave_overlay (etree_type *lma_expr
     {
       overlay_list->os->update_dot = 1;
       overlay_list->os->update_dot_tree
-	= exp_assign (".", exp_binop ('+', overlay_vma, overlay_max));
+	= exp_assign (".", exp_binop ('+', overlay_vma, overlay_max), FALSE);
     }
 
   l = overlay_list;
Index: binutils-fsf-trunk-quilt/ld/ldlex.l
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/ldlex.l	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/ldlex.l	2012-08-02 22:35:36.061761927 +0100
@@ -320,6 +320,7 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([
 <BOTH,SCRIPT>"PHDRS"			{ RTOKEN (PHDRS); }
 <EXPRESSION,BOTH,SCRIPT>"AT"		{ RTOKEN(AT);}
 <EXPRESSION,BOTH,SCRIPT>"SUBALIGN"	{ RTOKEN(SUBALIGN);}
+<EXPRESSION,BOTH,SCRIPT>"HIDDEN"	{ RTOKEN(HIDDEN); }
 <EXPRESSION,BOTH,SCRIPT>"PROVIDE"	{ RTOKEN(PROVIDE); }
 <EXPRESSION,BOTH,SCRIPT>"PROVIDE_HIDDEN" { RTOKEN(PROVIDE_HIDDEN); }
 <EXPRESSION,BOTH,SCRIPT>"KEEP"		{ RTOKEN(KEEP); }
Index: binutils-fsf-trunk-quilt/ld/mri.c
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/mri.c	2012-07-30 19:11:21.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/mri.c	2012-08-02 22:35:36.061761927 +0100
@@ -297,7 +297,7 @@ mri_format (const char *name)
 void
 mri_public (const char *name, etree_type *exp)
 {
-  lang_add_assignment (exp_assign (name, exp));
+  lang_add_assignment (exp_assign (name, exp, FALSE));
 }
 
 void
Index: binutils-fsf-trunk-quilt/ld/emultempl/beos.em
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/emultempl/beos.em	2012-07-30 19:11:20.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/emultempl/beos.em	2012-08-02 22:38:10.911761895 +0100
@@ -348,7 +348,8 @@ gld_${EMULATION_NAME}_set_symbols (void)
   for (j = 0; init[j].ptr; j++)
     {
       long val = init[j].value;
-      lang_add_assignment (exp_assign (init[j].symbol, exp_intop (val)));
+      lang_add_assignment (exp_assign (init[j].symbol, exp_intop (val),
+				       FALSE));
       if (init[j].size == sizeof(short))
 	*(short *)init[j].ptr = val;
       else if (init[j].size == sizeof(int))
Index: binutils-fsf-trunk-quilt/ld/emultempl/pe.em
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/emultempl/pe.em	2012-07-30 19:11:20.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/emultempl/pe.em	2012-08-02 22:38:10.911761895 +0100
@@ -944,7 +944,7 @@ gld_${EMULATION_NAME}_set_symbols (void)
       lang_assignment_statement_type *rv;
 
       rv = lang_add_assignment (exp_assign (GET_INIT_SYMBOL_NAME (j),
-					    exp_intop (val)));
+					    exp_intop (val), FALSE));
       if (init[j].size == sizeof (short))
 	*(short *) init[j].ptr = val;
       else if (init[j].size == sizeof (int))
@@ -1721,8 +1721,9 @@ gld_${EMULATION_NAME}_unrecognized_file 
 		= pe_def_file->base_address;
 	      init[IMAGEBASEOFF].inited = 1;
 	      if (image_base_statement)
-		image_base_statement->exp = exp_assign ("__image_base__",
-							exp_intop (pe.ImageBase));
+		image_base_statement->exp
+		  = exp_assign ("__image_base__", exp_intop (pe.ImageBase),
+				FALSE);
 	    }
 
 	  if (pe_def_file->stack_reserve != -1
Index: binutils-fsf-trunk-quilt/ld/emultempl/pep.em
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/emultempl/pep.em	2012-07-30 19:11:20.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/emultempl/pep.em	2012-08-02 22:38:10.911761895 +0100
@@ -856,7 +856,7 @@ gld_${EMULATION_NAME}_set_symbols (void)
       lang_assignment_statement_type *rv;
 
       rv = lang_add_assignment (exp_assign (GET_INIT_SYMBOL_NAME (j),
-					    exp_intop (val)));
+					    exp_intop (val), FALSE));
       if (init[j].size == sizeof (short))
 	*(short *) init[j].ptr = (short) val;
       else if (init[j].size == sizeof (int))
@@ -1522,8 +1522,9 @@ gld_${EMULATION_NAME}_unrecognized_file 
 		= pep_def_file->base_address;
 	      init[IMAGEBASEOFF].inited = 1;
 	      if (image_base_statement)
-		image_base_statement->exp = exp_assign ("__image_base__",
-							exp_intop (pep.ImageBase));
+		image_base_statement->exp
+		  = exp_assign ("__image_base__", exp_intop (pep.ImageBase),
+				FALSE);
 	    }
 
 	  if (pep_def_file->stack_reserve != -1
Index: binutils-fsf-trunk-quilt/ld/emultempl/spuelf.em
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/emultempl/spuelf.em	2012-07-30 19:11:20.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/emultempl/spuelf.em	2012-08-02 22:38:10.911761895 +0100
@@ -165,7 +165,7 @@ spu_place_special_section (asection *s, 
 
 	  push_stat_ptr (&os->children);
 	  e_size = exp_intop (params.line_size - s->size);
-	  lang_add_assignment (exp_assign (".", e_size));
+	  lang_add_assignment (exp_assign (".", e_size, FALSE));
 	  pop_stat_ptr ();
 	}
       lang_add_section (&os->children, s, NULL, os);
Index: binutils-fsf-trunk-quilt/ld/emultempl/xtensaelf.em
===================================================================
--- binutils-fsf-trunk-quilt.orig/ld/emultempl/xtensaelf.em	2012-07-30 19:11:20.000000000 +0100
+++ binutils-fsf-trunk-quilt/ld/emultempl/xtensaelf.em	2012-08-02 22:38:10.911761895 +0100
@@ -1887,7 +1887,7 @@ ld_xtensa_insert_page_offsets (bfd_vma d
 		etree_type *name_op = exp_nameop (NAME, ".");
 		etree_type *addend_op = exp_intop (1 << xtensa_page_power);
 		etree_type *add_op = exp_binop ('+', name_op, addend_op);
-		etree_type *assign_op = exp_assign (".", add_op);
+		etree_type *assign_op = exp_assign (".", add_op, FALSE);
 
 		lang_assignment_statement_type *assign_stmt;
 		lang_statement_union_type *assign_union;

Re: [PATCH 1/4] LD: Linker script hidden symbol support

[Alan Modra]

On Fri, Aug 03, 2012 at 07:00:05PM +0100, Maciej W. Rozycki wrote:
> http://sourceware.org/ml/binutils/2005-08/msg00045.html
> 
> switched the scope of symbols created in linker scripts from object to 
> global.

Not so.  The above patch did not change the scope of symbols created
in linker scripts.  What it did do is change some linker created
symbols to instead be created by PROVIDE in the linker script, and
shortly thereafter this was corrected to PROVIDE_HIDDEN.  Linker
script symbols have been global forever.

>  The original change also broke all MIPS SVR4 ABI binaries produced as 
> with the switch from the object to the global scope the required _gp 
> symbol was no longer defined such as the MIPS psABI mandated.

So the above change can't have been responsible for breaking MIPS..
Regardless of that, I like the patch.  Please commit.

> 	bfd/
> 	* elflink.c (bfd_elf_record_link_assignment): Remove --defsym
> 	symbols special case.
> 
> 	ld/
> 	* ldexp.h (etree_union): Add defsym member to the assign member
> 	structure.
> 	(exp_assign): Add hidden argument to prototype.
> 	* ldexp.c (exp_fold_tree_1): Use the defsym member to handle
> 	--defsym symbols.
> 	(exp_assop): Add defsym argument, initialize the defsym member
> 	of the assign structure.
> 	(exp_assign): Handle hidden symbols.
> 	(exp_defsym): Update to use the defsym argument to exp_assop.
> 	(exp_provide): Update to handle the defsym argument to exp_assop.
> 	* ldlex.l (HIDDEN): New token.
> 	* ldgram.y (HIDDEN): Likewise.
> 	(assignment, section): Update calls to exp_assign.
> 	* ldctor.c (ldctor_build_sets): Likewise.
> 	* mri.c (mri_format): Likewise.
> 	* ldlang.c (lang_insert_orphan, lang_leave_overlay): Likewise.
> 	(open_input_bfds): Remove --defsym symbols special case.
> 	* ld/emultempl/beos.em (gld_${EMULATION_NAME}_set_symbols):
> 	Update call to exp_assign.
> 	* ld/emultempl/pe.em (gld_${EMULATION_NAME}_set_symbols):
> 	Likewise.
> 	* ld/emultempl/pep.em (gld_${EMULATION_NAME}_set_symbols):
> 	Likewise.
> 	* ld/emultempl/spuelf.em (spu_place_special_section): Likewise.
> 	* ld/emultempl/xtensaelf.em (ld_xtensa_insert_page_offsets):
> 	Likewise.
> 	* ld.texinfo (Assigning Values to Symbols): Add HIDDEN.
> 	(HIDDEN): New subsection.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH 1/4] LD: Linker script hidden symbol support

[Maciej W. Rozycki]

On Sat, 4 Aug 2012, Alan Modra wrote:

> > http://sourceware.org/ml/binutils/2005-08/msg00045.html
> > 
> > switched the scope of symbols created in linker scripts from object to 
> > global.
> 
> Not so.  The above patch did not change the scope of symbols created
> in linker scripts.  What it did do is change some linker created
> symbols to instead be created by PROVIDE in the linker script, and
> shortly thereafter this was corrected to PROVIDE_HIDDEN.  Linker
> script symbols have been global forever.

 Hmm, I must have misinterpreted the change then, sorry about that.

> >  The original change also broke all MIPS SVR4 ABI binaries produced as 
> > with the switch from the object to the global scope the required _gp 
> > symbol was no longer defined such as the MIPS psABI mandated.
> 
> So the above change can't have been responsible for breaking MIPS..

 I have now checked some old MIPS/Linux binaries I have lurking around and 
_gp appears exported in some back to 2001; those from 2000 do not have the 
symbol in the dynamic table.  Obviously having access to main executable's 
value of GP in shared libraries rather than their own is less than useful 
anyway.
 
> Regardless of that, I like the patch.  Please commit.

 I have done so now, thanks for the review.

  Maciej