conditional linkage ?

conditional linkage ?

[jim.cromie]

[-- Attachment #1: Type: text/plain, Size: 6515 bytes --]

hi all

Id like to allow this linker script language:

       KEEP( *(sectA_contents AND .gnu.linkonce.*.sectA_header) )
       # and for completeness
       KEEP( *(sectB_contents OR .gnu.linkonce.*.sectB_alternative) )

    and for it to mean that the header is only linked if the module's
    _contents are not empty, otherwize the header is wasted space.  The
contents and headers for an object are interleaved, giving fixed
    offsets from content to the "parent" header.


The point of doing this is to be able to hoist columns/fields
which are repetitive out of a flat table, and put them in the header,
where they pertain to all the records in the table.
Basically, this is like "normalizing" a database table.

Put in structural terms, the header in each object
is placed in a fixed offset from each record, so it
can be found using that offset, without needing a full pointer
to the "parent" row.

Specifically, Id like to use it in linux source,
to be able to un-flatten the table stored in __dyndbg section.

#define DEFINE_DYNAMIC_DEBUG_METADATA(name, fmt)                \
        static struct _ddebug  __aligned(8)                     \
        __section("__dyndbg") name = {                          \
                .modname = KBUILD_MODNAME,                      \
                .function = __func__,                           \
                .filename = __FILE__,                           \
                .format = (fmt),                                \
                .lineno = __LINE__,                             \
                .flags = _DPRINTK_FLAGS_DEFAULT,                \
                _DPRINTK_KEY_INIT                               \
        }

modname, filename, function are all repetitive,
and conceivably could all be hoisted into successive headers.
However, I only need 1 header, and dont want to confuse
myself or others with -E_TOO_ELABORATE proposals.

that said, having this work would be nice:

KEEP( *(__dyndbg_) AND *(.gnu.linkonce.dyndbg_func) \
                             AND *(.gnu.linkonce.dyndbg_file) \
                             AND *(.gnu.linkonce.dyndbg_module) )


the attached patch changes only the grammar side.
It compiles, and tests clean, but there are no new tests
of the new grammar


I made no attempt to actually make the linkage conditional;
I figured Id ask where the patient's liver is, before I go trying
to resect it.

So, questions:

1 - is it sane in principle, or is there a fatal grammar flaw ?
(this is my 1st ever grammar hack)
2 - is it practical ?
3 - is ld open for features ?
4 - do I need to add the same feature to gold ?
5 - where to cut ?
6 - what testcase is best for copy-modify ?
7 - what else do I need to know ?

patch, inlined for ease of review:

[jimc@frodo binutils-gdb]$ cat 0001-1st-crack-at-conditional-linkage.patch
From 00a90052002baef592aab40fdf321c0b604006db Mon Sep 17 00:00:00 2001
From: Jim Cromie <jim.cromie@gmail.com>
Date: Mon, 8 Feb 2021 00:21:27 -0700
Subject: [PATCH] 1st crack at conditional linkage

my aim is to allow this linker script language:

   KEEP( *(sectA_contents AND .gnu.linkonce.*.sectA_header) )
   # and for completeness
   KEEP( *(sectB_contents OR .gnu.linkonce.*.sectB_alternative) )

and for it to mean that the header is only linked if the module's
_contents are not empty, otherwize the header is wasted space.  The
contents and headers for an object are interleaved, giving fixed
offsets from content to the "parent" header.

Before trying this patch, I tried variations of *(contents ? header:).
Once they didnt work, the above constructs seemed narrower in scope.

I did it by altering the grammar for section_name_list, basically
copying the "$3 opt_comma $1" clause, and instead accepting "AND" and
"OR", then adding wildcard_list.join = enum (jBOTH, jAND, jOR) to
distinguish between them.  It seemed like the path of least
resistance.

IOW, the new input is parsed and accepted, but the linkage of the
_header is unconditional.  I havent figured out how to write a test
case yet, nor where to actually decide whether or not to link a
wildcard_spec.
---
 ld/ld.h     |  6 ++++++
 ld/ldgram.y | 23 ++++++++++++++++++++++-
 ld/ldlex.l  |  2 ++
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/ld/ld.h b/ld/ld.h
index 93f5af92c7..8c4a5980ec 100644
--- a/ld/ld.h
+++ b/ld/ld.h
@@ -106,10 +106,16 @@ struct wildcard_spec
   struct flag_info *section_flag_list;
 };

+typedef enum
+{
+  jBOTH, jAND, jOR
+} join_type;
+
 struct wildcard_list
 {
   struct wildcard_list *next;
   struct wildcard_spec spec;
+  join_type join;
 };

 #define BYTE_SIZE (1)
diff --git a/ld/ldgram.y b/ld/ldgram.y
index 5912329c77..3c399969c4 100644
--- a/ld/ldgram.y
+++ b/ld/ldgram.y
@@ -153,7 +153,7 @@ static int error_index;
 %token INPUT_SCRIPT INPUT_MRI_SCRIPT INPUT_DEFSYM CASE EXTERN START
 %token <name> VERS_TAG VERS_IDENTIFIER
 %token GLOBAL LOCAL VERSIONK INPUT_VERSION_SCRIPT
-%token KEEP ONLY_IF_RO ONLY_IF_RW SPECIAL INPUT_SECTION_FLAGS
ALIGN_WITH_INPUT
+%token KEEP AND OR ONLY_IF_RO ONLY_IF_RW SPECIAL INPUT_SECTION_FLAGS
ALIGN_WITH_INPUT
 %token EXCLUDE_FILE
 %token CONSTANT
 %type <versyms> vers_defns
@@ -592,12 +592,33 @@ exclude_name_list:
  ;

 section_name_list:
+ section_name_list AND section_name_spec
+ {
+  struct wildcard_list *tmp;
+  tmp = (struct wildcard_list *) xmalloc (sizeof *tmp);
+  tmp->next = $1;
+  tmp->spec = $3;
+  tmp->join = jAND;
+  $$ = tmp;
+ }
+ |
+ section_name_list OR section_name_spec
+ {
+  struct wildcard_list *tmp;
+  tmp = (struct wildcard_list *) xmalloc (sizeof *tmp);
+  tmp->next = $1;
+  tmp->spec = $3;
+  tmp->join = jOR;
+  $$ = tmp;
+ }
+ |
  section_name_list opt_comma section_name_spec
  {
   struct wildcard_list *tmp;
   tmp = (struct wildcard_list *) xmalloc (sizeof *tmp);
   tmp->next = $1;
   tmp->spec = $3;
+  tmp->join = jBOTH;
   $$ = tmp;
  }
  |
diff --git a/ld/ldlex.l b/ld/ldlex.l
index c1b1526358..c1aeaa86b0 100644
--- a/ld/ldlex.l
+++ b/ld/ldlex.l
@@ -315,6 +315,8 @@ V_IDENTIFIER
[*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)*
 <EXPRESSION,BOTH,SCRIPT>"PROVIDE" { RTOKEN(PROVIDE); }
 <EXPRESSION,BOTH,SCRIPT>"PROVIDE_HIDDEN" { RTOKEN(PROVIDE_HIDDEN); }
 <EXPRESSION,BOTH,SCRIPT>"KEEP" { RTOKEN(KEEP); }
+<EXPRESSION,BOTH,SCRIPT>"AND" { RTOKEN(AND); }
+<EXPRESSION,BOTH,SCRIPT>"OR" { RTOKEN(OR); }
 <EXPRESSION,BOTH,SCRIPT>"EXCLUDE_FILE"  { RTOKEN(EXCLUDE_FILE); }
 <EXPRESSION,BOTH,SCRIPT>"CONSTANT" { RTOKEN(CONSTANT);}
 <MRI>"#".*\n? { ++ lineno; }
-- 
2.29.2

[-- Attachment #2: 0001-1st-crack-at-conditional-linkage.patch --]
[-- Type: text/x-patch, Size: 3760 bytes --]

From 00a90052002baef592aab40fdf321c0b604006db Mon Sep 17 00:00:00 2001
From: Jim Cromie <jim.cromie@gmail.com>
Date: Mon, 8 Feb 2021 00:21:27 -0700
Subject: [PATCH] 1st crack at conditional linkage

my aim is to allow this linker script language:

   KEEP( *(sectA_contents AND .gnu.linkonce.*.sectA_header) )
   # and for completeness
   KEEP( *(sectB_contents OR .gnu.linkonce.*.sectB_alternative) )

and for it to mean that the header is only linked if the module's
_contents are not empty, otherwize the header is wasted space.  The
contents and headers for an object are interleaved, giving fixed
offsets from content to the "parent" header.

Before trying this patch, I tried variations of *(contents ? header:).
Once they didnt work, the above constructs seemed narrower in scope.

I did it by altering the grammar for section_name_list, basically
copying the "$3 opt_comma $1" clause, and instead accepting "AND" and
"OR", then adding wildcard_list.join = enum (jBOTH, jAND, jOR) to
distinguish between them.  It seemed like the path of least
resistance.

IOW, the new input is parsed and accepted, but the linkage of the
_header is unconditional.  I havent figured out how to write a test
case yet, nor where to actually decide whether or not to link a
wildcard_spec.
---
 ld/ld.h     |  6 ++++++
 ld/ldgram.y | 23 ++++++++++++++++++++++-
 ld/ldlex.l  |  2 ++
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/ld/ld.h b/ld/ld.h
index 93f5af92c7..8c4a5980ec 100644
--- a/ld/ld.h
+++ b/ld/ld.h
@@ -106,10 +106,16 @@ struct wildcard_spec
   struct flag_info *section_flag_list;
 };
 
+typedef enum
+{
+  jBOTH, jAND, jOR
+} join_type;
+
 struct wildcard_list
 {
   struct wildcard_list *next;
   struct wildcard_spec spec;
+  join_type join;
 };
 
 #define BYTE_SIZE	(1)
diff --git a/ld/ldgram.y b/ld/ldgram.y
index 5912329c77..3c399969c4 100644
--- a/ld/ldgram.y
+++ b/ld/ldgram.y
@@ -153,7 +153,7 @@ static int error_index;
 %token INPUT_SCRIPT INPUT_MRI_SCRIPT INPUT_DEFSYM CASE EXTERN START
 %token <name> VERS_TAG VERS_IDENTIFIER
 %token GLOBAL LOCAL VERSIONK INPUT_VERSION_SCRIPT
-%token KEEP ONLY_IF_RO ONLY_IF_RW SPECIAL INPUT_SECTION_FLAGS ALIGN_WITH_INPUT
+%token KEEP AND OR ONLY_IF_RO ONLY_IF_RW SPECIAL INPUT_SECTION_FLAGS ALIGN_WITH_INPUT
 %token EXCLUDE_FILE
 %token CONSTANT
 %type <versyms> vers_defns
@@ -592,12 +592,33 @@ exclude_name_list:
 	;
 
 section_name_list:
+		section_name_list AND section_name_spec
+			{
+			  struct wildcard_list *tmp;
+			  tmp = (struct wildcard_list *) xmalloc (sizeof *tmp);
+			  tmp->next = $1;
+			  tmp->spec = $3;
+			  tmp->join = jAND;
+			  $$ = tmp;
+			}
+	|
+		section_name_list OR section_name_spec
+			{
+			  struct wildcard_list *tmp;
+			  tmp = (struct wildcard_list *) xmalloc (sizeof *tmp);
+			  tmp->next = $1;
+			  tmp->spec = $3;
+			  tmp->join = jOR;
+			  $$ = tmp;
+			}
+	|
 		section_name_list opt_comma section_name_spec
 			{
 			  struct wildcard_list *tmp;
 			  tmp = (struct wildcard_list *) xmalloc (sizeof *tmp);
 			  tmp->next = $1;
 			  tmp->spec = $3;
+			  tmp->join = jBOTH;
 			  $$ = tmp;
 			}
 	|
diff --git a/ld/ldlex.l b/ld/ldlex.l
index c1b1526358..c1aeaa86b0 100644
--- a/ld/ldlex.l
+++ b/ld/ldlex.l
@@ -315,6 +315,8 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)*
 <EXPRESSION,BOTH,SCRIPT>"PROVIDE"	{ RTOKEN(PROVIDE); }
 <EXPRESSION,BOTH,SCRIPT>"PROVIDE_HIDDEN" { RTOKEN(PROVIDE_HIDDEN); }
 <EXPRESSION,BOTH,SCRIPT>"KEEP"		{ RTOKEN(KEEP); }
+<EXPRESSION,BOTH,SCRIPT>"AND"		{ RTOKEN(AND); }
+<EXPRESSION,BOTH,SCRIPT>"OR"		{ RTOKEN(OR); }
 <EXPRESSION,BOTH,SCRIPT>"EXCLUDE_FILE"  { RTOKEN(EXCLUDE_FILE); }
 <EXPRESSION,BOTH,SCRIPT>"CONSTANT"	{ RTOKEN(CONSTANT);}
 <MRI>"#".*\n?			{ ++ lineno; }
-- 
2.29.2

Re: conditional linkage ?

[Nick Clifton]

Hi Jim,

> Id like to allow this linker script language:
> 
>         KEEP( *(sectA_contents AND .gnu.linkonce.*.sectA_header) )

I do not think that this is going to work.  The nearest equivalent
would be section groups where if both .sectA and .gnu.linkonce.sectA
are in a group, then if .sectA is discarded (by linker garbage collection,
because it is unused) then .gnu.linkonce.sectA will also be discarded.

But in general I do not think that the idea will work because this
is not the sort of thing that the linker is built to do.  If you can
break your table down into lots of small sections and then link them
with garbage collection enabled you may achieve some of what you need.
But I doubt if it will be a complete solution.  Sorry.

Cheers
   Nick

Re: conditional linkage ?

[jim.cromie]

On Fri, Mar 5, 2021 at 7:27 AM Nick Clifton <nickc@redhat.com> wrote:

> Hi Jim,
>
> > Id like to allow this linker script language:
> >
> >         KEEP( *(sectA_contents AND .gnu.linkonce.*.sectA_header) )
>
> I do not think that this is going to work.


I think I agree on several levels
1st, the specific $3 $1 ordering looks counter to what I thought,
 suggesting opposite semantics, like perls print if $expr;
2nd, when I tested
KEEP ( *(.gnulinkonce.*.sectA_header sectA_contents ) )
the linkage order didnt change.
I think if my inserted behavior were to work at all,
it would depend upon ordering.


> The nearest equivalent
> would be section groups where if both .sectA and .gnu.linkonce.sectA
> are in a group, then if .sectA is discarded (by linker garbage collection,
> because it is unused) then .gnu.linkonce.sectA will also be discarded.
>
>
that would be useful behavior -
I currently get headers for empty per-module tables
I re-read the man / info pages,
all group mentions seemed to be about archives

if groups exist to allow something like

{
._start_dyndbg = . ;
KEEP( __dyndbg)
._stop_dyndbg = . ;
._stop_dyndbg - .start_dynbg ? KEEP(__dyndbg_header) :
 }

then I missed all that.. (distracted by my need for interleaving)

I did try putting the "statements" (; terminated) inside the KEEP,
that failed hard.


But in general I do not think that the idea will work because this
> is not the sort of thing that the linker is built to do.  If you can
> break your table down into lots of small sections and then link them
> with garbage collection enabled you may achieve some of what you need.
> But I doubt if it will be a complete solution.  Sorry.


>
Cheers
>    Nick
>
> thank you Nick
Jim