[PATCH] Let '^' through the lexer

[PATCH] Let '^' through the lexer

[Michael Matz]

so that the (existing) code in parser and expression evaluator
actually get to see it and handle it as XOR.
---

A colleague was asking me about why XOR is missing from linker scripts and 
I initially wanted to say "but it is supported, only undocumented", 
because I distinctly remembered the code handling XOR in the expression 
parser and evaluator.  But ... he was right.  The lexer unhelpfully 
doesn't let '^' through and just spits out a "unrecognized character" 
error.

This is the case since the dawn of time it seems (ldgram end ldexp 
handling it, but ldlex not), but I don't see a reason.  While '^' 
might also be a meta-character in other lexer modes, that's no different 
from, say, '?' and '*'.  So, let's just handle it as well, document it, 
and leave it be :-)

(I have looked around for a testcase that systematically tests the 
expression syntax in linker scripts.  I can't find one, so I haven't added 
a case for this one either).

This is regtested on x86-64-linux only, my test-everything setup is 
missing right now.  Assuming that that works as well, okay for master?


Ciao,
Michael.

---
 ld/ld.texi | 22 ++++++++++++----------
 ld/ldlex.l |  1 +
 2 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/ld/ld.texi b/ld/ld.texi
index aa8b1aa86eb..8f82fe5a92f 100644
--- a/ld/ld.texi
+++ b/ld/ld.texi
@@ -6826,11 +6826,12 @@ precedence      associativity   Operators                Notes
 4               left            >>  <<
 5               left            ==  !=  >  <  <=  >=
 6               left            &
-7               left            |
-8               left            &&
-9               left            ||
-10              right           ? :
-11              right           &=  +=  -=  *=  /=       (2)
+7               left            ^
+8               left            |
+9               left            &&
+10              left            ||
+11              right           ? :
+12              right           &=  +=  -=  *=  /=       (2)
 (lowest)
 @end smallexample
 Notes:
@@ -6858,11 +6859,12 @@ height2pt&\omit&&\omit&&\omit&\cr
 &4&&left&&>>         <<&\cr
 &5&&left&&==         !=       >      <      <=      >=&\cr
 &6&&left&&\&&\cr
-&7&&left&&|&\cr
-&8&&left&&{\&\&}&\cr
-&9&&left&&||&\cr
-&10&&right&&?        :&\cr
-&11&&right&&\qquad\&=      +=       -=     *=     /=\qquad\ddag&\cr
+&7&&left&&^&\cr
+&8&&left&&|&\cr
+&9&&left&&{\&\&}&\cr
+&10&&left&&||&\cr
+&11&&right&&?        :&\cr
+&12&&right&&\qquad\&=      +=       -=     *=     /=\qquad\ddag&\cr
 &lowest&&&&&\cr
 height2pt&\omit&&\omit&&\omit&\cr}
 \hrule}
diff --git a/ld/ldlex.l b/ld/ldlex.l
index 1a6be1b6af2..9cb002452d8 100644
--- a/ld/ldlex.l
+++ b/ld/ldlex.l
@@ -247,6 +247,7 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)*
 <EXPRESSION,MRI>"/"			{ RTOKEN('/'); }
 <EXPRESSION,MRI>"%"			{ RTOKEN('%'); }
 <EXPRESSION,MRI>"<"			{ RTOKEN('<'); }
+<EXPRESSION,MRI>"^"			{ RTOKEN('^'); }
 <SCRIPT,EXPRESSION,MRI,WILD>"="		{ RTOKEN('='); }
 <SCRIPT,EXPRESSION,MRI,WILD>"}"		{ RTOKEN('}'); }
 <SCRIPT,EXPRESSION,MRI,WILD>"{"		{ RTOKEN('{'); }
-- 
2.39.1

Re: [PATCH] Let '^' through the lexer

[Alan Modra]

On Wed, Jul 12, 2023 at 01:56:54PM +0000, Michael Matz via Binutils wrote:
> so that the (existing) code in parser and expression evaluator
> actually get to see it and handle it as XOR.
> ---
> 
> A colleague was asking me about why XOR is missing from linker scripts and 
> I initially wanted to say "but it is supported, only undocumented", 
> because I distinctly remembered the code handling XOR in the expression 
> parser and evaluator.  But ... he was right.  The lexer unhelpfully 
> doesn't let '^' through and just spits out a "unrecognized character" 
> error.
> 
> This is the case since the dawn of time it seems (ldgram end ldexp 
> handling it, but ldlex not), but I don't see a reason.  While '^' 
> might also be a meta-character in other lexer modes, that's no different 
> from, say, '?' and '*'.  So, let's just handle it as well, document it, 
> and leave it be :-)
> 
> (I have looked around for a testcase that systematically tests the 
> expression syntax in linker scripts.  I can't find one, so I haven't added 
> a case for this one either).
> 
> This is regtested on x86-64-linux only, my test-everything setup is 
> missing right now.  Assuming that that works as well, okay for master?

OK, but..

> --- a/ld/ld.texi
> +++ b/ld/ld.texi
> @@ -6826,11 +6826,12 @@ precedence      associativity   Operators                Notes
>  4               left            >>  <<
>  5               left            ==  !=  >  <  <=  >=

Since you're adjusting this table, can you fix the precedence above?
== and != are lower precedence than the other relational operators.

>  6               left            &
> -7               left            |
> -8               left            &&
> -9               left            ||
> -10              right           ? :
> -11              right           &=  +=  -=  *=  /=       (2)
> +7               left            ^
> +8               left            |
> +9               left            &&
> +10              left            ||
> +11              right           ? :
> +12              right           &=  +=  -=  *=  /=       (2)

This line misses some assignment operators supported by ldgram.y.
It should be += -= *= /= <<= >>= &= |=

>  (lowest)
>  @end smallexample
>  Notes:
> @@ -6858,11 +6859,12 @@ height2pt&\omit&&\omit&&\omit&\cr
>  &4&&left&&>>         <<&\cr
>  &5&&left&&==         !=       >      <      <=      >=&\cr
>  &6&&left&&\&&\cr
> -&7&&left&&|&\cr
> -&8&&left&&{\&\&}&\cr
> -&9&&left&&||&\cr
> -&10&&right&&?        :&\cr
> -&11&&right&&\qquad\&=      +=       -=     *=     /=\qquad\ddag&\cr
> +&7&&left&&^&\cr
> +&8&&left&&|&\cr
> +&9&&left&&{\&\&}&\cr
> +&10&&left&&||&\cr
> +&11&&right&&?        :&\cr
> +&12&&right&&\qquad\&=      +=       -=     *=     /=\qquad\ddag&\cr
>  &lowest&&&&&\cr
>  height2pt&\omit&&\omit&&\omit&\cr}
>  \hrule}
> diff --git a/ld/ldlex.l b/ld/ldlex.l
> index 1a6be1b6af2..9cb002452d8 100644
> --- a/ld/ldlex.l
> +++ b/ld/ldlex.l
> @@ -247,6 +247,7 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)*
>  <EXPRESSION,MRI>"/"			{ RTOKEN('/'); }
>  <EXPRESSION,MRI>"%"			{ RTOKEN('%'); }
>  <EXPRESSION,MRI>"<"			{ RTOKEN('<'); }
> +<EXPRESSION,MRI>"^"			{ RTOKEN('^'); }
>  <SCRIPT,EXPRESSION,MRI,WILD>"="		{ RTOKEN('='); }
>  <SCRIPT,EXPRESSION,MRI,WILD>"}"		{ RTOKEN('}'); }
>  <SCRIPT,EXPRESSION,MRI,WILD>"{"		{ RTOKEN('{'); }
> -- 
> 2.39.1

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Let '^' through the lexer

[Jan Beulich]

On 13.07.2023 01:02, Alan Modra via Binutils wrote:
> On Wed, Jul 12, 2023 at 01:56:54PM +0000, Michael Matz via Binutils wrote:
>> so that the (existing) code in parser and expression evaluator
>> actually get to see it and handle it as XOR.
>> ---
>>
>> A colleague was asking me about why XOR is missing from linker scripts and 
>> I initially wanted to say "but it is supported, only undocumented", 
>> because I distinctly remembered the code handling XOR in the expression 
>> parser and evaluator.  But ... he was right.  The lexer unhelpfully 
>> doesn't let '^' through and just spits out a "unrecognized character" 
>> error.
>>
>> This is the case since the dawn of time it seems (ldgram end ldexp 
>> handling it, but ldlex not), but I don't see a reason.  While '^' 
>> might also be a meta-character in other lexer modes, that's no different 
>> from, say, '?' and '*'.  So, let's just handle it as well, document it, 
>> and leave it be :-)
>>
>> (I have looked around for a testcase that systematically tests the 
>> expression syntax in linker scripts.  I can't find one, so I haven't added 
>> a case for this one either).
>>
>> This is regtested on x86-64-linux only, my test-everything setup is 
>> missing right now.  Assuming that that works as well, okay for master?
> 
> OK, but..
> 
>> --- a/ld/ld.texi
>> +++ b/ld/ld.texi
>> @@ -6826,11 +6826,12 @@ precedence      associativity   Operators                Notes
>>  4               left            >>  <<
>>  5               left            ==  !=  >  <  <=  >=
> 
> Since you're adjusting this table, can you fix the precedence above?
> == and != are lower precedence than the other relational operators.
> 
>>  6               left            &
>> -7               left            |
>> -8               left            &&
>> -9               left            ||
>> -10              right           ? :
>> -11              right           &=  +=  -=  *=  /=       (2)
>> +7               left            ^
>> +8               left            |
>> +9               left            &&
>> +10              left            ||
>> +11              right           ? :
>> +12              right           &=  +=  -=  *=  /=       (2)
> 
> This line misses some assignment operators supported by ldgram.y.
> It should be += -= *= /= <<= >>= &= |=

At which point, considering the subject of the patch, I'm inclined
to ask: What about ^=?

Jan

Re: [PATCH] Let '^' through the lexer

[Alan Modra]

On Thu, Jul 13, 2023 at 09:54:46AM +0200, Jan Beulich wrote:
> On 13.07.2023 01:02, Alan Modra via Binutils wrote:
> > This line misses some assignment operators supported by ldgram.y.
> > It should be += -= *= /= <<= >>= &= |=
> 
> At which point, considering the subject of the patch, I'm inclined
> to ask: What about ^=?

It would need support in ldgram.y and ldexp.c.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Let '^' through the lexer

[Michael Matz]

Hey,

On Thu, 13 Jul 2023, Alan Modra wrote:

> On Thu, Jul 13, 2023 at 09:54:46AM +0200, Jan Beulich wrote:
> > On 13.07.2023 01:02, Alan Modra via Binutils wrote:
> > > This line misses some assignment operators supported by ldgram.y.
> > > It should be += -= *= /= <<= >>= &= |=
> > 
> > At which point, considering the subject of the patch, I'm inclined
> > to ask: What about ^=?
> 
> It would need support in ldgram.y and ldexp.c.

Exactly.  I felt that should be a separate patch.  But as it's easy to do 
and improves symmetry, here it is.  Okay for master if everything checks 
out (on x86-64 it does)?


Ciao,
Michael.
--------------
commit 894720ff9ce38ee1758d5a38cd1b0738c524f0c6
Author: Michael Matz <matz@suse.de>
Date:   Thu Jul 13 17:58:19 2023 +0200

    Also support '^=' in linker script expressions
    
    this requires also changes in ldgram.y and ldexp.c, unlike
    accepting '^' only.  But let's do this anyway, if only for
    symmetry.

diff --git a/ld/ld.texi b/ld/ld.texi
index 5009f0e5a6c..16ed74d7c9b 100644
--- a/ld/ld.texi
+++ b/ld/ld.texi
@@ -6832,7 +6832,7 @@ precedence      associativity   Operators                           Notes
 10              left            &&
 11              left            ||
 12              right           ? :
-13              right           +=  -=  *=  /=  <<=  >>=  &=  |=    (2)
+13              right           +=  -=  *=  /=  <<=  >>=  &=  |= ^= (2)
 (lowest)
 @end smallexample
 Notes:
@@ -6866,7 +6866,7 @@ height2pt&\omit&&\omit&&\omit&\cr
 &10&&left&&{\&\&}&\cr
 &11&&left&&||&\cr
 &12&&right&&?        :&\cr
-&13&&right&&\qquad +=       -=     *=     /= <<= >>= \&= |=\qquad\ddag&\cr
+&13&&right&&\qquad +=       -=     *=     /= <<= >>= \&= |= ^=\qquad\ddag&\cr
 &lowest&&&&&\cr
 height2pt&\omit&&\omit&&\omit&\cr}
 \hrule}
diff --git a/ld/ldexp.c b/ld/ldexp.c
index 170e1ed7f56..c0d90d3f93a 100644
--- a/ld/ldexp.c
+++ b/ld/ldexp.c
@@ -94,6 +94,7 @@ exp_print_token (token_code_type code, int infix_p)
     { RSHIFTEQ, ">>=" },
     { ANDEQ, "&=" },
     { OREQ, "|=" },
+    { XOREQ, "^=" },
     { OROR, "||" },
     { ANDAND, "&&" },
     { EQ, "==" },
diff --git a/ld/ldgram.y b/ld/ldgram.y
index 081176ba0f1..9dbf10b2b1f 100644
--- a/ld/ldgram.y
+++ b/ld/ldgram.y
@@ -108,7 +108,7 @@ static int error_index;
 %type <section_phdr> phdr_opt
 %type <integer> opt_nocrossrefs
 
-%right <token> PLUSEQ MINUSEQ MULTEQ DIVEQ  '=' LSHIFTEQ RSHIFTEQ   ANDEQ OREQ
+%right <token> PLUSEQ MINUSEQ MULTEQ DIVEQ  '=' LSHIFTEQ RSHIFTEQ   ANDEQ OREQ XOREQ
 %right <token> '?' ':'
 %left <token> OROR
 %left <token>  ANDAND
@@ -747,6 +747,8 @@ assign_op:
 			{ $$ = '&'; }
 	|	OREQ
 			{ $$ = '|'; }
+	|	XOREQ
+			{ $$ = '^'; }
 
 	;
 
diff --git a/ld/ldlex.l b/ld/ldlex.l
index 9cb002452d8..435172c08c3 100644
--- a/ld/ldlex.l
+++ b/ld/ldlex.l
@@ -233,6 +233,7 @@ V_IDENTIFIER [*?.$_a-zA-Z\[\]\-\!\^\\]([*?.$_a-zA-Z0-9\[\]\-\!\^\\]|::)*
 <SCRIPT,EXPRESSION,MRI,WILD>"/="	{ RTOKEN(DIVEQ); }
 <SCRIPT,EXPRESSION,MRI,WILD>"&="	{ RTOKEN(ANDEQ); }
 <SCRIPT,EXPRESSION,MRI,WILD>"|="	{ RTOKEN(OREQ); }
+<SCRIPT,EXPRESSION,MRI,WILD>"^="	{ RTOKEN(XOREQ); }
 <EXPRESSION,MRI>"&&"			{ RTOKEN(ANDAND); }
 <SCRIPT,EXPRESSION,MRI>">"		{ RTOKEN('>'); }
 <SCRIPT,EXPRESSION,MRI,INPUTLIST>","	{ RTOKEN(','); }

Re: [PATCH] Let '^' through the lexer

[Alan Modra]

On Thu, Jul 13, 2023 at 04:02:35PM +0000, Michael Matz wrote:
> Hey,
> 
> On Thu, 13 Jul 2023, Alan Modra wrote:
> 
> > On Thu, Jul 13, 2023 at 09:54:46AM +0200, Jan Beulich wrote:
> > > On 13.07.2023 01:02, Alan Modra via Binutils wrote:
> > > > This line misses some assignment operators supported by ldgram.y.
> > > > It should be += -= *= /= <<= >>= &= |=
> > > 
> > > At which point, considering the subject of the patch, I'm inclined
> > > to ask: What about ^=?
> > 
> > It would need support in ldgram.y and ldexp.c.
> 
> Exactly.  I felt that should be a separate patch.  But as it's easy to do 
> and improves symmetry, here it is.  Okay for master if everything checks 
> out (on x86-64 it does)?

Sure.

-- 
Alan Modra
Australia Development Lab, IBM