Linker script values as absolute / relative / number confusion

Linker script values as absolute / relative / number confusion

[Andrew Burgess]

I trying to take a linker script that used to work on older linkers and 
get it running on a more recent (2.22) linker.

I'm running into confusion relating to changes in how symbols in the 
linker script are treated as absolute, relative, or just numbers, and 
converted from one type to another.

I have a simple example,

 > cat obj.s
         .text
         .4byte 0x0
 > as -o obj.o obj.s
 > ld -T memmap1.ld -Map link1.map -o obj.x obj.o
 > cat memmap1.ld
SECTIONS
{
   . = 0x100;

   .text :
   {
     *(.text*)
   }

   _pad_to = 0x200;

   .padding :
   {
     BYTE(0); . = ALIGN(8);
     . += (. < _pad_to) ? (_pad_to - .) : 0;
   }

   .end :
   {
     BYTE(0);
   }

   /DISCARD/ : { *(*) }
}
 > grep "\.end" link1.map

## END ##

Ok, so, you'll notice a grep at the end, that's how I'm judging success 
/ failure.  There's probably lots wrong with the script, so here's what 
I'm trying to do,

The ".padding" section should fill enough space so that the section 
/after/ it starts at the address in _pad_to.  The address of ".end" then 
should be 0x200, and the output from the grep, would I hope look like this:
  .end            0x0000000000000200        0x1

[ I know there are "correct" ways to force the start address of ".end" 
to the right place, lets pretend there's a /really/ good reason why I 
want to do things this way for now. ]

On ld 2.19 I do get the behaviour I'd like.  On current HEAD I don't, 
instead I get:
.end            0x0000000000000304        0x1

The problem it turns out is the line:
   . += (. < _pad_to) ? (_pad_to - .) : 0;

So, I read up in the documentation, and in this page:
 
http://sourceware.org/binutils/docs-2.22/ld/Expression-Section.html#Expression-Section

If I look at the line in bits, first ". < _pad_to", whether "." is 
treated as relative or absolute it will be less than "_pad_to", so it's 
the "_pad_to - ." that's causing the problem.

The docs say that absolute symbols in output sections are treated as 
plain numbers, so I have "<plain-number: 0x200> - <relative-addr: 0x8>". 
  The docs further suggest that the result of this will be 0x1fc, 
leading to my change in behaviour.

Ok, so I apply the following patch to "fix" my linker script, the line 
in question now becomes:
   . += (ABSOLUTE(.) < _pad_to) ? (_pad_to - ABSOLUTE(.)) : 0;

Now I rerun the test, and grep for my output line .... and nothing has 
changed.

So docs again, and indeed, binary operations involving an absolute 
address and a plain-number convert the number to an absolute. Gah.

Ok, so I throw a few more ABSOLUTEs at the thing, and I get:
   . += (ABSOLUTE(.) < ABSOLUTE(_pad_to)) ? (ABSOLUTE(_pad_to) - 
ABSOLUTE(.)) : 0;

Now I rerun the test and joy, things work again.

So, finally, my questions,

(1) The thing I've ended up with doesn't feel great.  I create _pad_to 
as an absolute symbol, but then need to wrap it in ABSOLUTE in order to 
use it as an absolute symbol.  Is this /really/ the intended behaviour. 
  I'm sure there's a really good reason for the whole absolute addresses 
to numbers behaviour, but even after reading the docs I don't understand 
why we'd want it this way, I'm hoping someone can help me understand.

(2) In my second attempt, with the version of the line:
   . += (ABSOLUTE(.) < _pad_to) ? (_pad_to - ABSOLUTE(.)) : 0;
I see the size of "_pad_to - ABSOLUTE(.)" as "0x1fc", given that the 
".padding" section starts at 0x104, contains 1 byte then rounds up to an 
0x8 bounday, I think the value of ABSOLUTE(.) should be 0x108.  Given 
that then after converting the number 0x200 to be an absolute address ld 
comes up with the value "0x1fc + 0x108" = "0x304".  Given that 0x200 
(the original value of _pad_to) must be involved somewhere, then all I 
can think is that ld believes the absolute address for where this line 
is being executed is at 0x104, but this seems wrong to me, should it not 
be 0x108?


So, last bit of info is to figure out where all the changes came from. 
I'm using the git mirror (git://sourceware.org/git/binutils.git), the 
commit that actually changed the behaviour was 
dc027516e99b4795b1fba7179de6fe00606910, which was added as a result of 
bug, http://sourceware.org/bugzilla/show_bug.cgi?id=12066, which was 
raised as a result of commit 557189b0dd1f3dec6b00b91b9b0c5459ece72a52.

The mailing list thread for commit 557189b0 is:
   http://sourceware.org/ml/binutils/2010-08/msg00118.html

Thanks for your time,

Andrew













Question 1:

[PATCH] Re: Linker script values as absolute / relative / number confusion

[Andrew Burgess]

On 14/09/2012 5:20 PM, Andrew Burgess wrote:
> 
> So, last bit of info is to figure out where all the changes came from. 
> I'm using the git mirror (git://sourceware.org/git/binutils.git), the 
> commit that actually changed the behaviour was 
> dc027516e99b4795b1fba7179de6fe00606910, which was added as a result of 
> bug, http://sourceware.org/bugzilla/show_bug.cgi?id=12066, which was 
> raised as a result of commit 557189b0dd1f3dec6b00b91b9b0c5459ece72a52.

Just for fun I backed out commit dc02751... and ran the linker tests and there were no regressions, so I made the following patch based on the bug description in 12066.  This currently passes, but fails with dc02751 removed.

I don't have write permissions, so if anyone would like to apply it feel free.

Thanks,
Andrew


ld/testsuite/ChangeLog

2012-09-14  Andrew Burgess  <aburgess@broadcom.com>

	* ld-scripts/assign-loc.d: New file.  Test for assigning absolute
	symbol to location counter.
	* ld-scripts/assign-loc.t: New file.  Linker script for above
	test.
	* ld-scripts/expr.exp: Add new assignment test.

diff --git a/ld/testsuite/ld-scripts/assign-loc.d b/ld/testsuite/ld-scripts/assign-loc.d
new file mode 100644
index 0000000..dceff7b
--- /dev/null
+++ b/ld/testsuite/ld-scripts/assign-loc.d
@@ -0,0 +1,6 @@
+#ld: -T assign-loc.t
+#source: data.s
+#nm: -n
+0+0100 A HEAP_SIZE
+0+2000 B _start
+0+2100 B _end
diff --git a/ld/testsuite/ld-scripts/assign-loc.t b/ld/testsuite/ld-scripts/assign-loc.t
new file mode 100644
index 0000000..e8c5102
--- /dev/null
+++ b/ld/testsuite/ld-scripts/assign-loc.t
@@ -0,0 +1,17 @@
+SECTIONS
+{
+  . = 0x2000 ;
+
+  _start = .;
+
+  HEAP_SIZE = 0x100;
+
+  .heap : {
+    . = HEAP_SIZE;
+    . = ALIGN(4);
+  }
+
+  _end = .;
+
+  /DISCARD/ : { *(*) }
+}
diff --git a/ld/testsuite/ld-scripts/expr.exp b/ld/testsuite/ld-scripts/expr.exp
index 185fa4e..ea5002f 100644
--- a/ld/testsuite/ld-scripts/expr.exp
+++ b/ld/testsuite/ld-scripts/expr.exp
@@ -23,3 +23,4 @@
 run_dump_test expr1
 run_dump_test expr2
 run_dump_test sane1
+run_dump_test assign-loc

Re: Linker script values as absolute / relative / number confusion

[Alan Modra]

On Fri, Sep 14, 2012 at 05:20:58PM +0100, Andrew Burgess wrote:
> (1) The thing I've ended up with doesn't feel great.  I create
> _pad_to as an absolute symbol, but then need to wrap it in ABSOLUTE
> in order to use it as an absolute symbol.  Is this /really/ the
> intended behaviour.  I'm sure there's a really good reason for the
> whole absolute addresses to numbers behaviour, but even after
> reading the docs I don't understand why we'd want it this way, I'm
> hoping someone can help me understand.

If you've read the history then you know that ld script expression
evaluation was such an ad-hoc mess that it couldn't be documented
without pages of rules.  People wrote ld scripts without the benefit
of much documentation and experimented until "it worked".  The changes
I made probably didn't go far enough, but if I had rewritten
expression evaluation to be completely sane that would have broken too
many existing ld scripts.  Incidentally, a rewrite from scratch would
have been a much easier task than trying to minimize incompatibility.

> (2) In my second attempt, with the version of the line:
>   . += (ABSOLUTE(.) < _pad_to) ? (_pad_to - ABSOLUTE(.)) : 0;
> I see the size of "_pad_to - ABSOLUTE(.)" as "0x1fc", given that the
> ".padding" section starts at 0x104, contains 1 byte then rounds up
> to an 0x8 bounday, I think the value of ABSOLUTE(.) should be 0x108.

Yes, it is.

> Given that then after converting the number 0x200 to be an absolute
> address ld comes up with the value "0x1fc + 0x108" = "0x304".  Given
> that 0x200 (the original value of _pad_to) must be involved
> somewhere, then all I can think is that ld believes the absolute
> address for where this line is being executed is at 0x104, but this
> seems wrong to me, should it not be 0x108?

ld is doing ". = 0x200".  That results in the section ending at
0x104+0x200 = 0x304.  Neglecting the condition, your expression is

. = . + (_pad_to - ABSOLUTE(.));

So let's start with the innermost value of ".".
This has value 4, in section ".padding".
ABSOLUTE(.) then is 0x108, in the absolute section.
_pad_to is 0x200, and the ld docs say "Expressions appearing inside an
output section definition treat absolute symbols as numbers".  So this
is just a number without any section.
So, what is the result of the subtraction?  According to the ld docs,
since one operand is absolute the other should first be made absolute.
That doesn't happen here!  The ld docs are missing a rule.  Instead
the number is used as is, and the result is 0xf8 absolute.  Now we add
this to ".", which has value 4, section ".padding".  Given two
different sections, ld converts both values to absolute, to get
0x108 + 0xf8.  So the net result is ". = 0x200".

When you instead use ABSOLUTE(_pad_to) in the expression, the value of
ABSOLUTE(_pad_to) is 0x200 absolute.  0x200 absolute - 0x108 absolute
is 0xf8, and because the sections (absolute in this case) were the
same, the result is a plain number.  So . + 0xf8 results in 0xfc.
Net result, ". = 0xfc", as you require.


	* ld.texinfo (Expression Section): Correct description of
	conversions for a binary operation involving an absolute symbol
	and a number.

Index: ld/ld.texinfo
===================================================================
RCS file: /cvs/src/src/ld/ld.texinfo,v
retrieving revision 1.284
diff -u -p -r1.284 ld.texinfo
--- ld/ld.texinfo	6 Aug 2012 22:27:52 -0000	1.284
+++ ld/ld.texinfo	16 Sep 2012 13:03:02 -0000
@@ -5646,14 +5646,18 @@ addresses, ld follows these rules to eva
 
 @itemize @bullet
 @item
+Unary operations on an absolute address or number, and binary
+operations on two absolute addresses or two numbers, or between one
+absolute address and a number, apply the operator to the value(s).
+@item
 Unary operations on a relative address, and binary operations on two
 relative addresses in the same section or between one relative address
 and a number, apply the operator to the offset part of the address(es).
 @item
-Unary operations on an absolute address, and binary operations on one
-or more absolute addresses or on two relative addresses not in the
-same section, first convert any non-absolute term to an absolute
-address before applying the operator.
+Other binary operations, that is, between two relative addresses not
+in the same section, or between a relative address and an absolute
+address, first convert any non-absolute term to an absolute address
+before applying the operator.
 @end itemize
 
 The result section of each sub-expression is as follows:

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Re: Linker script values as absolute / relative / number confusion

[Alan Modra]

On Fri, Sep 14, 2012 at 11:53:09PM +0100, Andrew Burgess wrote:
> 2012-09-14  Andrew Burgess  <aburgess@broadcom.com>
> 
> 	* ld-scripts/assign-loc.d: New file.  Test for assigning absolute
> 	symbol to location counter.
> 	* ld-scripts/assign-loc.t: New file.  Linker script for above
> 	test.
> 	* ld-scripts/expr.exp: Add new assignment test.

Tests like this one need a little more work to ensure they pass on
targets other than i686-linux.  Applied as follows.

Index: ld/testsuite/ld-scripts/assign-loc.d
===================================================================
RCS file: ld/testsuite/ld-scripts/assign-loc.d
diff -N ld/testsuite/ld-scripts/assign-loc.d
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ ld/testsuite/ld-scripts/assign-loc.d	19 Sep 2012 02:46:53 -0000
@@ -0,0 +1,11 @@
+#ld: -T assign-loc.t
+#source: data.s
+#nm: -n
+#notarget: rs6000-*-aix*
+#...
+0+0100 A HEAP_SIZE
+#...
+0+2000 [AB] _start
+#...
+0+2100 [AB] _end
+#pass
Index: ld/testsuite/ld-scripts/assign-loc.t
===================================================================
RCS file: ld/testsuite/ld-scripts/assign-loc.t
diff -N ld/testsuite/ld-scripts/assign-loc.t
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ ld/testsuite/ld-scripts/assign-loc.t	19 Sep 2012 02:46:53 -0000
@@ -0,0 +1,17 @@
+SECTIONS
+{
+  . = 0x2000 ;
+
+  _start = .;
+
+  HEAP_SIZE = 0x100;
+
+  .heap : {
+    . = HEAP_SIZE;
+    . = ALIGN(4);
+  }
+
+  _end = .;
+
+  /DISCARD/ : { *(*) }
+}
Index: ld/testsuite/ld-scripts/expr.exp
===================================================================
RCS file: /cvs/src/src/ld/testsuite/ld-scripts/expr.exp,v
retrieving revision 1.4
diff -u -p -r1.4 expr.exp
--- ld/testsuite/ld-scripts/expr.exp	23 Jan 2011 03:29:29 -0000	1.4
+++ ld/testsuite/ld-scripts/expr.exp	19 Sep 2012 02:46:53 -0000
@@ -23,3 +23,4 @@
 run_dump_test expr1
 run_dump_test expr2
 run_dump_test sane1
+run_dump_test assign-loc

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH] Re: Linker script values as absolute / relative / number confusion

[Andrew Burgess]

On 19/09/2012 3:51 AM, Alan Modra wrote:
> On Fri, Sep 14, 2012 at 11:53:09PM +0100, Andrew Burgess wrote:
>> 2012-09-14  Andrew Burgess  <aburgess@broadcom.com>
>>
>> 	* ld-scripts/assign-loc.d: New file.  Test for assigning absolute
>> 	symbol to location counter.
>> 	* ld-scripts/assign-loc.t: New file.  Linker script for above
>> 	test.
>> 	* ld-scripts/expr.exp: Add new assignment test.
>
> Tests like this one need a little more work to ensure they pass on
> targets other than i686-linux.  Applied as follows.

Thanks for fixing this up for me and applying.

Cheers,

Andrew

Re: Linker script values as absolute / relative / number confusion

[Andrew Burgess]

On 16/09/2012 11:19 PM, Alan Modra wrote:
> On Fri, Sep 14, 2012 at 05:20:58PM +0100, Andrew Burgess wrote:
>> (1) The thing I've ended up with doesn't feel great.  I create
>> _pad_to as an absolute symbol, but then need to wrap it in ABSOLUTE
>> in order to use it as an absolute symbol.  Is this /really/ the
>> intended behaviour.  I'm sure there's a really good reason for the
>> whole absolute addresses to numbers behaviour, but even after
>> reading the docs I don't understand why we'd want it this way, I'm
>> hoping someone can help me understand.
>
> If you've read the history then you know that ld script expression
> evaluation was such an ad-hoc mess that it couldn't be documented
> without pages of rules.  People wrote ld scripts without the benefit
> of much documentation and experimented until "it worked".  The changes
> I made probably didn't go far enough, but if I had rewritten
> expression evaluation to be completely sane that would have broken too
> many existing ld scripts.  Incidentally, a rewrite from scratch would
> have been a much easier task than trying to minimize incompatibility.

I did indeed get that impression.  I just wanted to ask anyway in case 
the answer had been "Oh no! What a terrible mistake, quick lets fix it." 
:)   Instead, I think the answer is more "Yeah it's not ideal, but 
that's now the expected behaviour", which is fair enough given where we 
started out.  I just wanted to be sure.

>
>> (2) In my second attempt, with the version of the line:
>>    . += (ABSOLUTE(.) < _pad_to) ? (_pad_to - ABSOLUTE(.)) : 0;
>> I see the size of "_pad_to - ABSOLUTE(.)" as "0x1fc", given that the
>> ".padding" section starts at 0x104, contains 1 byte then rounds up
>> to an 0x8 bounday, I think the value of ABSOLUTE(.) should be 0x108.
>
> Yes, it is.
>
>> Given that then after converting the number 0x200 to be an absolute
>> address ld comes up with the value "0x1fc + 0x108" = "0x304".  Given
>> that 0x200 (the original value of _pad_to) must be involved
>> somewhere, then all I can think is that ld believes the absolute
>> address for where this line is being executed is at 0x104, but this
>> seems wrong to me, should it not be 0x108?
>
> ld is doing ". = 0x200".  That results in the section ending at
> 0x104+0x200 = 0x304.  Neglecting the condition, your expression is
>
> . = . + (_pad_to - ABSOLUTE(.));
>
> So let's start with the innermost value of ".".
> This has value 4, in section ".padding".
> ABSOLUTE(.) then is 0x108, in the absolute section.
> _pad_to is 0x200, and the ld docs say "Expressions appearing inside an
> output section definition treat absolute symbols as numbers".  So this
> is just a number without any section.
> So, what is the result of the subtraction?  According to the ld docs,
> since one operand is absolute the other should first be made absolute.
> That doesn't happen here!  The ld docs are missing a rule.  Instead
> the number is used as is, and the result is 0xf8 absolute.  Now we add
> this to ".", which has value 4, section ".padding".  Given two
> different sections, ld converts both values to absolute, to get
> 0x108 + 0xf8.  So the net result is ". = 0x200".

Thanks for explaining that, I see where I went wrong now.

Cheers,

Andrew