Re: [RFC] Allow linker scripts to specify multiple output regions for an output section?

[Tejas Belagod <tejas.belagod@foss.arm.com>]

Hi Erik,

Thanks for your comments. My comments inline below.

On 28/02/17 05:51, Erik Christiansen wrote:
> On 22.02.17 15:28, Thomas Preudhomme wrote:
>> There has been some interest in the past in having syntactic support for
>> specifying mapping of an output section to multiple memory regions in the
>> GNU LD scripting language (eg.
>> https://sourceware.org/bugzilla/show_bug.cgi?id=14299). I would like to
>> propose a scheme here and welcome any feedback.
>
> TL;DR: Detailed response begins after 6 paragraphs.
>
> OK, in the absence of prior discussion, I'll just think aloud as I
> correlate the proposal with my experience in three decades developing
> embedded systems. Unfortunately, the one time an MMU was involved, that
> was done by the time I became involved, but memory holes are all black.
>
> The closest scenario I recall is where there were disparate physical
> memories, both on and off chip, I simply added a MEMORY region for each
> such block, e.g. Flash, 16bit SRAM, 8bit SRAM, a couple of small ones
> for specific memory mapped system chips with bunches of config
> registers, and maybe an FPGA in the mix. Add comments for device names
> and the waitstate generator values, and the script serves as central
> documentation too.
>
> With that one-to-one region mapping, there was never any conflict over
> where stuff should be located, and non were interchangeable. It is as
> described by "some on-chip memory and some off-chip memory, but at
> non-contiguous addresses" in the above link. And where we had both 8 and
> 16 bit SRAMS, it was most definitely consistent with "a region of
> on-chip SRAM which performs better for code, and the remainder performs
> better for data", except that using the wrong one was fatal rather than
> merely inferior.
>
> One issue I've encountered is detecting region overflow when multiple
> output sections contribute to its content, but existing syntax supports
> that, e.g.:
>
> MEMORY
> {
>    flash   (rx)  : ORIGIN = 0, LENGTH = 32K
>    ram    (rw!x) : ORIGIN = 0x800060, LENGTH = 2K
>    eeprom (rw!x) : ORIGIN = 0x810000, LENGTH = 1K
> }
>
> . = ASSERT (_etext + SIZEOF (.data) <= LENGTH(flash) , "Error: .text + .data
> collectively overflow the flash memory." ) ;
>
> But the need to flow across memory holes never eventuated in practice,
> as a modest chunk of on-chip RAM could always be used for e.g. sdata,
> leaving no need for flowing. All other regions were always incompatible,
> making flowing impossible.
>
> ...
>> If LMA is specified, the image(startup code etc.) most likely handles
>> the copying from load address to output section VMA.
>
> Yes, it does. And in the generic init code I've encountered, it has just
> been a single copy loop for e.g. bss, performing a contiguous block copy.
> (And when I've written it, that was true too.)
>
>> Multiple segment spec means the output section can be part of more
>> than one segment and ‘fillexp’ simply fills the output section loaded
>> with the fill value.
>
> Trans-hole flowing would also require a runtime copy loop for each
> non-contiguous block, or a table-driven multi-block copier, with the
> run-time table somehow initialised from the linker script. (I can
> imagine using variables defined in the linker script, and the .RPT
> assembler directive - maybe.)
>
>> Now, this does not have a method to specify output section spanning multiple
>> memory regions. For example, if there are 2 RAM regions RAML and RAMU and
>> the user wants an output section to first fill RAML and then when RAML is
>> full, i.e. when the remaining space in RAML cannot accommodate a full input
>> section, start filling RAMU, the user has to split the sections into
>> multiple output sections. If we extend this syntax to specify multiple
>> output regions, we can make the linker map the output section to multiple
>> regions by filling the output region with input sections in the order
>> specified in the ‘output-section-command’ and when its full (meaning when
>> the remaining gap in a region cannot accommodate one full input section, it
>> starts from the next output region.
>
> This seems to be the alternate view of the problem of asking ld to flow
> code around holes in a region, something it still can't do, IIRC. I
> state it that way, because two non-contiguous memory regions over which
> code (or data) may be interchangeably flowed, are identical to a single
> region with a hole.
>
> The proposal does seem to be a way to think about addressing that issue:
>
>> Eg.
>>
>> MEMORY
>>
>> {
>>    RAML (rwx) : ORIGIN = 0x1FFF0000, LENGTH = 0x00010000
>>    RAMU (rwx) : ORIGIN = 0x20000000, LENGTH = 0x00040000
>>    RAMZ (rwx) : ORIGIN = 0x20040000, LENGTH = 0x00040000	
>> }
>>
>> SECTIONS
>> {
>>    .text 0x1000 : { *(.text) _etext = . ; }
>>    .mdata  :
>>    AT ( ADDR (.text) + SIZEOF (.text) )
>>    { _data = . ; *(.data) *(.data.*); _edata = . ; } > RAML, RAMU, RAMZ
>> }
>
> Without the need for new syntax or complex init code generators,
> having gcc flow code across up to 5 pages of flash plus .lowtext and a
> floating .hightext was compatible with the linker script and tests shown
> here:
>
> http://lists.nongnu.org/archive/html/avr-gcc-list/2012-12/msg00044.html
>
> While details have faded from wet RAM, ISTR that holes were
> manufacturable by not populating any of the 5 pages, which gcc sees as
> named spaces. The gcc stuff was done in the AVR back end, IIRC, while an
> implementation in ld would be generic.
>
>> Illustration:
>>
>> Consider an example where we have the following input .data sections:
>>
>> .data: size 0x0000FFF0
>> .data.a : size 0x000000F0
>> .data.b : size 0x00003000
>> .data.c : size 0x00000200
>>
>> With the above scheme, this will be mapped in the following way to RAML,RAMU
>> and RAMZ:
>>
>> RAML : (0x1FFF0000 - 0x1FFFFFF0): .data
>>         (0x1FFFFFF0 - 0x1FFFFFFF): *** GAP ***
>
> Would GAP use ALIGNMENT, or introduce a new parameter?
>

I wouldn't want to overload ALIGNMENT here - what if its needed simultaneously 
with ALIGNMENT. Can we not leave this space unassigned? More often than not if 
one's filling a memory region automatically, would they really care what goes 
into the gaps (if security is not a concern)? OTOH, if security is a concern, we 
can explore introducing a new syntax with a default behavior of zero-filling the 
gaps.

> How would the target-specific relocations required to break code across
> the hole be handled by ld? E.g. break a small AVR code loop (with 6-bit
> relative addressing range) and you'll need a LJMP to bridge the hole,
> and another with reversed loop conditionality to close the loop.
> Multiply that task by all the possible relocs, and again by all the
> possible CPU targets, and it's never-ending work for a software team for
> life.
>

As I understand, compilers generate references to objects within a section with a

  .<input_section_name> + offset_within_section

Now when a section that spans 2 or more regions inserts holes/padding to prevent 
an object from straddling 2 regions, the offsets within the section to other 
objects will change. This means all the compiler-generated "section + offset" of 
all objects that come after the padding will need to be fixed up. Its really 
difficult to know which ones to fix up - the relocations are only on the section 
label, not the object in the section. So, what I'm proposing here will not split 
the input sections - input sections will move as a block.

> It seems more
>
>> RAMU : (0x20000000 - 0x200000F0): .data.a
>>         (0x200000F0 - 0x200030F0): .data.b
>>         (0x200030F0 - 0x200032F0): .data.c
>>
>>
>> It will not affect the specification in terms of the other attributes, but
>> one (LMA):
>>
>> * Output section VMA: No change - this just specifies where the output
>> section will start.
>>
>> * type: No change - this is for the output section as a whole - output
>> memory regions will not change it.
>>
>> * LMA: The output section can still be loaded from one LMA and mapped to
>> output VMA - the only change here is that the loader will need to map the
>> output sections to VMA with the same pattern as the multiple output region
>> matching code above. Can a loader do that? Can ad-hoc loaders do this? Or do
>> all loaders assume that regions are continguous when output section is
>> mapped to VMAs?
>
> Contiguous. Hole-flowing is what you're proposing to implement, both the
> linker internal component (target-specific reloctions), and the generic
> (e.g. table driven) multi-block copy loop synthesiser for custom init
> code generation. How that would integrate with existing init code in
> various implementations, I have no idea.
>
> If LMA can also be flowed around a hole, then runtime init code must be
> able to handle not only non-contiguous delivery, but gapped pick-up. Has
> the complexity of simultaneously handling different gaps in both been
> considered?
>

I haven't thought about that. Can it be worked on the principle that when one 
specifies an LMA and there is user-written init code to copy blocks, the init 
code programmer knows the LMA gap layout and can handle the gaps accordingly? It 
could be the case currently where code from different non-contiguous ROMs are 
copied into a RAM during startup. This IMHO, is always specific to the 
particular embedded system being deployed.

> ...
>> For orthogonality and consistency, we would want to apply the multiple
>> region feature to overlays too. The semantics will not be different from the
>> algorithm mentioned above. The only caveat is that the overlay
>> manager/loader will need to handle the swapping in and out of sections that
>> run from the VMA consistently with the mapping algo described above. Do we
>> want this for overlays too?
>
> Expanding the complexity of a single-problem solution to cover other
> situations seems courageous, unless it naturally falls out of the
> narrower solution. As overlays are used e.g. when RAM size or CPU
> instruction addressing range is constrained, but there's ample flash,
> then the likelihood of holes in either is limited, I suspect.
>

Makes sense.

Thanks,
Tejas.

> Specifying discrete output sections with VMAs placed around the physical
> holes is another way to dodge them. They can all be allocated to a
> global encompassing memory region. Flowing is performed manually by
> assigning suitable code chunks to preferred input sections. Automating
> that, as intimated above, is non-trivial.
>
> Caveat: Above thoughts have flowed without aid of caffeine, and are
>          recollections from old battles.
>
> Erik
>