Introducing ELF2.0

Introducing ELF2.0

[The Cuthour]

I tried having the AI generate an introduction. What
do the people on this list actually think about this?


------------------------------------------------------------

Introducing ELF2.0:
The Next Generation of Efficient Linking

** Overview:**

ELF2.0 is an enhanced version of the Executable and
Linkable Format (ELF), designed to streamline and
optimize the linking process for developers. This
advanced tool introduces innovative features that
reduce the need for recompilation, thereby improving
development speed and efficiency.


** Key Features:**

1. ** Dynamic Offset Resolution:**
    - Member variable and function offsets are resolved
during the linking stage rather than at compile time.
This approach minimizes the need for recompilation and
accelerates the development process.

2. ** Advanced Symbol Resolution:**
    - ELF2.0 not only resolves symbols by locating their
definitions across files but also performs arithmetic
operations on the offsets. The arithmetic information
is stored in Reverse Polish Notation (RPN), enabling
efficient and accurate calculations during the linking
phase.

3. ** Enhanced Flexibility:**
    - The use of RPN for arithmetic operations allows
for greater flexibility and precision in symbol
resolution, making ELF2.0 a powerful tool for complex
projects that require intricate linking strategies.


** Benefits:**

- ** Reduced Compilation Time:**
By resolving offsets at the linking stage, developers
experience a significant reduction in the overall
compilation time.

- ** Increased Efficiency:**
The innovative features of ELF2.0 ensure a more
efficient development workflow, allowing developers
to focus on writing code rather than managing
recompilation issues.

- ** Greater Accuracy:**
The use of RPN for arithmetic operations enhances the
accuracy of symbol resolution, reducing the likelihood
of errors during the linking process.


** Conclusion:**

ELF2.0 represents a major leap forward in linking
technology, offering developers a robust and efficient
solution for their software development needs. Embrace
the future of efficient linking with ELF2.0 and
experience the difference in your development workflow.

------------------------------------------------------------

Re: Introducing ELF2.0

[Frank Ch. Eigler]

The Cuthour <cuthour@gmail.com> writes:

> 1. ** Dynamic Offset Resolution:**
> 2. ** Advanced Symbol Resolution:**
>    - ELF2.0 not only resolves symbols by locating their
> definitions across files but also performs arithmetic
> operations on the offsets. The arithmetic information
> is stored in Reverse Polish Notation (RPN), enabling
> efficient and accurate calculations during the linking
> phase.

See also the "complex relocation" capability that's been in elfutils for
a long long time, but not widely used by actual ports.  This capability
was added in 2007ish for a target ("mep") whose instruction set was
itself configurable.  See BSF_RELC, STT_RELC etc.

- FChE

Re: Introducing ELF2.0

[Frank Ch. Eigler]

fche@redhat.com (Frank Ch. Eigler) writes:

> [...]
> See also the "complex relocation" capability that's been in elfutils for
> [...]

I meant binutils, sorry.

- FChE

Re: Introducing ELF2.0

[Oleg Endo]

On Tue, 2025-01-28 at 03:26 +0900, The Cuthour wrote:
> I tried having the AI generate an introduction. What
> do the people on this list actually think about this?
> 

Sounds like typical AI-generated text :)

Best regards,
Oleg Endo

> 
> ------------------------------------------------------------
> 
> Introducing ELF2.0:
> The Next Generation of Efficient Linking
> 
> ** Overview:**
> 
> ELF2.0 is an enhanced version of the Executable and
> Linkable Format (ELF), designed to streamline and
> optimize the linking process for developers. This
> advanced tool introduces innovative features that
> reduce the need for recompilation, thereby improving
> development speed and efficiency.
> 
> 
> ** Key Features:**
> 
> 1. ** Dynamic Offset Resolution:**
>     - Member variable and function offsets are resolved
> during the linking stage rather than at compile time.
> This approach minimizes the need for recompilation and
> accelerates the development process.
> 
> 2. ** Advanced Symbol Resolution:**
>     - ELF2.0 not only resolves symbols by locating their
> definitions across files but also performs arithmetic
> operations on the offsets. The arithmetic information
> is stored in Reverse Polish Notation (RPN), enabling
> efficient and accurate calculations during the linking
> phase.
> 
> 3. ** Enhanced Flexibility:**
>     - The use of RPN for arithmetic operations allows
> for greater flexibility and precision in symbol
> resolution, making ELF2.0 a powerful tool for complex
> projects that require intricate linking strategies.
> 
> 
> ** Benefits:**
> 
> - ** Reduced Compilation Time:**
> By resolving offsets at the linking stage, developers
> experience a significant reduction in the overall
> compilation time.
> 
> - ** Increased Efficiency:**
> The innovative features of ELF2.0 ensure a more
> efficient development workflow, allowing developers
> to focus on writing code rather than managing
> recompilation issues.
> 
> - ** Greater Accuracy:**
> The use of RPN for arithmetic operations enhances the
> accuracy of symbol resolution, reducing the likelihood
> of errors during the linking process.
> 
> 
> ** Conclusion:**
> 
> ELF2.0 represents a major leap forward in linking
> technology, offering developers a robust and efficient
> solution for their software development needs. Embrace
> the future of efficient linking with ELF2.0 and
> experience the difference in your development workflow.
> 
> ------------------------------------------------------------
> 
> 

Re: Introducing ELF2.0

[The Cuthour]

 >> 1. ** Dynamic Offset Resolution:**
>> 2. ** Advanced Symbol Resolution:**
>>     - ELF2.0 not only resolves symbols by locating their
>> definitions across files but also performs arithmetic
>> operations on the offsets. The arithmetic information
>> is stored in Reverse Polish Notation (RPN), enabling
>> efficient and accurate calculations during the linking
>> phase.
> 
> See also the "complex relocation" capability that's been in elfutils for
> a long long time, but not widely used by actual ports.  This capability
> was added in 2007ish for a target ("mep") whose instruction set was
> itself configurable.  See BSF_RELC, STT_RELC etc.

Thank you.
Is there a syntax to write that in gas?

Re: Introducing ELF2.0

[Frank Ch. Eigler]

Hi -

> > See also the "complex relocation" capability that's been in elfutils for
> > a long long time, but not widely used by actual ports.  This capability
> > was added in 2007ish for a target ("mep") whose instruction set was
> > itself configurable.  See BSF_RELC, STT_RELC etc.
> 
> Thank you.
> Is there a syntax to write that in gas?

Some.  Each port would have to enable it individually.  See the #ifdef
OBJC_COMPLEX_RELC configuration in gas/symbols.c.


- FChE

Re: Introducing ELF2.0

[The Cuthour]

Hi

>>> See also the "complex relocation" capability that's been in elfutils for
>>> a long long time, but not widely used by actual ports.  This capability
>>> was added in 2007ish for a target ("mep") whose instruction set was
>>> itself configurable.  See BSF_RELC, STT_RELC etc.
>>
>> Thank you.
>> Is there a syntax to write that in gas?
> 
> Some.  Each port would have to enable it individually.  See the #ifdef
> OBJC_COMPLEX_RELC configuration in gas/symbols.c.

If the people on this list, who are struggling with 'complex
relocation,' have an idea of what they would like ELF to become,
I will respect that. In fact, Iwould encourage them to pursue it.

Re: Introducing ELF2.0

[Frank Ch. Eigler]

Hi -

> > Some.  Each port would have to enable it individually.  See the #ifdef
> > OBJC_COMPLEX_RELC configuration in gas/symbols.c.
> 
> If the people on this list, who are struggling with 'complex
> relocation,' have an idea of what they would like ELF to become,
> I will respect that. In fact, Iwould encourage them to pursue it.

The above functionality is compatible with - embedded in - normal ELF.
How individual target architectures express relocations etc. is within
their discretion, again normal ELF.  If you think you have a
significant problem that such link-time computations are needed to
express, please share them.

- FChE

Re: Introducing ELF2.0

[The Cuthour]

Hi

>>> Some.  Each port would have to enable it individually.  See the #ifdef
>>> OBJC_COMPLEX_RELC configuration in gas/symbols.c.
>>
>> If the people on this list, who are struggling with 'complex
>> relocation,' have an idea of what they would like ELF to become,
>> I will respect that. In fact, Iwould encourage them to pursue it.
> 
> The above functionality is compatible with - embedded in - normal ELF.
> How individual target architectures express relocations etc. is within
> their discretion, again normal ELF.  If you think you have a
> significant problem that such link-time computations are needed to
> express, please share them.

The goal here is Relinkability. Even in object-oriented languages, 
binary Relinkability should be a given. To achieve this, I propose 
implementing at least basic arithmetic operations during the linking 
phase with ELF2.0. If this proves successful, I envision further 
developments leading to ELF3.0 or even 4.0 and beyond.

Re: Introducing ELF2.0

[Frank Ch. Eigler]

Hi -

> The goal here is Relinkability. Even in object-oriented languages,
> binary Relinkability should be a given.  [...]

You have identified a Solution but not a Problem.

- FChE

Re: Introducing ELF2.0

[The Cuthour]

Hi

>> The goal here is Relinkability. Even in object-oriented languages,
>> binary Relinkability should be a given.  [...]
> 
> You have identified a Solution but not a Problem.

Suppose we have the following two classes:

=== Vec.h ===
class Vec {
     int x, y, z;
};
=== end Vec.h ===

=== Pix.h ===
class Pix: Vec {
     int r, g, b;
};
=== end Pix.h ===

If we add or remove a member variable in class Vec, it requires
recompiling not only Vec.cc but also Pix.cc. I believe this is
a problem. Pix.o should be relinkable.

Re: Introducing ELF2.0

[Frank Ch. Eigler]

Hi -

> > You have identified a Solution but not a Problem.
> 
> Suppose we have the following two classes:
> 
> === Vec.h ===
> class Vec {
>     int x, y, z;
> };
> === end Vec.h ===
> 
> === Pix.h ===
> class Pix: Vec {
>     int r, g, b;
> };
> === end Pix.h ===
> 
> If we add or remove a member variable in class Vec, it requires
> recompiling not only Vec.cc but also Pix.cc. I believe this is
> a problem. Pix.o should be relinkable.

OK, that's a problem, but it's not something obviously solvable simply
by link-time arithmetic.  A C++ compiler performs tons of
optimizations based on its compile-time knowledge of layouts of
structs.  These don't necessarily show up in assembly language
symbolic expressions at all, never mind in a form that would allow
resizing of things.

- FChE

Re: Introducing ELF2.0

[The Cuthour]

 > OK, that's a problem, but it's not something obviously solvable simply
 > by link-time arithmetic.

Some things can be resolved at link time, some at runtime
initialization, and some only during compile time. In some
cases, it might even require gathering all source code to
achieve the resolution. What I want to do is to discern
these distinctions.

 >                           A C++ compiler performs tons of
 > optimizations based on its compile-time knowledge of layouts of
 > structs.  These don't necessarily show up in assembly language
 > symbolic expressions at all, never mind in a form that would allow
 > resizing of things.

You are merely stating that access to struct members is
inlined and optimized. To that, I can only respond, "Let's
give up to a certain extent." Structured programming, and
binary tools, are the results of compromises with inlined
functions.

Re: Introducing ELF2.0

[Frank Ch. Eigler]

Hi -

> You are merely stating that access to struct members is
> inlined and optimized. To that, I can only respond, "Let's
> give up to a certain extent." Structured programming, and
> binary tools, are the results of compromises with inlined
> functions.

OK cool, then that's not a binutils or ELF question, that's a compiler
question.  If you feel like prototyping such capability, gcc (or llvm,
or other smaller compilers) may be what you need to play with.

- FChE

Re: Introducing ELF2.0

[The Cuthour]

> OK cool, then that's not a binutils or ELF question, that's a compiler
> question.  If you feel like prototyping such capability, gcc (or llvm,
> or other smaller compilers) may be what you need to play with.

After hearing about Linkable Constants, don't you get the idea that
this could be useful for other cases as well? A while ago, someone
from Sun mentioned that he didn't think such general tools were
useless for edge cases like thread-local data.

Re: Introducing ELF2.0

[The Cuthour]

Hi

When I think about it, structured programming is essentially
about relinkable functions. Structured programming is a
significant compromise with regard to inline functions. My
idea can be considered as a relinkable struct. It involves
compromising on the inline access to struct members.

Re: Introducing ELF2.0

[The Cuthour]

Here, somebpdy?

------------------------------------------------------------------------------

Suppose we have the following two classes:

=== Vec.h ===
class Vec {
     int x, y, z;
};
=== end Vec.h ===

=== Pix.h ===
class Pix: Vec {
     int r, g, b;
};
=== end Pix.h ===

If we add or remove a member variable in class Vec, it requires
recompiling not only Vec.cc but also Pix.cc. I believe this is
a problem. Pix.o should be relinkable.


To solve this problem, I propose the Linkable struct. The term
Linkable struct is proposed in analogy to Linkable function.
C language can be considered a language of Linkable functions.
C language enhances the relinkability of functions by
compromising the optimization of inline functions. C language
does not need Linkable structs because structures are declared
individually in C. Object-oriented languages need Linkable
structs because classes (structures) are described as
differences from their base classes.

To achieve Linkable structs, I propose the Linkable constant.
This mechanism ensures that certain constants are determined
at link time and is realized by extending the ELF format. The
offsets of structure members are defined in a distributed
manner across object files using these constants, with their
values resolved at link time. The information for calculating
these constants is ideally stored in each object file in
Reverse Polish Notation (RPN) for each symbol. The Linkable
constant is also a compromise concerning the optimization of
inline access to structure members.

Re: Introducing ELF2.0

[Nick Alcock]

On 29 Jan 2025, Frank Ch. Eigler spake thusly:

> Hi -
>
>> The goal here is Relinkability. Even in object-oriented languages,
>> binary Relinkability should be a given.  [...]
>
> You have identified a Solution but not a Problem.

I note that ld actually allowed you to relink already-linked binaries
until quite recently (libctf had to go through some contortions to
permit it -- in fact it still does because a lot of the same stuff is
needed to handle ld -r).

It was removed because of the lack of a use case, because it complicated
things noticeably, and because, well, it was kinda nuts.

Incremental linking, maybe. Relinking things that are already fully
linked... why? What would it even *mean*, given that linking is meant to
resolve everything that needs resolution, so after that there should be
nothing left for further linking passes to do?

-- 
NULL && (void)

Re: Introducing ELF2.0

[The Cuthour]

Hi

>>> The goal here is Relinkability. Even in object-oriented languages,
>>> binary Relinkability should be a given.  [...]
>>
>> You have identified a Solution but not a Problem.
> 
> I note that ld actually allowed you to relink already-linked binaries
> until quite recently (libctf had to go through some contortions to
> permit it -- in fact it still does because a lot of the same stuff is
> needed to handle ld -r).
> 
> It was removed because of the lack of a use case, because it complicated
> things noticeably, and because, well, it was kinda nuts.
> 
> Incremental linking, maybe. Relinking things that are already fully
> linked... why? What would it even *mean*, given that linking is meant to
> resolve everything that needs resolution, so after that there should be
> nothing left for further linking passes to do?

What I'm saying is whether the object files of derived classes
compiled before changes were made to the base class are still
relinkable after the base class has been modified and recompiled.
In C and Pascal languages, it was guaranteed that "only the files
with changes need to be recompiled, and all other object files are
relinkable."