Reserving specified size of RUNPATH entry in the dynamic section during linking

Reserving specified size of RUNPATH entry in the dynamic section during linking

[Jacob Kroon]

Hi,

As part of an effort to make binaries reproducible regardless of their
build path, I need to enforce the same size of the RUNPATH entry in the
dynamic section during linking, even though I don't fill it completely.
Is it possible to give some flag to gnu ld that allows me to set it to a
specific size ? Or is there a way to patch the elf file after linking,
so that the entry has a specified size ?

Regards
Jacob

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Tom Kacvinsky]

On Sat, Nov 27, 2021 at 5:05 PM Jacob Kroon via Gcc-help <
gcc-help@gcc.gnu.org> wrote:

> Hi,
>
> As part of an effort to make binaries reproducible regardless of their
> build path, I need to enforce the same size of the RUNPATH entry in the
> dynamic section during linking, even though I don't fill it completely.
> Is it possible to give some flag to gnu ld that allows me to set it to a
> specific size ? Or is there a way to patch the elf file after linking,
> so that the entry has a specified size ?
>

This tool doesn't quite do what you'd like (set a fixed size for the
RUNPATH entry in the dynamic table), but I have found it quite
useful:

https://github.com/NixOS/patchelf

Tom

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Jacob Kroon]

[-- Attachment #1: Type: text/plain, Size: 1064 bytes --]

On 11/27/21 23:27, Tom Kacvinsky wrote:
> 
> 
> On Sat, Nov 27, 2021 at 5:05 PM Jacob Kroon via Gcc-help
> <gcc-help@gcc.gnu.org <mailto:gcc-help@gcc.gnu.org>> wrote:
> 
>     Hi,
> 
>     As part of an effort to make binaries reproducible regardless of their
>     build path, I need to enforce the same size of the RUNPATH entry in the
>     dynamic section during linking, even though I don't fill it completely.
>     Is it possible to give some flag to gnu ld that allows me to set it to a
>     specific size ? Or is there a way to patch the elf file after linking,
>     so that the entry has a specified size ?
> 
> 
> This tool doesn't quite do what you'd like (set a fixed size for the
> RUNPATH entry in the dynamic table), but I have found it quite
> useful:
> 
> https://github.com/NixOS/patchelf <https://github.com/NixOS/patchelf>
> 

Thanks for the tip, but I can't get patchelf to produce identical
binaries, unless the rpath is already padded up to a common size in both
of the binaries.

I've attached a small Makefile I use to test with.

Jacob

[-- Attachment #2: reproducible.mk --]
[-- Type: text/x-makefile, Size: 596 bytes --]

workdir := $(shell mktemp -d)

all : compare

compare : $(workdir)/test-a $(workdir)/test-b
	diffoscope $(workdir)/test-a $(workdir)/test-b

rpath-a = "/foobar              "
rpath-b = "/a/much/longer/foobar"

$(workdir)/test-% : $(workdir)/%/test.c
	gcc -O2 $< -o $@ -Wl,--build-id=none -Wl,--rpath=$(rpath-$*)
	#chrpath -r "replacement" $@
	patchelf --set-rpath "my-new-rpath" $@

define sourcecode
#include <stdio.h>

int main() {
	printf("HelloWorld");
	return 0;
}
endef

define newline


endef

$(workdir)/%/test.c :
	mkdir -p $(dir $@)
	echo -e '$(subst $(newline),\n,$(sourcecode))' > $@

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Florian Weimer]

* Jacob Kroon via Gcc-help:

> As part of an effort to make binaries reproducible regardless of their
> build path, I need to enforce the same size of the RUNPATH entry in the
> dynamic section during linking, even though I don't fill it completely.
> Is it possible to give some flag to gnu ld that allows me to set it to a
> specific size ? Or is there a way to patch the elf file after linking,
> so that the entry has a specified size ?

Do you want to allocate specific size so that you can patch in a
different value later?

The RUNPATH strings are in the string table, so it's necessary to
allocate space there, and be able to find it during patching.

Solaris offers this mechanism:

| DT_SUNW_STRPAD
| 
|     The total size, in bytes, of the unused reserved space at the end of
|     the dynamic string table. If DT_SUNW_STRPAD is not present in an
|     object, no reserved space is available.

Would that help in your case as well?

Thanks,
Florian

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Jacob Kroon]

On 11/28/21 12:44, Florian Weimer wrote:
> * Jacob Kroon via Gcc-help:
> 
>> As part of an effort to make binaries reproducible regardless of their
>> build path, I need to enforce the same size of the RUNPATH entry in the
>> dynamic section during linking, even though I don't fill it completely.
>> Is it possible to give some flag to gnu ld that allows me to set it to a
>> specific size ? Or is there a way to patch the elf file after linking,
>> so that the entry has a specified size ?
> 
> Do you want to allocate specific size so that you can patch in a
> different value later?
> 

Yes exactly, but see below.

> The RUNPATH strings are in the string table, so it's necessary to
> allocate space there, and be able to find it during patching.
> 
> Solaris offers this mechanism:
> 
> | DT_SUNW_STRPAD
> | 
> |     The total size, in bytes, of the unused reserved space at the end of
> |     the dynamic string table. If DT_SUNW_STRPAD is not present in an
> |     object, no reserved space is available.
> 
> Would that help in your case as well?
> 

The problem is that for two different build I pass two different
-Wl,--rpath=<path>, and they are of different length. So I'd like to
reserve a maximum size, at link-time, which becomes the same in both
builds, so that when I later remove the rpath's, the binaries become
identical.

Jacob

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Dan Kegel]

I used to do this by padding initially with extra dots and slashes.

(On Mac, the linker had a flag -headerpad_max_install_names that did
something similar.)


On Sun, Nov 28, 2021, 06:32 Jacob Kroon via Gcc-help <gcc-help@gcc.gnu.org>
wrote:

> On 11/28/21 12:44, Florian Weimer wrote:
> > * Jacob Kroon via Gcc-help:
> >
> >> As part of an effort to make binaries reproducible regardless of their
> >> build path, I need to enforce the same size of the RUNPATH entry in the
> >> dynamic section during linking, even though I don't fill it completely.
> >> Is it possible to give some flag to gnu ld that allows me to set it to a
> >> specific size ? Or is there a way to patch the elf file after linking,
> >> so that the entry has a specified size ?
> >
> > Do you want to allocate specific size so that you can patch in a
> > different value later?
> >
>
> Yes exactly, but see below.
>
> > The RUNPATH strings are in the string table, so it's necessary to
> > allocate space there, and be able to find it during patching.
> >
> > Solaris offers this mechanism:
> >
> > | DT_SUNW_STRPAD
> > |
> > |     The total size, in bytes, of the unused reserved space at the end
> of
> > |     the dynamic string table. If DT_SUNW_STRPAD is not present in an
> > |     object, no reserved space is available.
> >
> > Would that help in your case as well?
> >
>
> The problem is that for two different build I pass two different
> -Wl,--rpath=<path>, and they are of different length. So I'd like to
> reserve a maximum size, at link-time, which becomes the same in both
> builds, so that when I later remove the rpath's, the binaries become
> identical.
>
> Jacob
>

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Florian Weimer]

* Jacob Kroon:

>> The RUNPATH strings are in the string table, so it's necessary to
>> allocate space there, and be able to find it during patching.
>> 
>> Solaris offers this mechanism:
>> 
>> | DT_SUNW_STRPAD
>> | 
>> |     The total size, in bytes, of the unused reserved space at the end of
>> |     the dynamic string table. If DT_SUNW_STRPAD is not present in an
>> |     object, no reserved space is available.
>> 
>> Would that help in your case as well?
>> 
>
> The problem is that for two different build I pass two different
> -Wl,--rpath=<path>, and they are of different length. So I'd like to
> reserve a maximum size, at link-time, which becomes the same in both
> builds, so that when I later remove the rpath's, the binaries become
> identical.

In the file, there is no array of some number of characters that
contains the null-terminated RUNPATH string, an array to which just more
null bytes could be added at the end.  DT_RUNPATH contains an offset
into the string table.  The null-terminated string could be used for
something else.  For example, if the RUNPATH is
"/usr/lib64/sys/library.so.3", and the link edit needs the string
"library.so.3" for something else, it can point into the RUNPATH string
to get that.  So it's not valid in general to patch the string in-place,
let alone change its length.

That's why I think something like the Solaris approach is needed: The
new RUNPATH value would come from the reservation, the offset in
DT_RUNPATH points to it, and the original string table is not modified
at all.

Thanks,
Florian

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Jacob Kroon]

On 11/28/21 16:17, Florian Weimer wrote:
> * Jacob Kroon:
> 
>>> The RUNPATH strings are in the string table, so it's necessary to
>>> allocate space there, and be able to find it during patching.
>>>
>>> Solaris offers this mechanism:
>>>
>>> | DT_SUNW_STRPAD
>>> | 
>>> |     The total size, in bytes, of the unused reserved space at the end of
>>> |     the dynamic string table. If DT_SUNW_STRPAD is not present in an
>>> |     object, no reserved space is available.
>>>
>>> Would that help in your case as well?
>>>
>>
>> The problem is that for two different build I pass two different
>> -Wl,--rpath=<path>, and they are of different length. So I'd like to
>> reserve a maximum size, at link-time, which becomes the same in both
>> builds, so that when I later remove the rpath's, the binaries become
>> identical.
> 
> In the file, there is no array of some number of characters that
> contains the null-terminated RUNPATH string, an array to which just more
> null bytes could be added at the end.  DT_RUNPATH contains an offset
> into the string table.  The null-terminated string could be used for
> something else.  For example, if the RUNPATH is
> "/usr/lib64/sys/library.so.3", and the link edit needs the string
> "library.so.3" for something else, it can point into the RUNPATH string
> to get that.  So it's not valid in general to patch the string in-place,
> let alone change its length.
> 
> That's why I think something like the Solaris approach is needed: The
> new RUNPATH value would come from the reservation, the offset in
> DT_RUNPATH points to it, and the original string table is not modified
> at all.
> 

Aha, thank you for explaining.

I suppose gnu ld doesn't try to optimize for usage of substrings in
RUNPATH, thus making if fairly easy to change it post linking.

But can I then specify during link time that RUNTIME should go in a
"reserved" area, whose size I can also control ? I hope I understood you
correctly.

Jacob

Re: Reserving specified size of RUNPATH entry in the dynamic section during linking

[Florian Weimer]

* Jacob Kroon:

> I suppose gnu ld doesn't try to optimize for usage of substrings in
> RUNPATH, thus making if fairly easy to change it post linking.

I think it does, it's just that RUNPATH with an overlapping tail is just
rare in practice because of its contents.

Try this:

$ gcc -nostartfiles -shared -Wl,-rpath,one-long-string,-soname,one-long-string  
$ readelf -dW a.out

Dynamic section at offset 0x1f20 contains 9 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [one-long-string]
 0x000000000000001d (RUNPATH)            Library runpath: [one-long-string]
 0x000000006ffffef5 (GNU_HASH)           0x180
 0x0000000000000005 (STRTAB)             0x1b8
 0x0000000000000006 (SYMTAB)             0x1a0
 0x000000000000000a (STRSZ)              27 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000000 (NULL)               0x0

But if you look at the file with a hex editor, you'll notice that the
string is only there once:

000001b0: 0000 0000 0000 0000 006c 6962 632e 736f  .........libc.so
000001c0: 2e36 006f 6e65 2d6c 6f6e 672d 7374 7269  .6.one-long-stri
000001d0: 6e67 0000 0000 0000 0000 0000 0000 0000  ng..............

> But can I then specify during link time that RUNTIME should go in a
> "reserved" area, whose size I can also control ? I hope I understood you
> correctly.

I think GNU ld doesn't have this feature today.

Thanks,
Florian