Re: [PATCH] dl: Use "adr" assembler command to get proper load address

[Adhemerval Zanella <adhemerval.zanella@linaro.org>]

On 07/10/2021 07:00, Lukasz Majewski wrote:
> On Thu, 7 Oct 2021 11:19:26 +0200
> Lukasz Majewski <lukma@denx.de> wrote:
> 
>> Hi Szabolcs,
>>
>>> The 10/06/2021 13:43, Lukasz Majewski wrote:  
>>>> Please find in-depth analyze about the issue:
>>>>
>>>> It was tested with Beagle Bone Black (BBB) and QEMU (the same
>>>> binary rootfs images).
>>>> (If needed I will upload images and script to run QEMU to some
>>>> server for reproduction).
>>>> Branch:
>>>> https://github.com/lmajewski/y2038_glibc/commits/y2038_edge 
>>>
>>> i think it is easier to look at if you upload the broken
>>> ld.so binary. or at least readelf -aW ld.so output.  
>>
>> Please find working and broken ld-linux-armhf.so.3:
>> https://owncloud.denx.de/s/wtAfktG6pXLffSA
>>
>>>   
>>>> On working setup to trigger the core dump:
>>>> /home/root/ld-linux-armhf.so.3 /sbin/init
>>>> gdb ./ld-linux-armhf.so.3 core
>>>>
>>>> (and the /home/root/ld-linux-armhf.so.3 is the "broken" one).
>>>>
>>>>
>>>> Not working (patch [1] not applied):
>>>> ====================================
>>>>
>>>> All the code is located in _dl_start in elf/rtld.c and
>>>> elf/get-dynamic-info.h files:
>>>>
>>>> (gdb) p/x $r5
>>>> $46 = 0xb6fc8000
>>>> r5 is set from the elf_machine_load_address()
>>>>
>>>> Then we enter the elf_get_dynamic_info()
>>>>
>>>> 0xb6fc95fc      99            ADJUST_DYN_INFO (DT_SYMTAB);
>>>>    0xb6fc95f8 <_dl_start+308>:  04 30 92 15     ldrne   r3, [r2,
>>>> #4] => 0xb6fc95fc <_dl_start+312>:  05 30 83 10     addne   r3,
>>>> r3, r5 0xb6fc9600 <_dl_start+316>:  04 30 82 15     strne   r3,
>>>> [r2, #4] (gdb) p/x $r3
>>>> $63 = 0x410003f4
>>>> (gdb) p/x $r5
>>>> $64 = 0xb6fc8000    
>>>
>>> it seems r5 is already wrong here, it's not the runtime
>>> address of 0. (looks more like the runtime address of
>>> 0x41000000)  
>>
>> The r5 is set according to the change in patch which now I'm trying to
>> fix.
>>
>> To be more specific - r5 is set according to code in patch SHA1:
>> bca0f5cbc9257c13322b99e55235c4f21ba0bd82
>>
>>>
>>> likely something goes wrong with the computation of r5.  
>>
>> Before the SHA1: bca0f5cbc9257c13322b99e55235c4f21ba0bd82 patch r5 was
>> computed with using 'adr' assembler instruction, not set from some
>> constant value.
>>
>> asm ("adr %0, _dl_start" : "=r" (pcrel_addr));
>>
>> And r5 value was 0x75fc8000
>>
>>>   
>>>> Links:
>>>> [1] -
>>>> https://github.com/lmajewski/y2038_glibc/commit/e67e0f589b88a44be8f8b9b770b08950dd7e5bd5
>>>>
>>>> readelf -e ld-linux-armhf.so.3 
>>>>
>>>> [10] .plt              PROGBITS        41000994 000994 000050 04
>>>> AX  0   0  4 [11] .text             PROGBITS        41000a00
>>>> 000a00 01fed0 00  AX  0   0 64 [12] .rodata           PROGBITS
>>>> 410208d0 0208d0 004b24 00   A  0   0  4 [13] .ARM.extab
>>>> PROGBITS        410253f4 0253f4 000018 00   A  0   0  4 [14]
>>>> .ARM.exidx        ARM_EXIDX       4102540c 02540c 0000c8 00  AL 11
>>>>  0  4 [15] .data.rel.ro      PROGBITS        41036200 026200
>>>> 000cf4 00  WA  0   0  8 [16] .dynamic          DYNAMIC
>>>> 41036ef4 026ef4 0000c8 08  WA  5   0  4 [17] .got
>>>> PROGBITS 41036fbc 026fbc 000040 04  WA  0   0  4    
>>>
>>> why are all addresses >0x41000000 ?
>>> in a shared library i expect all those addresses
>>> to be close to 0.  
>>
>> On this real HW system (the rootfs which is running) - libc.so.6 also
>> has address > 0x41000000
>> libm.so.6 also has the value > 0x41200000
>> (Entry point address:               0x412c9190)
>>
>> The offset of > 0x41000000 looks a bit strange indeed, but it is still
>> less than the kernel space. Moreover, with position independent code
>> it shall not matter.
>>
>>>
>>> is this made by some modified binutils?  
>>
>> I've double checked the ld-linux-armhf.so.3 and after build it has:
>> (Entry point address:               0xa00) which seems to be correct.
>>
>> So it looks like during installation of the glibc (on the Yocto/OE
>> created rootfs) the elf header is modified and this 0x41000000 offset
>> is added.
>>
> 
> And indeed it is the case. Yocto/OE by default perform prelinking (use
> prelink program) to speedup start time of dynamic program.
> 
> The prelink [1] itself assigns some virtual addresses to all required
> shared objects (in our case for /sbin/init), so no clashes are
> encountered.
> 
> And using prelink is a _default_ behaviour in Yocto/OE poky distro.

Does it work without prelink? Also, does it fail with prelink in real
hardware?

It indeed might be a prelink issue in fact.