Question about setjmp_aux.c on MIPS

Question about setjmp_aux.c on MIPS

[Huang Pei]

Hi everyone,

I add 'dummy' event to do_cpu in Linux/MIPS, and use perf tool with
Uprobe and dummy event, found my CU1 exception came from ld.so 

For ld.so, it is not intended to use floating point instruction, but the
__sigsetjmp from _dl_catch_exception touch FP regs repeatedly as loading
DSOs, which cause unneeded FP context saving and restoring between tasks
that does not use FP at all, I wonder if this can be avoided.

+. MIPS, link other RISC with hard floating point suppport need to save 
callee-saved FP in setjmp_aux.c, but setjmp.S for x86/x86_64 DOES NOT
save any x87/SSE regs, does anyone know why?

+. Does MIPS need to save MSA reg as PowerPC does for VSX in setjmp_aux.c?

+. I see x86/x86-64/ARM64 say no need to save sigmask in ld.so, can MIPS
does this too?

Re: Question about setjmp_aux.c on MIPS

[Florian Weimer]

* Huang Pei:

> For ld.so, it is not intended to use floating point instruction, but the
> __sigsetjmp from _dl_catch_exception touch FP regs repeatedly as loading
> DSOs, which cause unneeded FP context saving and restoring between tasks
> that does not use FP at all, I wonder if this can be avoided.

I think I have posted patches to move _dl_catch_exception into ld.so
(“elf: Rework exception handling in the dynamic loader [BZ #25486]”).
Once they are merged, you could special-case setjmp for use within ld.so
not to save & restore floating point state when compiled for
IS_IN (rtld).

> +. MIPS, link other RISC with hard floating point suppport need to save 
> callee-saved FP in setjmp_aux.c, but setjmp.S for x86/x86_64 DOES NOT
> save any x87/SSE regs, does anyone know why?

The official x86-64 ABI does not have any non-volatile/callee-saved
floating point registers.  We have code to save and restore floating
point state in the lazy binding trampoline because floating point
registers are used to pass arguments and return results, but that's it.

> +. Does MIPS need to save MSA reg as PowerPC does for VSX in setjmp_aux.c?

Hard to tell, it really depends on ABI details.

> +. I see x86/x86-64/ARM64 say no need to save sigmask in ld.so, can
> MIPS does this too?

I think so, yes.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Re: Question about setjmp_aux.c on MIPS

[Huang Pei]

Hi, 
On Thu, Nov 26, 2020 at 01:37:53PM +0100, Florian Weimer wrote:
> * Huang Pei:
> 
> > For ld.so, it is not intended to use floating point instruction, but the
> > __sigsetjmp from _dl_catch_exception touch FP regs repeatedly as loading
> > DSOs, which cause unneeded FP context saving and restoring between tasks
> > that does not use FP at all, I wonder if this can be avoided.
> 
> I think I have posted patches to move _dl_catch_exception into ld.so
> (“elf: Rework exception handling in the dynamic loader [BZ #25486]”).
> Once they are merged, you could special-case setjmp for use within ld.so
> not to save & restore floating point state when compiled for
> IS_IN (rtld).
> 
I got it, there is one more __sigsetjmp call from __libc_start_main in
libc.so, not from _dl_catch_exception, like this one:
......

(gdb) c
Continuing.

Breakpoint 1, __sigsetjmp_aux (env=0xffffbff2a0, savemask=0,
sp=1099507430048, fp=0, gp=1099376871424) at
../sysdeps/mips/mips64/setjmp_aux.c:39
39        asm volatile ("s.d $f24, %0" : : "m"
(env[0].__jmpbuf[0].__fpregs[0]));
(gdb) bt
#0  __sigsetjmp_aux (env=0xffffbff2a0, savemask=0, sp=1099507430048,
fp=0, gp=1099376871424) at ../sysdeps/mips/mips64/setjmp_aux.c:39
#1  0x000000fff7ded7c0 in __libc_start_main (warning: GDB can't find the
start of the function at 0xfff7f9704f.
main=0xfff7f966e0, argc=<optimized out>, argv=0xffffbff3e0,
init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=<optimized out>) at libc-start.c:283
    #2  0x000000fff7f97050 in ?? ()
(gdb) 

......

Can __libc_start_main call this special-case setjmp?

> > +. MIPS, link other RISC with hard floating point suppport need to save 
> > callee-saved FP in setjmp_aux.c, but setjmp.S for x86/x86_64 DOES NOT
> > save any x87/SSE regs, does anyone know why?
> 
> The official x86-64 ABI does not have any non-volatile/callee-saved
> floating point registers.  We have code to save and restore floating
> point state in the lazy binding trampoline because floating point
> registers are used to pass arguments and return results, but that's it.
> 
> > +. Does MIPS need to save MSA reg as PowerPC does for VSX in setjmp_aux.c?
> 
> Hard to tell, it really depends on ABI details.
> 
> > +. I see x86/x86-64/ARM64 say no need to save sigmask in ld.so, can
> > MIPS does this too?
> 
> I think so, yes.
> 
> Thanks,
> Florian
> -- 
> Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
> Commercial register: Amtsgericht Muenchen, HRB 153243,
> Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

PS: 

libc.so in glibc-2.28 in both x86-64 and MIPS64 calling __setjmp in
ld.so instead of libc.so itself

Re: Question about setjmp_aux.c on MIPS

[Florian Weimer]

* Huang Pei:

> Hi, 
> On Thu, Nov 26, 2020 at 01:37:53PM +0100, Florian Weimer wrote:
>> * Huang Pei:
>> 
>> > For ld.so, it is not intended to use floating point instruction, but the
>> > __sigsetjmp from _dl_catch_exception touch FP regs repeatedly as loading
>> > DSOs, which cause unneeded FP context saving and restoring between tasks
>> > that does not use FP at all, I wonder if this can be avoided.
>> 
>> I think I have posted patches to move _dl_catch_exception into ld.so
>> (“elf: Rework exception handling in the dynamic loader [BZ #25486]”).
>> Once they are merged, you could special-case setjmp for use within ld.so
>> not to save & restore floating point state when compiled for
>> IS_IN (rtld).
>> 
> I got it, there is one more __sigsetjmp call from __libc_start_main in
> libc.so, not from _dl_catch_exception, like this one:
> ......
>
> (gdb) c
> Continuing.
>
> Breakpoint 1, __sigsetjmp_aux (env=0xffffbff2a0, savemask=0,
> sp=1099507430048, fp=0, gp=1099376871424) at
> ../sysdeps/mips/mips64/setjmp_aux.c:39
> 39        asm volatile ("s.d $f24, %0" : : "m"
> (env[0].__jmpbuf[0].__fpregs[0]));
> (gdb) bt
> #0  __sigsetjmp_aux (env=0xffffbff2a0, savemask=0, sp=1099507430048,
> fp=0, gp=1099376871424) at ../sysdeps/mips/mips64/setjmp_aux.c:39
> #1  0x000000fff7ded7c0 in __libc_start_main (warning: GDB can't find the
> start of the function at 0xfff7f9704f.
> main=0xfff7f966e0, argc=<optimized out>, argv=0xffffbff3e0,
> init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
>     stack_end=<optimized out>) at libc-start.c:283
>     #2  0x000000fff7f97050 in ?? ()
> (gdb) 
>
> ......
>
> Can __libc_start_main call this special-case setjmp?

It should be possible if __libc_start_main is also built in such a way
that it does not floating-point registers.  __libc_start_main does not
return into user code, so it does not matter if it actually preserves
callee-saved registers.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill