RE: [EXTERNAL] Re: [PATCH] Collect both user and kernel events for autofdo tests and autoprofiledbootstrap

[Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>]

There is no warning and perf /uk succeeds when kptr_restrict is set to 1 and perf_event_paranoid set to 2. However, create_gcov may fail since it won't be able to understand kernel addresses and it requires at least 95% of events to be successfully mapped.

If I set both kptr_restrict and perf_event_paranoid to 1, then I do get warnings from perf (but it still succeeds and exits with a 0 code). And, of course create_gcov will also fail to map some events since it won't understand kernel addresses.

WARNING: Kernel address maps (/proc/{kallsyms,modules}) are restricted,
check /proc/sys/kernel/kptr_restrict and /proc/sys/kernel/perf_event_paranoid.

Samples in kernel functions may not be resolved if a suitable vmlinux
file is not found in the buildid cache or in the vmlinux path.

Samples in kernel modules won't be resolved at all.

If some relocation was applied (e.g. kexec) symbols may be misresolved
even with a suitable vmlinux or kallsyms file.

Couldn't record kernel reference relocation symbol
Symbol resolution may be skewed if relocation was used (e.g. kexec).
Check /proc/kallsyms permission or run as root.
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.037 MB /home/erozen/gcc1_objdir/gcc/testsuite/gcc/indir-call-prof.perf.data (86 samples) ]

Eugene

-----Original Message-----
From: Richard Biener <richard.guenther@gmail.com> 
Sent: Monday, July 3, 2023 12:47 AM
To: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>
Cc: Sam James <sam@gentoo.org>; gcc-patches@gcc.gnu.org
Subject: Re: [EXTERNAL] Re: [PATCH] Collect both user and kernel events for autofdo tests and autoprofiledbootstrap

On Sat, Jul 1, 2023 at 12:05 AM Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com> wrote:
>
> I also set /proc/sys/kernel/perf_event_paranoid to 1 instead of the default 2.

Does the perf attempt fail when the privileges are not adjusted and you specify --all?  I see it adds /uk as flags, when I do

> perf record -e instructions//uk ./a.out

it doesn't complain in any way with

> cat /proc/sys/kernel/kptr_restrict
1
> cat /proc/sys/kernel/perf_event_paranoid
2

so in case the 'kernel' side is simply ignored when profiling there isn't permitted/possible then I guess the patch is OK?

Can you confirm?

Thanks,
Richard.

> -----Original Message-----
> From: Gcc-patches 
> <gcc-patches-bounces+erozen=microsoft.com@gcc.gnu.org> On Behalf Of 
> Eugene Rozenfeld via Gcc-patches
> Sent: Friday, June 30, 2023 2:44 PM
> To: Sam James <sam@gentoo.org>; Richard Biener 
> <richard.guenther@gmail.com>
> Cc: gcc-patches@gcc.gnu.org
> Subject: RE: [EXTERNAL] Re: [PATCH] Collect both user and kernel 
> events for autofdo tests and autoprofiledbootstrap
>
> I don't run this with elevated privileges but I set /proc/sys/kernel/kptr_restrict to 0. Setting that does require elevated privileges.
>
> If that's not acceptable, the only fix I can think of is to make that event mapping threshold percentage a parameter to create_gcov and pass something low enough. 80% instead of the current threshold of 95% should work, although it's a bit fragile.
>
> Eugene
>
> -----Original Message-----
> From: Sam James <sam@gentoo.org>
> Sent: Friday, June 30, 2023 1:59 AM
> To: Richard Biener <richard.guenther@gmail.com>
> Cc: Eugene Rozenfeld <Eugene.Rozenfeld@microsoft.com>; 
> gcc-patches@gcc.gnu.org
> Subject: [EXTERNAL] Re: [PATCH] Collect both user and kernel events 
> for autofdo tests and autoprofiledbootstrap
>
> [You don't often get email from sam@gentoo.org. Learn why this is 
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Richard Biener via Gcc-patches <gcc-patches@gcc.gnu.org> writes:
>
> > On Fri, Jun 30, 2023 at 7:28 AM Eugene Rozenfeld via Gcc-patches 
> > <gcc-patches@gcc.gnu.org> wrote:
> >>
> >> When we collect just user events for autofdo with lbr we get some 
> >> events where branch sources are kernel addresses and branch targets 
> >> are user addresses. Without kernel MMAP events create_gcov can't 
> >> make sense of kernel addresses. Currently create_gcov fails if it 
> >> can't map at least 95% of events. We sometimes get below this threshold with just user events. The change is to collect both user events and kernel events.
> >
> > Does this require elevated privileges?  Can we instead "fix" create_gcov here?
>
> Right, requiring privileges for this is going to be a no-go for a lot of builders. In a distro context, for example, it means we can't consider autofdo at all.