[Bug dynamic-link/27220] New: Migrate away from nested functions

[Bug dynamic-link/27220] New: Migrate away from nested functions

[i at maskray dot me]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

            Bug ID: 27220
           Summary: Migrate away from nested functions
           Product: glibc
           Version: unspecified
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: dynamic-link
          Assignee: unassigned at sourceware dot org
          Reporter: i at maskray dot me
  Target Milestone: ---

Some files use this obscure GCC feature 
https://gcc.gnu.org/onlinedocs/gcc/Nested-Functions.html

Most of them are loader code. These instances are likely the biggest problem
preventing glibc from being built with Clang.

posix/regcomp.c
nss/makedb.c
elf/do-rel.h
elf/dynamic-link.h
elf/get-dynamic-info.h
elf/dl-reloc-static-pie.c
elf/dl-reloc.c
elf/dl-conflict.c
elf/rtld.c
sysdeps/aarch64/dl-machine.h
sysdeps/x86_64/dl-machine.h
sysdeps/powerpc/powerpc64/dl-machine.h

For many instances, migrating away from nested functions is a clear readability
win, even for GCC, e.g. a #include "dynamic-link.h" in the middle of
elf/dl-reloc.c , which uses RESOLVE_MAP/RESOLVE/RESOLVE_CONFLICT_FIND_MAP
defined above.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |carlos at redhat dot com

--- Comment #1 from Carlos O'Donell <carlos at redhat dot com> ---
I agree that removing nested functions is a win, both in readability *and*
debugging. The feature has never been well supported across the toolchain. The
glibc community has already reviewed and accepted other patches to remove
nested functions. I don't think there are any non-technical blockers here, we
just need someone to spend the time and effort to carry out the refactoring and
engage the community for review and commits.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[stanshebs at earthlink dot net]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

Stan Shebs <stanshebs at earthlink dot net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |stanshebs at earthlink dot net

--- Comment #2 from Stan Shebs <stanshebs at earthlink dot net> ---
The google/grte/v5-2.27/master branch has the necessary code, and it's in
production use at Google, so we have some confidence in its correctness. :-)

The essence of the dynamic linking change is to pass the bootstrap map as an
argument.  Not too complicated, but there is a messy aspect in that every
elf_machine_rela in sysdeps needs an additional argument.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

--- Comment #3 from Carlos O'Donell <carlos at redhat dot com> ---
(In reply to Stan Shebs from comment #2)
> The google/grte/v5-2.27/master branch has the necessary code, and it's in
> production use at Google, so we have some confidence in its correctness. :-)

Great to hear.

> The essence of the dynamic linking change is to pass the bootstrap map as an
> argument.  Not too complicated, but there is a messy aspect in that every
> elf_machine_rela in sysdeps needs an additional argument.

Sure. Refactoring always has some "messiness." It would be great to see those
patches split up logically and sent to libc-alpha for review. If they are
mechanical it should be easy to review them.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[tbaeder at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

Timm Bäder <tbaeder at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tbaeder at redhat dot com

--- Comment #4 from Timm Bäder <tbaeder at redhat dot com> ---
Is anyone working on getting the nested-function related changes from
google/grte/v5-2.27/master merged? What about the other clang build fixes in
that branch?

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[i at maskray dot me]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

--- Comment #5 from Fangrui Song <i at maskray dot me> ---
(In reply to Timm Bäder from comment #4)
> Is anyone working on getting the nested-function related changes from
> google/grte/v5-2.27/master merged? What about the other clang build fixes in
> that branch?

https://sourceware.org/pipermail/libc-alpha/2021-August/130391.html
("[PATCH] elf: Avoid nested functions in the loader (all ports) [BZ #27220]")

This patch fixes all ports.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

Carlos O'Donell <carlos at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
   Last reconfirmed|                            |2022-04-12
             Status|UNCONFIRMED                 |NEW

--- Comment #6 from Carlos O'Donell <carlos at redhat dot com> ---
In glibc 2.35 we committed this change which fixes all sources and avoid nested
functions.

commit 490e6c62aa31a8aa5c4a059f6e646ede121edf0a
Author: Fangrui Song <maskray@google.com>
Date:   Thu Oct 7 11:55:02 2021 -0700

    elf: Avoid nested functions in the loader [BZ #27220]

    dynamic-link.h is included more than once in some elf/ files (rtld.c,
    dl-conflict.c, dl-reloc.c, dl-reloc-static-pie.c) and uses GCC nested
    functions. This harms readability and the nested functions usage
    is the biggest obstacle prevents Clang build (Clang doesn't support GCC
    nested functions).

    The key idea for unnesting is to add extra parameters (struct link_map
    *and struct r_scope_elm *[]) to RESOLVE_MAP,
    ELF_MACHINE_BEFORE_RTLD_RELOC, ELF_DYNAMIC_RELOCATE, elf_machine_rel[a],
    elf_machine_lazy_rel, and elf_machine_runtime_setup. (This is inspired
    by Stan Shebs' ppc64/x86-64 implementation in the
    google/grte/v5-2.27/master which uses mixed extra parameters and static
    variables.)

    Future simplification:
    * If mips elf_machine_runtime_setup no longer needs RESOLVE_GOTSYM,
      elf_machine_runtime_setup can drop the `scope` parameter.
    * If TLSDESC no longer need to be in elf_machine_lazy_rel,
      elf_machine_lazy_rel can drop the `scope` parameter.

    Tested on aarch64, i386, x86-64, powerpc64le, powerpc64, powerpc32,
    sparc64, sparcv9, s390x, s390, hppa, ia64, armhf, alpha, and mips64.
    In addition, tested build-many-glibcs.py with
{arc,csky,microblaze,nios2}-linux-gnu
    and riscv64-linux-gnu-rv64imafdc-lp64d.

    Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>


However this is not yet complete because configure sources still have tests
with nested functions that could fail when using clang during configure. So
this is almost complete, but not quite there. Adhemerval is working on full
clang support in the branch azanella/clang which includes the further required
fixes.

The last part required to fix this bug is here (from azanella/clang):
https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=b4338da21fc709d408ed2bb0f93f082838de0d1c;hp=2116e8544f0e21767dd0c1b9a97f4d4c09a7a68a

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[carlos at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

--- Comment #7 from Carlos O'Donell <carlos at redhat dot com> ---
Removed further nested functions.
~~~
commit 453b88efe6fa79f5c7c6fccc3a520c75fdd43074
Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date:   Tue Jul 26 10:34:01 2022 -0300

    arm: Remove nested functionf rom relocate_pc24

    Checked on arm-linux-gnueabihf.
~~~

Do we have any more nested functions left?

Otherwise this should be marked CLOSED/FIXED with Target Milestone set to 2.36
since glibc-2.36 would have been released with no nested functions.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[adhemerval.zanella at linaro dot org]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

Adhemerval Zanella <adhemerval.zanella at linaro dot org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |adhemerval.zanella at linaro dot o
                   |                            |rg

--- Comment #8 from Adhemerval Zanella <adhemerval.zanella at linaro dot org> ---
(In reply to Carlos O'Donell from comment #7)
> Removed further nested functions.
> ~~~
> commit 453b88efe6fa79f5c7c6fccc3a520c75fdd43074
> Author: Adhemerval Zanella <adhemerval.zanella@linaro.org>
> Date:   Tue Jul 26 10:34:01 2022 -0300
> 
>     arm: Remove nested functionf rom relocate_pc24
>     
>     Checked on arm-linux-gnueabihf.
> ~~~
> 
> Do we have any more nested functions left?
> 
> Otherwise this should be marked CLOSED/FIXED with Target Milestone set to
> 2.36 since glibc-2.36 would have been released with no nested functions.

I think that is the last usage, at least for x86 and ARM.  I think we can close
it.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug dynamic-link/27220] Migrate away from nested functions

[i at maskray dot me]

https://sourceware.org/bugzilla/show_bug.cgi?id=27220

Fangrui Song <i at maskray dot me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|---                         |2.36
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #9 from Fangrui Song <i at maskray dot me> ---
Resolved as of 2.36.

-- 
You are receiving this mail because:
You are on the CC list for the bug.