Re: [RFC,patch] Linker plugin - extend API for offloading corner case (aka: LDPT_REGISTER_CLAIM_FILE_HOOK_V2 linker plugin hook [GCC PR109128])

[Tobias Burnus <tobias@codesourcery.com>]

Based on the the discussion with Richard B. on #gcc, some more details why the linker does
(and might want) to call the plugin for files it does not need:

For LTO with no-fat binaries, a symbol in a static library might still be needed but the linker
does not know as the file might only contain LTO objects. The linker then calls:

   bfd_link_plugin_object_p  -> ld_plugin_object_p
which calls in case of GCC's lto-plugin:
   claim_file_handler

* The latter file checks for LTO symbols and, if found, sets *claimed = true and
   registers the found symbols via add_symbols
→ The linker than ignores all other symbols and only uses the plugin-provided symbols.

* When all files have been processed, the linker calls
   all_symbols_read_handler and the lto-plugin can ask (get_symbols) whether a given
   symbol was used.


For the offload usage, the problem is:

* The code currently assumes that a file is used when 'claim_file_handler' is invoked
   by the linker
→ It cannot claim the file itself as then symbols in the file are ignored by the linker.
→ Thus, if the file contains host-side LTO and is *claimed = true, it is fine.
   Likewise, if symbols in the file cause the linker itself to use the file, it is fine.
   However, if neither the plugin claims the file nor the file is linked via the linker,
   there is a problem (→ mismatch between host and device side).

Inside the all_symbols_read_handler, GCC's lto-plugin.cc could remove a file from the
to-be-used for device-side lto-wrapper list, but there is no way to ask the linker if
a given file is will be linked/is used or not. (Only per-symbol enquiringly for LTO exists.)

The additional flag now permits for the offload use to ignore the file if the linker has
no use for it (and passes it only for non-fat symbols host-side LTO check to the linker);
thus, we can then ignore the file for offloading purpose, unless it is used for LTO - but
that's something the plugin knows itself.


Note: That's really only an issue about whether a file (from a static library) is used
and not whether a certain symbol is used. For the host side, the function-pointer table
is constructed by section merging and the device side also forces the output of the table.
But all output forcing/keep-symbol handling fails if the whole file is dropped on one side
(linker for the host side) but not on the other (linker plugin for the non-host offload side).


I hope this clarifies the problem, background and solution a bit better.

Tobias

On 02.05.23 19:19, Tobias Burnus wrote:
> See also https://gcc.gnu.org/PR109128 (+ description in the patch log)
>
> The linker plugin API was designed to handle LTO - such that the
> compiler (i.e. GCC's lto-plugin)
> can claim an input file if it finds LTO code. In that case, the
> symbols inside that file are ignored
> by 'ld'.
>
> However, GCC also uses the LTO for offloading: code designated for
> running on a non-host device
> (GPUs) is saved in a special section in LTO format. This code then
> ends up being compiled for
> offloading but otherwise not the file is not claimed, keeping the
> symbols for 'ld' to process,
> unless that file is also uses real, host-side LTO.
>
> This mostly works okay, but a corner case exists (see PR for an
> example) where 'ld' calls the
> GCC's lto-plugin but does not actually use the symbols of that file.
> That's fine, in principle,
> but if that file contains offloading code, there is a problem: To
> match host and device functions,
> a table is created on both sides, but that table obviously must match.
> However, when lto-plugin's
> offload code processes those while ld does not link them, it fails.
>
> It turned out (kudos to Joseph for debugging + writing the patches)
> that in that case ld.bfd does
> not actually regards that file as being used but just offers it to
> llto-plugin in case it needs
> symbols from it.
>
> To get this working, the current API is insufficient.
>
> Possible solutions:
> * Tell lto-plugin whether 'ld' actually needs symbols from a file or
> it just offers the file
>   in case that lto-plugin wants to claim that file
>   => That's implemented in the attached patch.
> * Make it possible to "claim" a file without discarding the ld-visible
> symbols
> * Asking the linker later whether the file/some symbols are actually
> used.
> * something else ...
>
>
> What this patch does:
> * It adds a new API callback (LDPT_REGISTER_CLAIM_FILE_HOOK_V2) that
> takes an additional
>   boolean argument which states whether ld.bdf intens to use that
> file/symbols from that
>   file or whether it just asks the plugin in case it wants to claim it.
> * On the ld.bfd side, it wires this up.
> * On the GCC lto-plugin side, it uses that API is available, otherwise
> it uses the existing API.
>
> The way the linker plugin handling is written, it works fine at
> runtime if only one side
> supports the new hook. (Except, of course, that for fixing the issue
> both need to support it.)
>
> Regarding those patches: Are they ok for mainline? Any comment, better
> approach, suggestion?
>
> Tobias
>
> PS: Attached is the Binutils' ld side of the patch and the GCC
> lto-plugin side of the patch set.
> -----------------
> Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße
> 201, 80634 München; Gesellschaft mit beschränkter Haftung;
> Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft:
> München; Registergericht München, HRB 106955
-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955