Re: [PATCH] lto-plugin: add support for feature detection

["Martin Liška" <mliska@suse.cz>]

[-- Attachment #1: Type: text/plain, Size: 4536 bytes --]

On 5/16/22 12:28, Richard Biener wrote:
> On Mon, May 16, 2022 at 11:58 AM Rui Ueyama <rui314@gmail.com> wrote:
>>
>> Version handshaking is doable, but it feels like we are over-designing
>> an API, given that the real providers of this plugin API are only GCC
>> and LLVM and the users of the API are BFD ld, gold and mold. It is
>> unlikely that we'll have dozens of more compilers or linkers in the
>> near future. So, I personally prefer the following style
>>
>> if (!strcmp(plugin_compiler_name, "gcc") && plugin_major >= 12)
>>
>> than versioning various API-provided functions. It'll just work and be
>> easy to understand.
>>
>> Besides that, even if we version GCC-provided plugin API functions, we
>> still need a logic similar to the above to distinguish GCC from LLVM,
>> as they behave slightly differently in various corner cases. We can't
>> get rid of the heuristic version detection logic from the linker
>> anyways.
>>
>> So, from the linker's point of view, exporting a compiler name and
>> version numbers is enough.
> 
> I agree that this might be convenient enough but the different behaviors
> are because of inadequate documentation of the API - that's something
> we should fix.  And for this I think a plugin API version might help
> as we can this way also handle your case of the need of querying
> whether v2 will be used or not.
> 
> I wouldn't go to enumerate past API versions - the version handshake
> hook requirement alone makes that not so useful.  Instead I'd require
> everybody implementing the handshare hook implementing also all
> other hooks defined at that point in time and set the version to 1.
> 
> I'd eventually keep version 2 to indicate thread safety (of a part of the API).

That seems reasonable to me.

> 
> That said, I'm not opposed to add a "GCC 12.1" provider, maybe the
> version handshake should be
> 
> int api_version (int linker, const char **identifier);
> 
> where the linker specifies the desired API version and passes an
> identifier identifying itself ("mold 1.0") and it will get back the API
> version the plugin intends to use plus an identifier of the plugin
> ("GCC 12.1").

I've implemented first version of the patch, please take a look.

Note there's a bit name collision of api_version with:

/* The version of the API specification.  */

enum ld_plugin_api_version
{
  LD_PLUGIN_API_VERSION = 1
};

@Rui: Am I correct that you're interested in thread-safe claim_file? Is there any
other function being called paralely?

Thoughts?

> 
> Richard.
> 
>>
>>
>> On Mon, May 16, 2022 at 5:38 PM Martin Liška <mliska@suse.cz> wrote:
>>>
>>> On 5/16/22 11:25, Jan Hubicka via Gcc-patches wrote:
>>>>>
>>>>> Sure having a 'plugin was compiled from sources of the GCC N.M compiler'
>>>>> is useful if bugs are discovered in old versions that you by definition cannot
>>>>> fix but can apply workarounds to.  Note the actual compiler used might still
>>>>> differ.  Note that still isn't clean API documentation / extension of the plugin
>>>>> API itself.  As of thread safety we can either add a claim_file_v2_hook
>>>>> or try to do broader-level versioning of the API with a new api_version
>>>>> hook which could also specify that with say version 2 the plugin will
>>>>> not use get_symbols_v2 but only newer, etc.  The linker would announce
>>>>> the API version it likes to use and the plugin would return the
>>>>> (closest) version
>>>>> it can handle.  A v2 could then also specify that claim_file needs to be
>>>>
>>>> Yep, I think having the API version handshake is quite reasonable way to
>>>> go as the _v2,_v3 symbols seems already getting bit out of hand and we
>>>> essentially have 3 revisions of API to think of
>>>>  (first adding LDPR_PREVAILING_DEF_IRONLY_EXP, second adding
>>>>  GET_SYMBOLS_V3 and now we plan third adding thread safety and solving
>>>>  the file handler problems)
>>>
>>> How should be design such a version handshake?
>>>
>>>>
>>>>> thread safe, or that cleanup should allow a followup onload again with
>>>>> a state identical to after dlopen, etc.
>>>>>
>>>>> Is there an API document besides the header itself somewhere?
>>>>
>>>> There is https://gcc.gnu.org/wiki/whopr/driver
>>>> I wonder if this can't be moved into more official looking place since
>>>> it looks like it is internal to GCC WHOPR implementation this way.
>>>
>>> We can likely add it here:
>>> https://gcc.gnu.org/onlinedocs/gccint/LTO.html#LTO
>>>
>>> What do you think? I can port it.
>>>
>>> Martin
>>>
>>>>
>>>> Honza
>>>>>
>>>>> Richard.
>>>

[-- Attachment #2: 0001-Implement-LDPT_REGISTER_GET_API_VERSION.patch --]
[-- Type: text/x-patch, Size: 4053 bytes --]

From 0ef5f678144b8ff3a1d247a992aa2e14128b82d1 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Mon, 16 May 2022 14:01:52 +0200
Subject: [PATCH] Implement LDPT_REGISTER_GET_API_VERSION.

---
 include/plugin-api.h    | 14 ++++++++++++++
 lto-plugin/lto-plugin.c | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/include/plugin-api.h b/include/plugin-api.h
index 8aebe2ff267..f01f4301fc9 100644
--- a/include/plugin-api.h
+++ b/include/plugin-api.h
@@ -483,6 +483,18 @@ enum ld_plugin_level
   LDPL_FATAL
 };
 
+/* The linker's interface for API version negotiation.  */
+
+typedef
+int (*ld_plugin_get_api_version) (char *linker_identifier, int linker_version,
+				  int preferred_linker_api,
+				  const char **compiler_identifier,
+				  int *compiler_version);
+
+typedef
+enum ld_plugin_status
+(*ld_plugin_register_get_api_version) (ld_plugin_get_api_version handler);
+
 /* Values for the tv_tag field of the transfer vector.  */
 
 enum ld_plugin_tag
@@ -521,6 +533,7 @@ enum ld_plugin_tag
   LDPT_REGISTER_NEW_INPUT_HOOK,
   LDPT_GET_WRAP_SYMBOLS,
   LDPT_ADD_SYMBOLS_V2,
+  LDPT_REGISTER_GET_API_VERSION,
 };
 
 /* The plugin transfer vector.  */
@@ -556,6 +569,7 @@ struct ld_plugin_tv
     ld_plugin_get_input_section_size tv_get_input_section_size;
     ld_plugin_register_new_input tv_register_new_input;
     ld_plugin_get_wrap_symbols tv_get_wrap_symbols;
+    ld_plugin_register_get_api_version tv_register_get_api_version;
   } tv_u;
 };
 
diff --git a/lto-plugin/lto-plugin.c b/lto-plugin/lto-plugin.c
index 00b760636dc..49484decd89 100644
--- a/lto-plugin/lto-plugin.c
+++ b/lto-plugin/lto-plugin.c
@@ -69,6 +69,7 @@ along with this program; see the file COPYING3.  If not see
 #include "../gcc/lto/common.h"
 #include "simple-object.h"
 #include "plugin-api.h"
+#include "ansidecl.h"
 
 /* We need to use I64 instead of ll width-specifier on native Windows.
    The reason for this is that older MS-runtimes don't support the ll.  */
@@ -166,6 +167,10 @@ static ld_plugin_add_input_file add_input_file;
 static ld_plugin_add_input_library add_input_library;
 static ld_plugin_message message;
 static ld_plugin_add_symbols add_symbols, add_symbols_v2;
+static ld_plugin_register_get_api_version register_get_api_version;
+
+/* By default, use version 1 if there is not negotiation.  */
+static int used_api_version = 1;
 
 static struct plugin_file_info *claimed_files = NULL;
 static unsigned int num_claimed_files = 0;
@@ -1407,6 +1412,29 @@ process_option (const char *option)
   verbose = verbose || debug;
 }
 
+static int
+get_api_version (char *linker_identifier, int linker_version,
+		 int preferred_linker_api,
+		 const char **compiler_identifier,
+		 int *compiler_version)
+{
+  *compiler_identifier = "GCC";
+  *compiler_version = GCC_VERSION;
+
+  if (preferred_linker_api >= 2)
+    {
+      check (get_symbols_v3, LDPL_FATAL,
+	     "get_symbols_v3 requires for API version 2");
+      check (add_symbols_v2, LDPL_FATAL,
+	     "add_symbols_v2 requires for API version 2");
+
+      /* The plug-in supports version 2.  */
+      used_api_version = 2;
+    }
+
+  return used_api_version;
+}
+
 /* Called by a linker after loading the plugin. TV is the transfer vector. */
 
 enum ld_plugin_status
@@ -1467,12 +1495,22 @@ onload (struct ld_plugin_tv *tv)
 	  /* We only use this to make user-friendly temp file names.  */
 	  link_output_name = p->tv_u.tv_string;
 	  break;
+	case LDPT_REGISTER_GET_API_VERSION:
+	  register_get_api_version = p->tv_u.tv_register_get_api_version;
+	  break;
 	default:
 	  break;
 	}
       p++;
     }
 
+  if (register_get_api_version)
+    {
+      status = register_get_api_version (get_api_version);
+      check (status == LDPS_OK, LDPL_FATAL,
+	     "could not register the get_api_version callback");
+    }
+
   check (register_claim_file, LDPL_FATAL, "register_claim_file not found");
   check (add_symbols, LDPL_FATAL, "add_symbols not found");
   status = register_claim_file (claim_file_handler);
-- 
2.36.1