Re: [PATCH 2/3] lto-plugin: make claim_file_handler thread-safe

[Richard Biener <richard.guenther@gmail.com>]

> Am 21.06.2022 um 10:43 schrieb Martin Liška <mliska@suse.cz>:
> 
> On 6/21/22 09:56, Richard Biener wrote:
>>> On Mon, Jun 20, 2022 at 12:20 PM Martin Liška <mliska@suse.cz> wrote:
>>> 
>>> On 6/20/22 11:32, Richard Biener wrote:
>>>> On Thu, Jun 16, 2022 at 9:01 AM Martin Liška <mliska@suse.cz> wrote:
>>>>> 
>>>>> lto-plugin/ChangeLog:
>>>>> 
>>>>>        * lto-plugin.c (plugin_lock): New lock.
>>>>>        (claim_file_handler): Use mutex for critical section.
>>>>>        (onload): Initialize mutex.
>>>>> ---
>>>>> lto-plugin/lto-plugin.c | 16 +++++++++++++++-
>>>>> 1 file changed, 15 insertions(+), 1 deletion(-)
>>>>> 
>>>>> diff --git a/lto-plugin/lto-plugin.c b/lto-plugin/lto-plugin.c
>>>>> index 00b760636dc..13118c4983c 100644
>>>>> --- a/lto-plugin/lto-plugin.c
>>>>> +++ b/lto-plugin/lto-plugin.c
>>>>> @@ -55,6 +55,7 @@ along with this program; see the file COPYING3.  If not see
>>>>> #include <unistd.h>
>>>>> #include <fcntl.h>
>>>>> #include <sys/types.h>
>>>>> +#include <pthread.h>
>>>> 
>>>> Not sure if we support any non-pthread target for building the LTO
>>>> plugin, but it
>>>> seems we have
>>>> 
>>>>  # Among non-ELF, only Windows platforms support the lto-plugin so far.
>>>>  # Build it unless LTO was explicitly disabled.
>>>>  case $target in
>>>>    *-cygwin* | *-mingw*) build_lto_plugin=$enable_lto ;;
>>>> 
>>>> which suggests that at least build validating the above with --enable-lto
>>> 
>>> Verified that it's fine.
>>> 
>>>> 
>>>> IIRC we have gthr-*.h in libgcc/, not sure if that's usable in a
>>>> host linker plugin.
>>>> 
>>>>> #ifdef HAVE_SYS_WAIT_H
>>>>> #include <sys/wait.h>
>>>>> #endif
>>>>> @@ -157,6 +158,9 @@ enum symbol_style
>>>>>   ss_uscore,   /* Underscore prefix all symbols.  */
>>>>> };
>>>>> 
>>>>> +/* Plug-in mutex.  */
>>>>> +static pthread_mutex_t plugin_lock;
>>>>> +
>>>>> static char *arguments_file_name;
>>>>> static ld_plugin_register_claim_file register_claim_file;
>>>>> static ld_plugin_register_all_symbols_read register_all_symbols_read;
>>>>> @@ -1262,15 +1266,18 @@ claim_file_handler (const struct ld_plugin_input_file *file, int *claimed)
>>>>>                              lto_file.symtab.syms);
>>>>>       check (status == LDPS_OK, LDPL_FATAL, "could not add symbols");
>>>>> 
>>>>> +      pthread_mutex_lock (&plugin_lock);
>>>>>       num_claimed_files++;
>>>>>       claimed_files =
>>>>>        xrealloc (claimed_files,
>>>>>                  num_claimed_files * sizeof (struct plugin_file_info));
>>>>>       claimed_files[num_claimed_files - 1] = lto_file;
>>>>> +      pthread_mutex_unlock (&plugin_lock);
>>>>> 
>>>>>       *claimed = 1;
>>>>>     }
>>>>> 
>>>>> +  pthread_mutex_lock (&plugin_lock);
>>>>>   if (offload_files == NULL)
>>>>>     {
>>>>>       /* Add dummy item to the start of the list.  */
>>>>> @@ -1333,11 +1340,12 @@ claim_file_handler (const struct ld_plugin_input_file *file, int *claimed)
>>>>>        offload_files_last_lto = ofld;
>>>>>       num_offload_files++;
>>>>>     }
>>>>> +  pthread_mutex_unlock (&plugin_lock);
>>>>> 
>>>>>   goto cleanup;
>>>>> 
>>>>>  err:
>>>>> -  non_claimed_files++;
>>>>> +  __atomic_fetch_add (&non_claimed_files, 1, __ATOMIC_RELAXED);
>>>> 
>>>> is it worth "optimizing" this with yet another need for target specific support
>>>> (just use pthread_mutex here as well?)
>>> 
>>> Sure.
>>> 
>>> May I install the patch with the change?
>> 
>> Can you at least add a configure check for pthread.h and maybe disable
>> locking when not found or erroring out?  I figure we have GCC_AC_THREAD_HEADER
> 
> All right, let's error out then.
> 
>> for the gthr.h stuff using $target_thread_file (aka --enable-threads=XYZ),
>> but as said that's for the target and I don't see any host uses.  We might also
>> add an explicit list of hosts (*-linux*?) where we enable thread support for
>> lto-plugin, providing opt-in (so you'd have to wrap the mutex taking or
>> if-def it out).
>> 
>> I think you also need to link lto-plugin with -pthread, no?
> 
> Yep.
> 
> Please see the updated patch.
> 

Please use -pthread instead of -lpthread 

Otherwise looks OK to me.

>> On linux
>> it might work omitting that but I'm not sure other libc have serial pthread
>> stubs in their libc.  BFD ld definitely doesn't link against pthread so
>> dlopening lto-plugin will fail (also not all libc might like
>> initializing threads
>> from a dlopen _init).
> 
> What initializing threads do you mean?

It’s target dependent what kind of global state init need to be done when any pthread facility is used.

Richard 

> Martin
> 
>> 
>> Richard.
>> 
>>> Cheers,
>>> Martin
>>> 
>>>> 
>>>>>   free (lto_file.name);
>>>>> 
>>>>>  cleanup:
>>>>> @@ -1415,6 +1423,12 @@ onload (struct ld_plugin_tv *tv)
>>>>>   struct ld_plugin_tv *p;
>>>>>   enum ld_plugin_status status;
>>>>> 
>>>>> +  if (pthread_mutex_init (&plugin_lock, NULL) != 0)
>>>>> +    {
>>>>> +      fprintf (stderr, "mutex init failed\n");
>>>>> +      abort ();
>>>>> +    }
>>>>> +
>>>>>   p = tv;
>>>>>   while (p->tv_tag)
>>>>>     {
>>>>> --
>>>>> 2.36.1
>>>>> 
>>>>> 
> <0001-lto-plugin-make-claim_file_handler-thread-safe.patch>