Re: [PATCH] jit, Darwin: Implement library exports list.

[Iain Sandoe <iain@sandoe.co.uk>]

[-- Attachment #1: Type: text/plain, Size: 3508 bytes --]

Hi David,

> On 25 Jan 2024, at 10:16, Iain Sandoe <iain@sandoe.co.uk> wrote:
> 

>> On 24 Jan 2024, at 18:31, David Malcolm <dmalcolm@redhat.com> wrote:
>> 
>> On Tue, 2024-01-16 at 11:10 +0000, Iain Sandoe wrote:
>>> Tested on x86_64, i686 Darwin and x86_64 Linux,
>>> OK for trunk? when ?
>>> thanks,
>>> Iain
>> 
>> Hi Iain, thanks for the patch.
>> 
>> I'll have to defer to your Darwin expertise here; given that you've
>> tested it on the above configurations I'll assume it's correct, but...
>> 
>>> 
>>> --- 8< ---
>>> 
>>> Currently, we have no exports list for libgccjit, which means that
>>> all symbols are exported, including those from libstdc++ which is
>>> linked statically into the lib.  This causes failures when the
>>> shared libstdc++ is used but some c++ symbols are satisfied from
>>> libgccjit.
>>> 
>>> This implements an export file for Darwin (which is currently
>>> manually created by cross-checking libgccjit.map).
>> 
>> ...I'm a little nervous about this; Antoyo has a number of out-of-tree
>> patches we're working towards merging, and almost all of these touch
>> libgccjit.map.
>> 
>> 
>>>  Ideally we'd
>>> script this, at some point.  
>> 
>> Yes.  How about a Python 3 script (inside "contrib", or in "gcc/jit")
>> that would do that.  
> 
> I’m not sure we want to make a build dependency on Python 3.. 
> the reason I say ‘build’ is ...
> 
>> Then whenever a patch touches libgccjit.map we'd
>> run that script to regenerate libgccjit.exp in the source tree.  I can
>> have a go at writing it, if you think that's the best way to go.
> 
> … there are two other places in the current sources where ld map files
> are converted to Darwin (and Solaris) symbol export files [libgcc, libstdc++].
> 
> In these cases, the export file is created on-the-fly at build time by scripts
> (IIRC a mixture of awk, sh, perl).
> 
> This requires more surgery to the Make stuff and that we have a suitable
> script - but it does mean that we do not need to commit the Darwin (and
> potentially Solaris) versions to the source tree.
> 
> I’m actually happy with either solution - since we do not expect this to
> be a daily occurance, we could have a maintainter’s python3 script to
> update a committed export file or we could try the mechanism used in
> the other two places (but then the script would need to use awk/sh/perl
> to avoid new build deps).
> 
>> I take it .exp is the standard extension for these exports file in the
>> Darwin world.  If so, it's a shame (but unavoidable) that it clashes
>> with the existing uses of .exp in our source tree for our
>> expect/Tcl/DejaGnu sources.
> 
> I suspect the linker will accept other extensions, although ‘exp’ is a
> convention used elsewhere, it is unfortunate that it clashes indeed.
> - let me try an alternate (e.g. .export) and report back.
> 
>> I think the patch as-is is OK for trunk now, assuming that you've
>> tested it as above.
> 
> I’m going to hold off on this for now (but do want some solution before
> 14 branches, because there are quite a few new fails from it).

It seems that we are not going to get time to implement something better
for GCC-14; this is what I applied (I renamed the extension to .exports 
which the linker is fine with) so that it is not confused with .exp files.

I guess I’ll need to do a final pass of checking I’ve copied all the syms
before 14 branches.

thanks
Iain


[-- Attachment #2: 0001-jit-Darwin-Implement-library-exports-list.patch --]
[-- Type: application/octet-stream, Size: 11858 bytes --]

From b120e355e59142bd15d6b010461d07236288d843 Mon Sep 17 00:00:00 2001
From: Iain Sandoe <iain@sandoe.co.uk>
Date: Sat, 13 Jan 2024 17:20:47 +0000
Subject: [PATCH] jit, Darwin: Implement library exports list.

Currently, we have no exports list for libgccjit, which means that
all symbols are exported, including those from libstdc++ which is
linked statically into the lib.  This causes failures when the
shared libstdc++ is used but some c++ symbols are satisfied from
libgccjit.

This implements an export file for Darwin (which is currently
manually created by cross-checking libgccjit.map).  Ideally we'd
script this, at some point.  Update libtool current and age to
reflect the current ABI version (we are not bumping the SO name
at this stage).

This fixes a number of new failures in jit testing.

gcc/jit/ChangeLog:

	* Make-lang.in: Implement exports list, and use a shared
	libgcc.
	* libgccjit.exports: New file.

Signed-off-by: Iain Sandoe <iain@sandoe.co.uk>
---
 gcc/jit/Make-lang.in      |  47 +++++---
 gcc/jit/libgccjit.exports | 233 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 263 insertions(+), 17 deletions(-)
 create mode 100644 gcc/jit/libgccjit.exports

diff --git a/gcc/jit/Make-lang.in b/gcc/jit/Make-lang.in
index b1f0ce73e12..99b3d549d5d 100644
--- a/gcc/jit/Make-lang.in
+++ b/gcc/jit/Make-lang.in
@@ -55,7 +55,17 @@ else
 
 ifneq (,$(findstring darwin,$(host)))
 
-LIBGCCJIT_AGE = 1
+# NOTE that we are building here for the host, and so tests for target-
+# specific functionality will only work when host == target.  This causes
+# fails when building cross-compilers with different object formats (at
+# least when the respective linkers do not accept the same command line
+# options).  Fortunately, for Darwin we can safely hard-code the relevant
+# host options, since all usable linkers support them).
+
+LIBGCCJIT_CURRENT = 26
+LIBGCCJIT_REVISION = 0
+LIBGCCJIT_AGE = 26
+LIBGCCJIT_COMPAT = 0
 LIBGCCJIT_BASENAME = libgccjit
 
 LIBGCCJIT_SONAME = \
@@ -63,18 +73,18 @@ LIBGCCJIT_SONAME = \
 LIBGCCJIT_FILENAME = $(LIBGCCJIT_BASENAME).$(LIBGCCJIT_VERSION_NUM).dylib
 LIBGCCJIT_LINKER_NAME = $(LIBGCCJIT_BASENAME).dylib
 
-# Conditionalize the use of the LD_VERSION_SCRIPT_OPTION and
-# LD_SONAME_OPTION depending if configure found them, using $(if)
-# We have to define a COMMA here, otherwise the commas in the "true"
-# result are treated as separators by the $(if).
+# Darwin does not have a version script option. Exported symbols are controlled
+# by the following, and library versioning is done using libtool.
+# We have to define a COMMA here, otherwise the commas are treated as
+# separators.
 COMMA := ,
 LIBGCCJIT_VERSION_SCRIPT_OPTION = \
-	$(if $(LD_VERSION_SCRIPT_OPTION),\
-	  -Wl$(COMMA)$(LD_VERSION_SCRIPT_OPTION)$(COMMA)$(srcdir)/jit/libgccjit.map)
+  -Wl$(COMMA)-exported_symbols_list$(COMMA)$(srcdir)/jit/libgccjit.exports
 
+# For Darwin host, we need a l64 or ld64-compatible linker, that uses
+# -install_name to introduce this.
 LIBGCCJIT_SONAME_OPTION = \
-	$(if $(LD_SONAME_OPTION), \
-	     -Wl$(COMMA)$(LD_SONAME_OPTION)$(COMMA)$(LIBGCCJIT_SONAME))
+  -Wl$(COMMA)-install_name$(COMMA)$(LIBGCCJIT_SONAME)
 
 LIBGCCJIT_SONAME_SYMLINK = $(LIBGCCJIT_FILENAME)
 LIBGCCJIT_LINKER_NAME_SYMLINK = $(LIBGCCJIT_LINKER_NAME)
@@ -143,15 +153,18 @@ ifneq (,$(findstring mingw,$(target)))
 # Create import library
 LIBGCCJIT_EXTRA_OPTS = -Wl,--out-implib,$(LIBGCCJIT_IMPORT_LIB)
 else
-
 ifneq (,$(findstring darwin,$(host)))
-# TODO : Construct a Darwin-style symbol export file.
-LIBGCCJIT_EXTRA_OPTS = -Wl,-compatibility_version,$(LIBGCCJIT_VERSION_NUM) \
-	-Wl,-current_version,$(LIBGCCJIT_VERSION_NUM).$(LIBGCCJIT_MINOR_NUM).$(LIBGCCJIT_AGE) \
-	$(LIBGCCJIT_VERSION_SCRIPT_OPTION) \
-	$(LIBGCCJIT_SONAME_OPTION)
+LIBGCCJIT_VERS = $(LIBGCCJIT_CURRENT).$(LIBGCCJIT_REVISION).$(LIBGCCJIT_AGE)
+LIBGCCJIT_EXTRA_OPTS = -Wl,-current_version,$(LIBGCCJIT_VERS) \
+ -Wl,-compatibility_version,$(LIBGCCJIT_COMPAT) \
+  $(LIBGCCJIT_VERSION_SCRIPT_OPTION) $(LIBGCCJIT_SONAME_OPTION)
+# Use the default (shared) libgcc.
+JIT_LDFLAGS = $(filter-out -static-libgcc, $(LDFLAGS))
+ifeq (,$(findstring darwin8,$(host)))
+JIT_LDFLAGS += -Wl,-rpath,@loader_path
+endif
 else
-
+JIT_LDFLAGS = $(LDFLAGS)
 LIBGCCJIT_EXTRA_OPTS = $(LIBGCCJIT_VERSION_SCRIPT_OPTION) \
 	$(LIBGCCJIT_SONAME_OPTION)
 endif
@@ -170,7 +183,7 @@ $(LIBGCCJIT_FILENAME): $(jit_OBJS) \
 	$(LIBDEPS) $(srcdir)/jit/libgccjit.map \
 	$(EXTRA_GCC_OBJS_EXCLUSIVE) $(jit.prev)
 	@$(call LINK_PROGRESS,$(INDEX.jit),start)
-	+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ -shared \
+	+$(LLINKER) $(ALL_LINKERFLAGS) $(JIT_LDFLAGS) -o $@ -shared \
 	     $(jit_OBJS) libbackend.a libcommon-target.a libcommon.a \
 	     $(CPPLIB) $(LIBDECNUMBER) $(EXTRA_GCC_LIBS) $(LIBS) $(BACKENDLIBS) \
 	     $(EXTRA_GCC_OBJS_EXCLUSIVE) \
diff --git a/gcc/jit/libgccjit.exports b/gcc/jit/libgccjit.exports
new file mode 100644
index 00000000000..e32bbe2fd40
--- /dev/null
+++ b/gcc/jit/libgccjit.exports
@@ -0,0 +1,233 @@
+# Linker export list for Darwin libgccjit.dylib
+
+#   Contributed by Iain Sandoe <iain@sandoe.co.uk>.
+#
+# This file is part of GCC.
+#
+# GCC is free software you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation either version 3, or (at your option)
+# any later version.
+#
+# GCC is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GCC see the file COPYING3.  If not see
+# <http://www.gnu.org/licenses/>.  */
+
+# The initial release of the library.
+# Keep this list sorted alphabetically:
+_gcc_jit_block_add_assignment
+_gcc_jit_block_add_assignment_op
+_gcc_jit_block_add_comment
+_gcc_jit_block_add_eval
+_gcc_jit_block_as_object
+_gcc_jit_block_end_with_conditional
+_gcc_jit_block_end_with_jump
+_gcc_jit_block_end_with_return
+_gcc_jit_block_end_with_void_return
+_gcc_jit_block_get_function
+_gcc_jit_context_acquire
+_gcc_jit_context_compile
+_gcc_jit_context_compile_to_file
+_gcc_jit_context_dump_to_file
+_gcc_jit_context_dump_reproducer_to_file
+_gcc_jit_context_enable_dump
+_gcc_jit_context_get_builtin_function
+_gcc_jit_context_get_first_error
+_gcc_jit_context_get_last_error
+_gcc_jit_context_get_type
+_gcc_jit_context_get_int_type
+_gcc_jit_context_new_array_access
+_gcc_jit_context_new_array_type
+_gcc_jit_context_new_binary_op
+_gcc_jit_context_new_call
+_gcc_jit_context_new_call_through_ptr
+_gcc_jit_context_new_cast
+_gcc_jit_context_new_child_context
+_gcc_jit_context_new_comparison
+_gcc_jit_context_new_field
+_gcc_jit_context_new_function
+_gcc_jit_context_new_function_ptr_type
+_gcc_jit_context_new_global
+_gcc_jit_context_new_location
+_gcc_jit_context_new_opaque_struct
+_gcc_jit_context_new_param
+_gcc_jit_context_new_rvalue_from_double
+_gcc_jit_context_new_rvalue_from_int
+_gcc_jit_context_new_rvalue_from_long
+_gcc_jit_context_new_rvalue_from_ptr
+_gcc_jit_context_new_string_literal
+_gcc_jit_context_new_struct_type
+_gcc_jit_context_new_unary_op
+_gcc_jit_context_new_union_type
+_gcc_jit_context_null
+_gcc_jit_context_one
+_gcc_jit_context_release
+_gcc_jit_context_set_bool_option
+_gcc_jit_context_set_int_option
+_gcc_jit_context_set_logfile
+_gcc_jit_context_set_str_option
+_gcc_jit_context_zero
+_gcc_jit_field_as_object
+_gcc_jit_function_as_object
+_gcc_jit_function_dump_to_dot
+_gcc_jit_function_get_param
+_gcc_jit_function_new_block
+_gcc_jit_function_new_local
+_gcc_jit_location_as_object
+_gcc_jit_lvalue_as_object
+_gcc_jit_lvalue_as_rvalue
+_gcc_jit_lvalue_access_field
+_gcc_jit_lvalue_get_address
+_gcc_jit_object_get_context
+_gcc_jit_object_get_debug_string
+_gcc_jit_param_as_lvalue
+_gcc_jit_param_as_object
+_gcc_jit_param_as_rvalue
+_gcc_jit_result_get_code
+_gcc_jit_result_get_global
+_gcc_jit_result_release
+_gcc_jit_rvalue_access_field
+_gcc_jit_rvalue_as_object
+_gcc_jit_rvalue_dereference
+_gcc_jit_rvalue_dereference_field
+_gcc_jit_rvalue_get_type
+_gcc_jit_struct_as_type
+_gcc_jit_struct_set_fields
+_gcc_jit_type_as_object
+_gcc_jit_type_get_const
+_gcc_jit_type_get_pointer
+_gcc_jit_type_get_volatile
+
+# Add support for adding arbitrary command-line options (PR jit/66628).
+# LIBGCCJIT_ABI_1
+_gcc_jit_context_add_command_line_option
+
+# Add support for disabling the check for unreachable blocks (PR jit/66546).
+# LIBGCCJIT_ABI_2
+_gcc_jit_context_set_bool_allow_unreachable_blocks
+
+# Add support for switch statements.
+# LIBGCCJIT_ABI_3
+_gcc_jit_block_end_with_switch
+_gcc_jit_case_as_object
+_gcc_jit_context_new_case
+
+# Add timing API.
+#LIBGCCJIT_ABI_4
+_gcc_jit_context_get_timer
+_gcc_jit_context_set_timer
+_gcc_jit_timer_new
+_gcc_jit_timer_release
+_gcc_jit_timer_push
+_gcc_jit_timer_pop
+_gcc_jit_timer_print
+
+# LIBGCCJIT_ABI_5
+_gcc_jit_context_set_bool_use_external_driver
+
+# LIBGCCJIT_ABI_6
+_gcc_jit_rvalue_set_bool_require_tail_call
+
+# LIBGCCJIT_ABI_7
+_gcc_jit_type_get_aligned
+
+# LIBGCCJIT_ABI_8
+_gcc_jit_type_get_vector
+
+# LIBGCCJIT_ABI_9
+_gcc_jit_function_get_address
+
+# LIBGCCJIT_ABI_10
+_gcc_jit_context_new_rvalue_from_vector
+
+# LIBGCCJIT_ABI_11
+_gcc_jit_context_add_driver_option
+
+# LIBGCCJIT_ABI_12
+_gcc_jit_context_new_bitfield
+
+# LIBGCCJIT_ABI_13
+_gcc_jit_version_major
+_gcc_jit_version_minor
+_gcc_jit_version_patchlevel
+
+# LIBGCCJIT_ABI_14
+_gcc_jit_global_set_initializer
+
+# LIBGCCJIT_ABI_15
+_gcc_jit_block_add_extended_asm
+_gcc_jit_block_end_with_extended_asm_goto
+_gcc_jit_extended_asm_as_object
+_gcc_jit_extended_asm_set_volatile_flag
+_gcc_jit_extended_asm_set_inline_flag
+_gcc_jit_extended_asm_add_output_operand
+_gcc_jit_extended_asm_add_input_operand
+_gcc_jit_extended_asm_add_clobber
+_gcc_jit_context_add_top_level_asm
+
+# LIBGCCJIT_ABI_16
+_gcc_jit_function_get_return_type
+_gcc_jit_function_get_param_count
+_gcc_jit_function_type_get_return_type
+_gcc_jit_function_type_get_param_count
+_gcc_jit_function_type_get_param_type
+_gcc_jit_type_unqualified
+_gcc_jit_type_dyncast_array
+_gcc_jit_type_is_bool
+_gcc_jit_type_dyncast_function_ptr_type
+_gcc_jit_type_is_integral
+_gcc_jit_type_is_pointer
+_gcc_jit_type_dyncast_vector
+_gcc_jit_vector_type_get_element_type
+_gcc_jit_vector_type_get_num_units
+_gcc_jit_struct_get_field
+_gcc_jit_type_is_struct
+_gcc_jit_struct_get_field_count
+
+# LIBGCCJIT_ABI_17
+_gcc_jit_lvalue_set_tls_model
+
+# LIBGCCJIT_ABI_18
+_gcc_jit_lvalue_set_link_section
+
+#LIBGCCJIT_ABI_19
+_gcc_jit_context_new_array_constructor
+_gcc_jit_context_new_struct_constructor
+_gcc_jit_context_new_union_constructor
+_gcc_jit_global_set_initializer_rvalue
+
+# LIBGCCJIT_ABI_20
+_gcc_jit_compatible_types
+_gcc_jit_type_get_size
+
+# LIBGCCJIT_ABI_21
+_gcc_jit_context_new_bitcast
+
+# LIBGCCJIT_ABI_22
+_gcc_jit_lvalue_set_register_name
+
+# LIBGCCJIT_ABI_23
+_gcc_jit_context_set_bool_print_errors_to_stderr
+
+# LIBGCCJIT_ABI_24
+_gcc_jit_lvalue_set_alignment
+_gcc_jit_lvalue_get_alignment
+
+# LIBGCCJIT_ABI_25
+_gcc_jit_type_get_restrict
+
+# LIBGCCJIT_ABI_26 
+_gcc_jit_function_add_attribute
+_gcc_jit_function_add_string_attribute
+_gcc_jit_lvalue_add_string_attribute
+_gcc_jit_function_add_integer_array_attribute
+
+# LIBGCCJIT_ABI_27
+_gcc_jit_context_new_sizeof
+
+
-- 
2.39.2 (Apple Git-143)


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]