Link time optimization(LTO) segmentation fault issue

Link time optimization(LTO) segmentation fault issue

[Jason Yang]

[-- Attachment #1.1.1: Type: text/plain, Size: 2266 bytes --]

Dear GCC developers:

If this email address is not correct for questions, please help forward 
this email. Sincerely sorry that I just find this email on gcc.gnu.org 
website and hope my question can be answered.

I am currently working on building cpython with gcc. The problem is that 
after turning on lto option, such as the following

|if [[ ${_OPTIMIZED} == yes ]]; then 
_extra_opts+=(--enable-optimizations) _extra_opts+=(--with-lto) 
_MAKE_TARGET=profile-opt # To speed up build times during testing (1): # 
_PROFILE_TASK="./python -m test.regrtest --pgo test_builtin" if [[ ${CC} 
=~ .*gcc.* && ! ${c_compiler} =~ .*toolchain.* ]]; then 
LTO_CFLAGS+=(-fuse-linker-plugin) LTO_CFLAGS+=(-ffat-lto-objects) # 
-flto must come after -flto-partition due to the replacement code 
LTO_CFLAGS+=(-flto-partition=none) LTO_CFLAGS+=(-flto) else # TODO :: 
Check if -flto=thin gives better results. It is about faster # 
compilation rather than faster execution so probably not: # 
http://clang.llvm.org/docs/ThinLTO.html # 
http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html 
LTO_CFLAGS+=(-flto) fi export CFLAGS="${CFLAGS} ${LTO_CFLAGS[@]}" else 
_MAKE_TARGET= fi |

When I enabled the LTO flags and built cpython with optimization (enable 
–with-lto, -fuse-linker-plugin, -ffat-lto-objets, -flto-partition=none, 
-flto), the built python3 will encounter segmentation fault when I 
trying to run it on another host (not using Docker ,but using some 
LD_PRELOAD hacks created sandbox, which means host libc will be involved 
during the initialization), when I use Docker (isolated namespaces) with 
LTO compiled python3, there is no such segmentation fault issue.

My guess is that LTO has requirements for gcc link/compiler environments?

Though I’ve read this url 
https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html 
<https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html>

but still could not understand whether the LTO is link/compiler 
specific? and how about its portability(compiled in one environment, and 
can be used in another environment with different link/compiler versions? )

Thanks so much for the help! And looking forward to your responses!

Best regards,

Jason

​

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3191 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Link time optimization(LTO) segmentation fault issue

[Xi Ruoyao]

On Tue, 2021-12-28 at 00:42 +0900, Jason Yang via Gcc-help wrote:
> Dear GCC developers:
> 
> If this email address is not correct for questions, please help forward 
> this email. Sincerely sorry that I just find this email on gcc.gnu.org
> website and hope my question can be answered.
> 
> I am currently working on building cpython with gcc. The problem is that 
> after turning on lto option, such as the following

I've been using --with-lto building Python 3 for a while.  No problems
observed.

/* snip */

> When I enabled the LTO flags and built cpython with optimization (enable 
> –with-lto, -fuse-linker-plugin, -ffat-lto-objets, -flto-partition=none, 
> -flto), the built python3 will encounter segmentation fault when I 
> trying to run it on another host (not using Docker ,but using some 
> LD_PRELOAD hacks created sandbox, which means host libc will be involved 
> during the initialization), when I use Docker (isolated namespaces) with 
> LTO compiled python3, there is no such segmentation fault issue.


> My guess is that LTO has requirements for gcc link/compiler environments?

Yes.  But it does not have requirements for runtime environments, so it
should be irrelevent to the segfault.

> Though I’ve read this url 
> https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html 
> <https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html>
> 
> but still could not understand whether the LTO is link/compiler 
> specific? and how about its portability(compiled in one environment, and 
> can be used in another environment with different link/compiler versions? )

It's not portable.  For example, you can't link GCC LTO objects with
LLVM linker plugin, or vice versa.  And you can't use GCC linker plugin
to link the LTO objects compiled by an old GCC release if
LTO_major_version has changed (for example, see
https://gcc.gnu.org/pipermail/gcc-patches/2021-April/568286.html).
-- 
Xi Ruoyao <xry111@mengyan1223.wang>
School of Aerospace Science and Technology, Xidian University