From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 84728 invoked by alias); 11 Jun 2019 18:39:44 -0000 Mailing-List: contact libc-help-help@sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Post: List-Help: , Sender: libc-help-owner@sourceware.org Received: (qmail 84633 invoked by uid 89); 11 Jun 2019 18:39:43 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-10.8 required=5.0 tests=AWL,BAYES_00,GIT_PATCH_2,RCVD_IN_DNSWL_NONE,SPF_PASS autolearn=ham version=3.3.1 spammy= X-HELO: mail-vs1-f65.google.com Received: from mail-vs1-f65.google.com (HELO mail-vs1-f65.google.com) (209.85.217.65) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 11 Jun 2019 18:39:40 +0000 Received: by mail-vs1-f65.google.com with SMTP id 190so5004599vsf.9 for ; Tue, 11 Jun 2019 11:39:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:references:openpgp:autocrypt:subject:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=q/wSRSeRElQhurU5nQ4ROS1t+3nuFay5jdkOoNpYBd4=; b=yzx4NTinJ0ZtAFZbVMi6LHGoBKMgzy22eBLfyhDmcu1s0aMuefn7OpfMVKXIquxlaD zoWccaGZPL8h13pg3s3mEKIxpXpyKczhIMLBRrdeyBfT9BLp0Twgl+hVWS+DMs/4x2i7 J2laOHOJU3hQayrbOQ8NFchTlloAE7h7ta8U4J/96Byz72O+LjzTVjRoDT8vdcr65nN7 BPKtFQgfM4OW6S1t0hjnUSfWKpnfL2qOLMdog+cYs/0ZP2NPsfnX+TiO1BO4FG/ZoRLC WwCKK4QZk3pkbHP1+FrdKsuHcKhNkLH0sznByQ4xB8rreNEsfBwTQVhAg1cuij5jJGOz 8Gpg== Return-Path: Received: from [192.168.1.132] ([177.194.125.152]) by smtp.googlemail.com with ESMTPSA id b18sm3477203uae.14.2019.06.11.11.39.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Jun 2019 11:39:37 -0700 (PDT) From: Adhemerval Zanella To: Nat! , libc-help@sourceware.org References: <87blzypg5j.fsf@mid.deneb.enyo.de> <0a7c2435-43f8-8dfb-83ab-22ceff7ca51c@mulle-kybernetik.com> <9497a5c2-0dc8-18fe-6120-deb551f7ddd8@mulle-kybernetik.com> Openpgp: preference=signencrypt Subject: Re: Problem with atexit and _dl_fini Message-ID: <31060c89-404f-e8e1-6c18-d75e0b63f6ad@linaro.org> Date: Tue, 11 Jun 2019 18:39:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-IsSubscribed: yes X-SW-Source: 2019-06/txt/msg00009.txt.bz2 On 10/06/2019 17:27, Adhemerval Zanella wrote: > > > On 10/06/2019 10:07, Nat! wrote: >> >> On 10.06.19 13:48, Adhemerval Zanella wrote: >>> >>> On 09/06/2019 17:59, Nat! wrote: >>>> Another datapoint to support my claim that _dl-fini breaks atexit. This time its very easy to reproduce ;) >>>> >>>> Here 's the README.md from the Github Repo https://github.com/mulle-nat/atexit-breakage-linux >>>> >>>> >>>> ``` >>>> >>>> # Shows another breakage involving `atexit` on linux >>>> >>>> Here the `atexit` callback is invoked mistakenly multiple times. >>> This 'example' does not really show the issue because ldd script issues >>> the loader multiple times, see below. You can check exactly what ldd is >>> doing by calling with sh -x. >> >> I agree it doesn't show the same issue, but it shows that something else is going very wrong. :) Or are you happy, that atexit is called multiple times ? Who's calling exit here anyway ? Check out the debugger output too (see updated README.md) > > The ldd is not a program, but rather a shell script that issues the target > binary along with system loader multiple times. What you are seeing is not > atexit called multiple times, but rather how the script is called. > > When you set LD_PRELOAD *before* issuing ldd you will make the shell binary > to also pre-load the library. I instrumented the binary to also print the > output command line from the issue binary (get either by program_invocation_name > or /proc/self/cmdline): > > $ LD_PRELOAD=./libld-preload.so ./ldd ./main > /bin/bash: load > /bin/bash: unload > /bin/bash: unload > /bin/bash: unload > linux-vdso.so.1 (0x00007ffd445ef000) > ./libld-preload.so (0x00007fa866ac5000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa8664b5000) > /lib64/ld-linux-x86-64.so.2 (0x00007fa8668a6000) > /bin/bash: unload > /bin/bash: unload > > The program is not load since although ldd does call the loader, it calls > in a trace mode that does not actually load any shared library. The first > 'load' is issued by library when bash is first executed and later multiple > 'unload' is due bash forks and then exits multiple times. > >> >> >>> >>> I will try to use your instruction to run on docker to see what exactly >>> is happening in your environment. >> >> That's not necessary anymore. I managed to make it reproducible in a much simpler form just now. >> >> The ld-so-breakage project is basically a recreation of the original "docker" scenario written from scratch. I try to explain in the README , what is going on. But if there are questions hit me up (maybe as an issue ?) : >> >>     https://github.com/mulle-nat/ld-so-breakage > > Thanks, it is way more useful. I now I understand what is happening and > IMHO this behaviour is a required because on glibc we set that atexit/on_exit > handlers are ran when deregister a library (as for dlclose). > > Using the example in your testcase: > > --- > USE_A=YES ./build/main_adbc > -- install atexit_b > -- install atexit_a > -- run atexit_a > -- run atexit_b > --- > > The behaviour of atexit handlers being called in wrong order is they are > being registered with '__cxa_atexit' which in turn sets its internal type > as 'ef_cxa'. Since _dl_init is registered last (after all shared library > loading and constructors calls), it will call _dl_fini which in turn will > call '__cxa_finalize' (through __do_global_dtors_aux generated by compiler). > > The '__cxa_finalize' will then all 'ef_cxa' function for the module passed > by __do_global_dtors_aux and set the function as 'ef_free'. It will then > prevent '__run_exit_handlers' to run the handlers more than once. > > So the question you might ask is why not just to use 'ef_at' for atexit > handlers, make them no to run on __cxa_finalize and thus make your example > run as you expect? The issue is glibc does not know whether your library > would be dlopened or not. > > If you set an atfork handler by a constructor that references to a function > inside the shared library and if do *not* set to *not* be ran later you might, > a case of dlopen -> constructor -> dlclose -> exit will try to execute and > invalid mapping. This is exactly what dlfcn/bug-atexit{1,2}.c. > > So the question is why exactly glibc defined that atexit should be called > by dlclose. I understand that __cxa_finalize / destructor make sense to > make it possible the shared library to free allocated resources, but I > can't really get why there a need to extend it to 'atexit' as well. It seems that this requirement seems to come from LSB, although I am not sure which one came first (the specification or the implementation). It also states that __cxa_atexit should register a function to be called by exit or when a shared library is unloaded. And __cxa_finalize requires to call atexit registers functions as well. It also states __cxa_finalize should be called on dlclose. I think it might due the fact old gcc version uses atexit to register C++ destructors for local static and global objects. However it seems to be enabled as default for GLIBC (since it support __cxa_atexit since initial versions). So I think there is no impeding reason to make atexit not be called from __cxa_finalize, although I am not sure how we would handle the LSB deviation. I will write down a libc-alpha to check what other developer think. [1] http://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic.pdf