From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 834163858C53 for ; Wed, 13 Apr 2022 18:00:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 834163858C53 Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-sTIzhlniNOmwwaOvxa34LA-1; Wed, 13 Apr 2022 14:00:47 -0400 X-MC-Unique: sTIzhlniNOmwwaOvxa34LA-1 Received: by mail-pj1-f71.google.com with SMTP id u4-20020a17090a5e4400b001cba059a4fbso1674059pji.7 for ; Wed, 13 Apr 2022 11:00:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=5cwk9oBLSeasDotHPqm8O8F6Oulb2jKOlJ2LvlU+SIQ=; b=5cael6Z80Da8rNF0M01O/PZnrjjFiBn4WXTFAwXXVzRqoNVUTFajJKHDEeD874CYts LSCWQqCDFRjEzRwrubcpjxFTNZXNhkfuYtJFfbi9l5JGI2NQcinvvymP2nl/b46LdP1m bocVo5bcsPtQCar160JNO10QYgV1Yt/bnJ/759bLlCwepnj8RIKAehtrO/wLBmGWLM1A SQEP8dAsODFshK+OXmSekTdNl66jhKFnBgu+lLp5ftKROrQwVdFM0mY42z/zqjdVRS1l LXTCoYy3kzJE/kZLWrqbKPivGsmu9BnqSJwz7QbUXpCI05Ul8R45mD8k2zbAaeL3lfcE nC8w== X-Gm-Message-State: AOAM530Gjzuy+4bapmh4ojTAp12tMl8bi7+pNR6zqJR6lvEJFqBtF7xJ tfDrqmy+U8TZsHnIOef9L9JdirhuU24kfc+L1n+zz/sHsaRiWYIFIFYP5+WaffNv4DZwwY3E8CY CcMKAYgHuVscGffkPfpk2 X-Received: by 2002:a17:90b:4f4e:b0:1cd:49fa:911d with SMTP id pj14-20020a17090b4f4e00b001cd49fa911dmr60215pjb.26.1649872845651; Wed, 13 Apr 2022 11:00:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw7R12/riYoHnOt8dr7+qmSywCWUhuBW2ZuIwWtTtvtXuCWXWx5yhkl6TE+5uJdV//kahE20w== X-Received: by 2002:a17:90b:4f4e:b0:1cd:49fa:911d with SMTP id pj14-20020a17090b4f4e00b001cd49fa911dmr60160pjb.26.1649872845021; Wed, 13 Apr 2022 11:00:45 -0700 (PDT) Received: from smtpclient.apple ([47.208.199.57]) by smtp.gmail.com with ESMTPSA id c138-20020a624e90000000b005081f92826dsm1189615pfb.99.2022.04.13.11.00.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Apr 2022 11:00:44 -0700 (PDT) From: Ben Woodard Message-Id: <88333826-883C-4BB5-9EDF-8C65E2AF1798@redhat.com> Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\)) Subject: Re: 'src/abg-dwarf-reader.cc:compare_dies_string_attribute_value' optimization Date: Wed, 13 Apr 2022 11:00:41 -0700 In-Reply-To: <04665D9C-C7CB-419C-9DBB-C68D03713D77@redhat.com> Cc: Ben Woodard via Libabigail To: Dodji Seketeli References: <87wnijv616.fsf@dirichlet.schwinge.homeip.net> <87r18oynpn.fsf@euler.schwinge.homeip.net> <877d7vhcmv.fsf@seketeli.org> <8735iigtu1.fsf@seketeli.org> <04665D9C-C7CB-419C-9DBB-C68D03713D77@redhat.com> X-Mailer: Apple Mail (2.3693.60.0.1.1) X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, HTML_MESSAGE, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: libabigail@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list of the Libabigail project List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Apr 2022 18:00:51 -0000 Just a FYI these all completed successfully with no issues. It just took a = long time. I don=E2=80=99t know exactly how long they took but the last tim= e I looked they were about 400min of runtime each.=20 > On Apr 12, 2022, at 7:19 PM, Ben Woodard wrote: >=20 > There also appears to be some sort of performance regression. I know that= these are your problem children but=20 > /lib64/libmozjs-68.so.0.0.0=20 > /lib64/libwebkit2gtk-4.0.so.37.56.4=20 > /lib64/libgs.so.9.55=20 > /lib64/libperl.so.5.34.1=20 > have been running solidly for more than 3 hours. I=E2=80=99ll let them co= ntinue all night and see if they complete or if they are in some sort of in= finite loop. We have been doing lots of work here to really carefully understand as many= ABI issues as we can and I know that this release is taking much longer th= an you anticipated with all the fall out from this bug and the typedef coll= apsing. However, we have been accumulating an ever increasing list of ABI r= elevant artifacts that libabigail is not detecting. It would be really nice= if these could also be added to the 2.1 release: These two should be fairly easy to add: 1) abicompat - Weak symbol replacement by a library. If a library replaces = a weak symbol in the application with a strong symbol in the library, the s= ymbols must have the same ABI signature. This is a very localized piece of = code that just needs to be repeated. Right now abicompat only looks for und= efined symbols in the ELF. It just needs a new ELF helper function which fi= nds the weak symbols provided by the application and adds those to the list= of functions and variables that need to be inspected in the library. 2) abicompat - Underlinked symbols in a library. Basically the same thing b= ut instead of weak symbols or undefined symbols consider symbols who are NO= TYPE. Other things that we have uncovered which would be nice to have but are pro= bably harder to implement: 1) RTTI - any type defined in the LSDA for exceptions must be included in t= he ABI analysis. If a type is thrown from a library and caught by a the app= lication, then their types must match. These types do not necessarily appea= r in the list of undefined symbols in the application and so they are not c= onsidered when comparing library compatibility. Therefore, libabigail needs= to go through the LSDA to find the types in the application to ensure that= they are added to to the types being compared in the library. 2) Inline functions - Obviously we can=E2=80=99t compare that inline functi= ons are semantically the same but we can at least make sure that the functi= ons and the types involved with them are the same. Once again, these are no= t considered by abicompat because they do not appear as UNDEF symbols in th= e ELF. To be able to pick up these inline functions, you need to go through= the DWARF callsites of both the application and the library and make sure = that the function prototypes and their types which exist in both the librar= y and the applicatoin are the same. 3) The type of TLS Thread Local Storage that a library uses must match the = application. This is the difference between TLS and TLSDESC, This needs to = be considered an ABI artifact. It is a little bit weird, you can probably c= onsider it at the library level the same way that you look at SONAMEs but t= he type of TLS is defined at a symbol level. I have never seen someone mix = objects with different kinds of TLS in one library. So I don=E2=80=99t know= how you want to handle this. Maybe put a flag has_tls and then enumerate t= he flavor in the library and if the library and the application don=E2=80= =99t have the same flavor of TLS or if two libraries have different flavors= of TLS announce it. Or maybe store it in the variable where it lives in th= e ELF. 4) Library dependencies - When examining libraries for ABI compatibility it= isn=E2=80=99t just the symbols used by the application itself that must be= inspected, all of the other libraries that this application depends on mus= t be considered.=20 5) bidirectional compatibility - It isn=E2=80=99t just the undefined, weak,= underlinked symbols in the application which must be compared in the libra= ries, it is also the symbols that are undefined in the library which are de= fined in the application which must be considered. This is all just deeper processing of the ELF. I have sourceware bugs open = on most of these and reproducers for several of them. I think it is very im= portant that self-compares work and the fundamentals of type comparison are= correct but we also need to not give people false notions about compatibil= ity by ignoring things which we already know about which have ABI implicati= ons. -ben