From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from zmcc-2-mx.zmailcloud.com (zmcc-2-mx.zmailcloud.com [52.37.197.7]) by sourceware.org (Postfix) with ESMTPS id CE53C3857C52 for ; Thu, 24 Sep 2020 09:19:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org CE53C3857C52 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=symas.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=hyc@symas.com Received: from zmcc-2.zmailcloud.com (zmcc-2-mta-1.zmailcloud.com [146.148.52.56]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by zmcc-2-mx.zmailcloud.com (Postfix) with ESMTPS id 0754F40654; Thu, 24 Sep 2020 04:19:11 -0500 (CDT) Received: from zmcc-2.zmailcloud.com (localhost [127.0.0.1]) by zmcc-2-mta-1.zmailcloud.com (Postfix) with ESMTPS id 84D08CE9E4; Thu, 24 Sep 2020 04:19:10 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by zmcc-2-mta-1.zmailcloud.com (Postfix) with ESMTP id 6B370CE9E0; Thu, 24 Sep 2020 04:19:10 -0500 (CDT) DKIM-Filter: OpenDKIM Filter v2.10.3 zmcc-2-mta-1.zmailcloud.com 6B370CE9E0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=symas.com; s=37C7994C-28CA-11EA-A30F-68F90BB9D764; t=1600939150; bh=wsKTyZppAVRHNkhvZPPtDZGBZYtUwuUbyqDFCqXpmLI=; h=To:From:Message-ID:Date:MIME-Version; b=VZBXDcTWzLC2qKjd4zXAykLGJyTtsl36DUqw/+sgqLcga15ulWyqjIXnkGg97Brsq 6mMou9kxQ5e7WcI8wTG/MNJHKZNY63WprE5uzlx/eilH8wtnFG58GzHUOhe9JOo3f0 hDfWYj2i32DaHz3SkF4Fuhkb5/0KXd/o1tzeu2dl3pH9ioTwVPYN619iJNHXcRgl0V kFrIEwWq0TuPwwsS6swnz74T11J2nSEklO9zmdH7KIo8JJwEN5hZTe82XDkfjkl4rx bB22gwFsS9ltBIhdKC2RTQzC/HN8u31cQ0Y0l7yjoPUl6/M96SIQvsjZEXwVaTqNd1 QqTMYdLz1Af5w== Received: from zmcc-2.zmailcloud.com ([127.0.0.1]) by localhost (zmcc-2-mta-1.zmailcloud.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id P9M6snZe09ql; Thu, 24 Sep 2020 04:19:10 -0500 (CDT) Received: from [192.168.1.155] (unknown [84.203.28.168]) by zmcc-2-mta-1.zmailcloud.com (Postfix) with ESMTPSA id 8695EC21AF; Thu, 24 Sep 2020 04:19:09 -0500 (CDT) Subject: Re: [PATCH] dependency list for static libraries To: Fangrui Song Cc: Nick Clifton , binutils@sourceware.org References: <64fe82bd-9c00-b232-98d2-f46182fb16ba@symas.com> <9889c54b-4dd3-2275-6621-c2391cfd268d@redhat.com> <31f9062e-175d-06e9-695a-797c7ee11420@symas.com> <58620dc1-3bb9-aaae-b476-ebb613ecb627@redhat.com> <20200924052111.3i5codebro4qqxia@gmail.com> From: Howard Chu Message-ID: <37636222-1f01-068f-41e0-a3f3f660506b@symas.com> Date: Thu, 24 Sep 2020 10:19:07 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0 SeaMonkey/2.53.3 MIME-Version: 1.0 In-Reply-To: <20200924052111.3i5codebro4qqxia@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_RED autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2020 09:19:13 -0000 Fangrui Song wrote: > Thanks to Nick for mentioning me :) >=20 > On 2020-09-23, Howard Chu wrote: >> Nick Clifton wrote: >> I can send an updated patch after the major issue is addressed. >>> The major issue is that I would really like for this extension to be = supported >>> by the LLVM community as well.=C2=A0 It would be a shame to add it to= the binutils >>> only to have a different solution implemented there.=C2=A0 I might ha= ve misread the >>> emails but I believe that Fangrui still has some misgivings about thi= s approach. >>> Is that correct ?=C2=A0 If so, then I would very much like to see the= m resolved >>> before we commit the patch. >> >> It still seems pretty obvious that handling dependencies once at the l= ibrary >> level is more efficient than sticking a bunch of free-form notes in ev= ery single >> object file. And it should also be obvious that choice of library is a= buildtime >> decision, not a decision made solely at the time the source code is wr= itten. >> (And for shared libraries, it's a runtime decision - even further remo= ved from >> the time of code being written.) >=20 > From my experience (I have investigated hundreds of link issues), > archive link errors are usually caused by the following two problems: >=20 > * There is a backward reference between two archives. This is related t= o a.out/ELF style linker behaviors > (https://sourceware.org/pipermail/binutils/2020-September/113194.html )= . > =C2=A0 + A dependency exists but is incorrectly omitted. The linker act= ually captures a layering problem. > =C2=A0 + A dependency is intentionally omitted. If the linker allows ba= ckward references, this will not be a problem. >=20 > =C2=A0 I have recently thought a lot on the topic and written http://ll= d.llvm.org/ELF/warn_backrefs.html That is not a problem I encounter frequently, and is not in the scope of = what I'm concerned with. > * Some archives are incorrectly omitted: the executable uses A, but A's > =C2=A0 dependency B is not on the command line.=C2=A0 Now I hope Howard= can clarify on this > =C2=A0 topic:) What do you want to achieve with the dependency recorded= in the archive? > =C2=A0 I assume that you want the linker to smartly link B. Yes. > This is indeed the #pragma > =C2=A0 comment(lib, ...) and .deplibs feature clang/LLD support. Perhaps, but I remain skeptical. > =C2=A0 The proposal does not mention the ld part, which seems important= . How > =C2=A0 does ld handle __.LIBDEP ? ld effectively inserts the flags from __.LIBDEP into its command line imm= ediately following its archive. > =C2=A0 + The executable does not references B. I am still unsure why yo= u want to move the dependency information from the > =C2=A0=C2=A0=C2=A0 object file to the archive. The most common case is we reference a library we know about, and it has = dependencies we don't know about. An example I currently have in mind is libzmq, which= may or may not be built against libsodium, and may or may not have other library dep= endencies as well. The "traditional" solution to this for the past couple decades h= as been to use pkg-config or libtool .la files to propagate this information but tho= se are both ugly hacks that require excessive tooling to support. The information cle= arly belongs to the archive file itself. > =C2=A0=C2=A0=C2=A0 - First, I want to mention that in some cases archiv= es are not needed. Irrelevant. I'm talking about software that is shipped in library form. > =C2=A0=C2=A0=C2=A0 - Moving the dependency to the archive can actually = increases the work for > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the linker.=C2=A0 Say the two members of= A are a0.o and a1.o. If a0.o depends on B > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 but a0.o is not selected, then B does no= t need to be linked. Having the > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dependency in the archive will cause B t= o be linked. > =C2=A0=C2=A0=C2=A0 - How do you ensure the dependency information does = not become stale? i.e. > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 How do you maintain the dependency infor= mation when members are added to/removed from > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the archive? The information is more dif= ficult to maintain than the index, which only contains > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 the definitions. You may argue that the = archive is finalized after it is created - then > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 we go back to square one, --start-lib ma= y be more suitable. How do you maintain the dependency information of a shared library? This = is a problem developers already deal with, not a new problem. >=20 > --- >=20 > Last, >=20 >> =C2=A0fprintf (s, _("=C2=A0 [L LIBDEPS]=C2=A0 - specify dependencies o= f this library\n")); >=20 > In llvm-ar, 'L' is an extension used with 'q' to add all members of an = archive to the current archive. > It is similar to ADDLIB (https://sourceware.org/binutils/docs/binutils/= ar-scripts.html ) > Hope the two tools don't have conflicting operations. We could just use 'l' instead, which has historically been a no-op. The s= pecific choice of letter option here is not significant. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/