From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from zmcc-2-mx.zmailcloud.com (zmcc-2-mx.zmailcloud.com [52.37.197.7]) by sourceware.org (Postfix) with ESMTPS id 462DA385782D for ; Tue, 1 Dec 2020 00:08:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 462DA385782D Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=symas.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=hyc@symas.com Received: from zmcc-2.zmailcloud.com (zmcc-2-mta-1.zmailcloud.com [146.148.52.56]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by zmcc-2-mx.zmailcloud.com (Postfix) with ESMTPS id 6C1444050C; Mon, 30 Nov 2020 18:08:15 -0600 (CST) Received: from zmcc-2.zmailcloud.com (localhost [127.0.0.1]) by zmcc-2-mta-1.zmailcloud.com (Postfix) with ESMTPS id D87F4CE817; Mon, 30 Nov 2020 18:08:14 -0600 (CST) Received: from localhost (localhost [127.0.0.1]) by zmcc-2-mta-1.zmailcloud.com (Postfix) with ESMTP id C0E70CE946; Mon, 30 Nov 2020 18:08:14 -0600 (CST) DKIM-Filter: OpenDKIM Filter v2.10.3 zmcc-2-mta-1.zmailcloud.com C0E70CE946 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=symas.com; s=37C7994C-28CA-11EA-A30F-68F90BB9D764; t=1606781294; bh=5xEFr7Qzl/i9X4lSAHKRVyeLjEnE2tFlDo0xz0I4wKY=; h=To:From:Message-ID:Date:MIME-Version; b=CzxERIdri+WNENBWMbpQPQhLJ0KFxPaeBgrTgUMqK4GtyOCLkD8uGsrYBoJEyrJmD WI+LeP8mZgJnHk84ufE5MaZR8NzfvN0sgGGs15/7LteITHc06haJ9e8+BTEDEx6Qkk qVSRpB2sEzWRiL/Zp9ogKW8kNFVfrOIJU3+PhVOeDSjRpxjNGF/Tbs+eDnOR1oK3fP 9CZlwrHPdb3ZkvCPxzR4RDe2AvQHlv3b8N6+U+lPYJn04/i2G7Qpmu2y8nut20twdk WhA8DgzbyE9xDKWPXwnORzQRxSuAMFbm2jOhRs+vvyTlDLXnwqAyqdlctbfS60Sjc7 /EXWPqcA23OUQ== Received: from zmcc-2.zmailcloud.com ([127.0.0.1]) by localhost (zmcc-2-mta-1.zmailcloud.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id oGvoBMYIk7Os; Mon, 30 Nov 2020 18:08:14 -0600 (CST) Received: from [192.168.1.155] (unknown [84.203.30.144]) by zmcc-2-mta-1.zmailcloud.com (Postfix) with ESMTPSA id E28C4CE863; Mon, 30 Nov 2020 18:08:13 -0600 (CST) Subject: Re: [PATCH] dependency list for static libraries To: Nick Clifton Cc: binutils@sourceware.org References: <53b8973b-40a4-2550-3307-66d7f13707d5@symas.com> <64fe82bd-9c00-b232-98d2-f46182fb16ba@symas.com> <9889c54b-4dd3-2275-6621-c2391cfd268d@redhat.com> <31f9062e-175d-06e9-695a-797c7ee11420@symas.com> <58620dc1-3bb9-aaae-b476-ebb613ecb627@redhat.com> <1aaf4429-074c-7912-e6e6-4d2f82d6ef10@redhat.com> <7b118db2-9502-c45e-9f47-6b6f6a5f9646@symas.com> <7dba4046-3d50-c7e6-915d-08bf590bf057@redhat.com> <7b124cd4-529b-0069-3044-9d733a4e4e5c@symas.com> <20ad87a3-283d-7e49-9c2e-ed13001f6c87@symas.com> <70139665-0b8c-63dc-3a9d-b24a25bd8122@redhat.com> From: Howard Chu Message-ID: <2475d2a6-c63e-6dac-0f56-fa9acb6f20de@symas.com> Date: Tue, 1 Dec 2020 00:08:11 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0 SeaMonkey/2.53.5.1 MIME-Version: 1.0 In-Reply-To: <70139665-0b8c-63dc-3a9d-b24a25bd8122@redhat.com> Content-Type: multipart/mixed; boundary="------------E614D1EF9C78B58D16B3E94D" X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_RED autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2020 00:08:18 -0000 This is a multi-part message in MIME format. --------------E614D1EF9C78B58D16B3E94D Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Nick Clifton wrote: > Hi Howard, >=20 >>> This linker plugin works with both ld and gold. >=20 >> Ping: anything else needed for this, docs, etc.? >=20 > Sorry, sorry - my mind has been a mile away. >=20 > So - the code in the patch looks good.=C2=A0 I am running regression te= sts > now but I am not expecting to encounter any problems.=C2=A0 There are s= ome > things missing however, the first of which is documentation as you > guessed: I'm still working on writing up some tests, but the rest is here to get a head start on reviewing. >=20 > =C2=A0 * A entry in ld/NEWS describing the new plugin. Done. > =C2=A0 * Documentation in ld/ld.texi describing how to use the plugin, > =C2=A0=C2=A0=C2=A0 preferably with examples. I took a stab at this, open to suggestions. > =C2=A0 * One or more new tests in linker's testsuite to check the > =C2=A0=C2=A0=C2=A0 functionality of the plugin.=C2=A0 Obviously these w= ould have to be > =C2=A0=C2=A0=C2=A0 native only tests, but there are places in the tests= uite > =C2=A0=C2=A0=C2=A0 where this is checked.=C2=A0 (Eg ld/testsuite/ld-boo= tstrap/bootstrap.exp) Working on this. > One other thing - is the plugin installed into the correct location > if "make install" is run ?=C2=A0 (Ie:=C2=A0 ${libdir}/bfd-plugins).=C2=A0= I have not > actually checked this yet, so maybe it works, but I would have guessed > that some extra work would need to be done on ld/Makefile.am to manage > the installation. I've tweaked the Makefile.am to install into $(libdir)/bfd-plugins. --=20 -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --------------E614D1EF9C78B58D16B3E94D Content-Type: text/x-patch; name="0001-Add-libdep-linker-plugin.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0001-Add-libdep-linker-plugin.patch" >From 5e5dccb81d36d6198093d0991b669c0e9dc5a626 Mon Sep 17 00:00:00 2001 From: Howard Chu Date: Fri, 13 Nov 2020 14:07:56 +0000 Subject: [PATCH] Add libdep linker plugin --- ld/Makefile.am | 5 + ld/NEWS | 3 + ld/ld.texi | 53 +++++++ ld/libdep_plugin.c | 366 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 427 insertions(+) create mode 100644 ld/libdep_plugin.c diff --git a/ld/Makefile.am b/ld/Makefile.am index c9f85e5292..2f59ee6169 100644 --- a/ld/Makefile.am +++ b/ld/Makefile.am @@ -998,6 +998,11 @@ libldtestplug4_la_SOURCES = testplug4.c libldtestplug4_la_CFLAGS= -g -O2 libldtestplug4_la_LDFLAGS = -no-undefined -rpath /nowhere +bfdplugindir = $(libdir)/bfd-plugins +bfdplugin_LTLIBRARIES = libdep.la +libdep_la_SOURCES = libdep_plugin.c +libdep_la_LDFLAGS = -no-undefined -rpath /nowhere + # DOCUMENTATION TARGETS # Manual configuration file; not usually attached to normal configuration, # because almost all configs use "gen" version of manual. diff --git a/ld/NEWS b/ld/NEWS index 607030bddd..1c3f9c156e 100644 --- a/ld/NEWS +++ b/ld/NEWS @@ -1,5 +1,8 @@ -*- text -*- +* Add libdep plugin, for linking dependencies of static libraries that + were recorded by ar in the __.LIBDEP archive member. + * Add --error-handling-script= command line option to allow a helper script to be invoked when an undefined symbol or a missing library is encountered. This option can be suppressed via the configure time diff --git a/ld/ld.texi b/ld/ld.texi index 8e3c7da145..51c51a3ec1 100644 --- a/ld/ld.texi +++ b/ld/ld.texi @@ -126,6 +126,7 @@ in the section entitled ``GNU Free Documentation License''. * Overview:: Overview * Invocation:: Invocation * Scripts:: Linker Scripts +* Plugins:: Linker Plugins @ifset GENERIC * Machine Dependent:: Machine Dependent Features @end ifset @@ -6954,6 +6955,58 @@ Any input files read because of an implicit linker script will be read at the position in the command line where the implicit linker script was read. This can affect archive searching. +@node Plugins +@chapter Linker Plugins + +@cindex plugins +@cindex linker plugins +The linker can use dynamically loaded plugins to modify its behavior. +For example, the link-time optimization feature that some compilers +support is implemented with a linker plugin. + +Currently there is only one plugin shipped by default, but more may +be added here later. + +@menu +* libdep Plugin:: Static Library Dependencies Plugin +@end menu + +@node libdep Plugin +@section Static Library Dependencies Plugin +@cindex static library dependencies +Originally, static libraries were contained in an archive file consisting +just of a collection of relocatable object files. Later they evolved to +optionally include a symbol table, to assist in finding the needed objects +within a library. There their evolution ended, and dynamic libraries +rose to ascendance. + +One useful feature of dynamic libraries was that, more than just collecting +multiple objects into a single file, they also included a list of their +dependencies, such that one could specify just the name of a single dynamic +library at link time, and all of its dependencies would be implicitly +referenced as well. But static libraries lacked this feature, so if a +link invocation was switched from using dynamic libraries to static +libraries, the link command would usually fail unless it was rewritten to +explicitly list the dependencies of the static library. + +The GNU @command{ar} utility now supports a @option{--record-libdeps} option +to embed dependency lists into static libraries as well, and the @file{libdep} +plugin may be used to read this dependency information at link time. The +dependency information is stored as a single string, carrying @option{-l} +and @option{-L} arguments as they would normally appear in a linker +command line. As such, the information can be written with any text +utility and stored into any archive, even if GNU @command{ar} is not +being used to create the archive. The information is stored in an +archive member named @samp{__.LIBDEP}. + +For example, given a library @file{libssl.a} that depends on another +library @file{libcrypto.a} which may be found in @file{/usr/local/lib}, +the @samp{__.LIBDEP} member of @file{libssl.a} would contain + +@smallexample +-L/usr/local/lib -lcrypto +@end smallexample + @ifset GENERIC @node Machine Dependent @chapter Machine Dependent Features diff --git a/ld/libdep_plugin.c b/ld/libdep_plugin.c new file mode 100644 index 0000000000..2a7fdc4d0b --- /dev/null +++ b/ld/libdep_plugin.c @@ -0,0 +1,366 @@ +/* libdeps plugin for the GNU linker. + Copyright (C) 2020 Free Software Foundation, Inc. + + This file is part of the GNU Binutils. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, + MA 02110-1301, USA. */ + +#include "sysdep.h" +#include "bfd.h" +#if BFD_SUPPORTS_PLUGINS +#include "plugin-api.h" + +#include /* For isspace. */ + +extern enum ld_plugin_status onload (struct ld_plugin_tv *tv); + +/* Helper for calling plugin api message function. */ +#define TV_MESSAGE if (tv_message) (*tv_message) + +/* Function pointers to cache hooks passed at onload time. */ +static ld_plugin_register_claim_file tv_register_claim_file = 0; +static ld_plugin_register_all_symbols_read tv_register_all_symbols_read = 0; +static ld_plugin_register_cleanup tv_register_cleanup = 0; +static ld_plugin_message tv_message = 0; +static ld_plugin_add_input_library tv_add_input_library = 0; +static ld_plugin_set_extra_library_path tv_set_extra_library_path = 0; + +/* Handle/record information received in a transfer vector entry. */ +static enum ld_plugin_status +parse_tv_tag (struct ld_plugin_tv *tv) +{ +#define SETVAR(x) x = tv->tv_u.x + switch (tv->tv_tag) + { + case LDPT_REGISTER_CLAIM_FILE_HOOK: + SETVAR(tv_register_claim_file); + break; + case LDPT_REGISTER_ALL_SYMBOLS_READ_HOOK: + SETVAR(tv_register_all_symbols_read); + break; + case LDPT_REGISTER_CLEANUP_HOOK: + SETVAR(tv_register_cleanup); + break; + case LDPT_MESSAGE: + SETVAR(tv_message); + break; + case LDPT_ADD_INPUT_LIBRARY: + SETVAR(tv_add_input_library); + break; + case LDPT_SET_EXTRA_LIBRARY_PATH: + SETVAR(tv_set_extra_library_path); + break; + default: + break; + } +#undef SETVAR + return LDPS_OK; +} + +/* Defs for archive parsing. */ +#define ARMAGSIZE 8 +typedef struct arhdr +{ + char ar_name[16]; + char ar_date[12]; + char ar_uid[6]; + char ar_gid[6]; + char ar_mode[8]; + char ar_size[10]; + char ar_fmag[2]; +} arhdr; + +typedef struct linerec +{ + struct linerec *next; + char line[]; +} linerec; + +#define LIBDEPS "__.LIBDEP/ " + +static linerec *line_head, **line_tail = &line_head; + +static enum ld_plugin_status +get_libdeps (int fd) +{ + arhdr ah; + int len; + unsigned long mlen; + linerec *lr; + enum ld_plugin_status rc = LDPS_NO_SYMS; + + lseek (fd, ARMAGSIZE, SEEK_SET); + for (;;) + { + len = read (fd, (void *) &ah, sizeof (ah)); + if (len != sizeof (ah)) + break; + mlen = strtoul (ah.ar_size, NULL, 10); + if (!mlen || strncmp (ah.ar_name, LIBDEPS, sizeof (LIBDEPS)-1)) + { + lseek (fd, mlen, SEEK_CUR); + continue; + } + lr = malloc (sizeof (linerec) + mlen); + if (!lr) + return LDPS_ERR; + lr->next = NULL; + len = read (fd, lr->line, mlen); + lr->line[mlen-1] = '\0'; + *line_tail = lr; + line_tail = &lr->next; + rc = LDPS_OK; + break; + } + return rc; +} + +/* Turn a string into an argvec. */ +static char ** +str2vec (char *in) +{ + char **res; + char *s, *first, *end; + char *sq, *dq; + int i; + + end = in + strlen (in); + s = in; + while (isspace (*s)) s++; + first = s; + + i = 1; + while ((s = strchr (s, ' '))) + { + s++; + i++; + } + res = (char **)malloc ((i+1) * sizeof (char *)); + if (!res) + return res; + + i = 0; + sq = NULL; + dq = NULL; + res[0] = first; + for (s = first; *s; s++) + { + if (*s == '\\') + { + memmove (s, s+1, end-s-1); + end--; + } + if (isspace (*s)) + { + if (sq || dq) + continue; + *s++ = '\0'; + while (isspace (*s)) s++; + if (*s) + res[++i] = s; + } + if (*s == '\'' && !dq) + { + if (sq) + { + memmove (sq, sq+1, s-sq-1); + memmove (s-2, s+1, end-s-1); + end -= 2; + s--; + sq = NULL; + } + else + { + sq = s; + } + } + if (*s == '"' && !sq) + { + if (dq) + { + memmove (dq, dq+1, s-dq-1); + memmove (s-2, s+1, end-s-1); + end -= 2; + s--; + dq = NULL; + } + else + { + dq = s; + } + } + } + res[++i] = NULL; + return res; +} + +static char *prevfile; + +/* Standard plugin API registerable hook. */ +static enum ld_plugin_status +onclaim_file (const struct ld_plugin_input_file *file, int *claimed) +{ + enum ld_plugin_status rv; + + *claimed = 0; + + /* If we've already seen this file, ignore it. */ + if (prevfile && !strcmp (file->name, prevfile)) + return LDPS_OK; + + /* If it's not an archive member, ignore it. */ + if (!file->offset) + return LDPS_OK; + + if (prevfile) + free (prevfile); + + prevfile = strdup (file->name); + if (!prevfile) + return LDPS_ERR; + + /* This hook only gets called on actual object files. + * We have to examine the archive ourselves, to find + * our LIBDEPS member. */ + rv = get_libdeps (file->fd); + if (rv == LDPS_ERR) + return rv; + + if (rv == LDPS_OK) + { + linerec *lr = (linerec *)line_tail; + /* Inform the user/testsuite. */ + TV_MESSAGE (LDPL_INFO, "got deps for library %s: %s", + file->name, lr->line); + fflush (NULL); + } + + return LDPS_OK; +} + +/* Standard plugin API registerable hook. */ +static enum ld_plugin_status +onall_symbols_read (void) +{ + linerec *lr; + char **vec; + enum ld_plugin_status rv = LDPS_OK; + + while ((lr = line_head)) + { + line_head = lr->next; + vec = str2vec (lr->line); + if (vec) + { + int i; + for (i = 0; vec[i]; i++) + { + if (vec[i][0] != '-') + { + TV_MESSAGE (LDPL_WARNING, "ignoring libdep argument %s", + vec[i]); + fflush (NULL); + continue; + } + if (vec[i][1] == 'l') + rv = tv_add_input_library (vec[i]+2); + else if (vec[i][1] == 'L') + rv = tv_set_extra_library_path (vec[i]+2); + else + { + TV_MESSAGE (LDPL_WARNING, "ignoring libdep argument %s", + vec[i]); + fflush (NULL); + } + if (rv != LDPS_OK) + break; + } + free (vec); + } + free (lr); + } + line_tail = NULL; + return rv; +} + +/* Standard plugin API registerable hook. */ +static enum ld_plugin_status +oncleanup (void) +{ + if (prevfile) + { + free (prevfile); + prevfile = NULL; + } + if (line_head) + { + linerec *lr; + while ((lr = line_head)) + { + line_head = lr->next; + free (lr); + } + line_tail = NULL; + } + return LDPS_OK; +} + +/* Standard plugin API entry point. */ +enum ld_plugin_status +onload (struct ld_plugin_tv *tv) +{ + enum ld_plugin_status rv; + + /* This plugin requires a valid tv array. */ + if (!tv) + return LDPS_ERR; + + /* First entry should always be LDPT_MESSAGE, letting us get + hold of it easily so we can send output straight away. */ + if (tv[0].tv_tag == LDPT_MESSAGE) + tv_message = tv[0].tv_u.tv_message; + + do + if ((rv = parse_tv_tag (tv)) != LDPS_OK) + return rv; + while ((tv++)->tv_tag != LDPT_NULL); + + /* Register hooks. */ + if (!tv_register_claim_file) + { + TV_MESSAGE (LDPL_FATAL, "No register_claim_file hook"); + fflush (NULL); + return LDPS_ERR; + } + (*tv_register_claim_file) (onclaim_file); + if (!tv_register_all_symbols_read) + { + TV_MESSAGE (LDPL_FATAL, "No register_all_symbols_read hook"); + fflush (NULL); + return LDPS_ERR; + } + (*tv_register_all_symbols_read) (onall_symbols_read); + if (!tv_register_cleanup) + { + TV_MESSAGE (LDPL_FATAL, "No register_cleanup hook"); + fflush (NULL); + return LDPS_ERR; + } + (*tv_register_cleanup) (oncleanup); + fflush (NULL); + return LDPS_OK; +} +#endif /* BFD_SUPPORTS_PLUGINS */ -- 2.20.1 --------------E614D1EF9C78B58D16B3E94D--