From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by sourceware.org (Postfix) with ESMTPS id 10428385783D for ; Thu, 22 Oct 2020 21:08:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 10428385783D Received: by mail-qt1-x844.google.com with SMTP id c13so2419047qtx.6 for ; Thu, 22 Oct 2020 14:08:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:references:from:autocrypt:subject:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=QaiidH9VtGM59gTVpCC3saGFSQqgR8Ff6QZiy0AuXTM=; b=BPONcPGzpjCD3NOOph8PjMaxhc0UTweL5ZeIFnl3EXJ05S0eTBsxeD2oxm8BfwAjJx h0K2UHTVto4DTYeU8gK5Yw3rVSgsF7gzLAXkOMfFHseIixwYQ9pJJdxm6xJ5bLp866MQ vfLWlseXbpHX35bSjvuxuQwj0IkBFUmD6DDXerLtk3alvFsJtbz25NW2mpvLKHPWqjjn Z7LTr51kQyoaanwtIoJupNAx0ZkYRr9t3vUXk6odL70z76md+kPVtCJDkDGmdiuFXIui vaaz9WVv2n/aE14t0GzdN2gC6VdxI6291YzaEB/s8rE5bUb5xh9VEoKbLypUIlNQ2hWQ PqvA== X-Gm-Message-State: AOAM532Ix1dKwj6KCS3WUdqzD5N1lBXBh7YkBGNm9AgtSp2Jvro7FCMy h7a/6gDuPQyD0a6mXXoKbxmbLMc2bSBZpQ== X-Google-Smtp-Source: ABdhPJxo2OCApOPDl3J1n45xnI3xXev0nX8NZQl3wXWNaMkinvmZwMMyMVb0fWkkkMW02XpySp5BaA== X-Received: by 2002:aed:3264:: with SMTP id y91mr4119356qtd.100.1603400928248; Thu, 22 Oct 2020 14:08:48 -0700 (PDT) Received: from [192.168.1.4] ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d56sm2032941qte.34.2020.10.22.14.08.47 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 22 Oct 2020 14:08:47 -0700 (PDT) To: libc-alpha@sourceware.org References: From: Adhemerval Zanella Autocrypt: addr=adhemerval.zanella@linaro.org; prefer-encrypt=mutual; keydata= mQINBFcVGkoBEADiQU2x/cBBmAVf5C2d1xgz6zCnlCefbqaflUBw4hB/bEME40QsrVzWZ5Nq 8kxkEczZzAOKkkvv4pRVLlLn/zDtFXhlcvQRJ3yFMGqzBjofucOrmdYkOGo0uCaoJKPT186L NWp53SACXguFJpnw4ODI64ziInzXQs/rUJqrFoVIlrPDmNv/LUv1OVPKz20ETjgfpg8MNwG6 iMizMefCl+RbtXbIEZ3TE/IaDT/jcOirjv96lBKrc/pAL0h/O71Kwbbp43fimW80GhjiaN2y WGByepnkAVP7FyNarhdDpJhoDmUk9yfwNuIuESaCQtfd3vgKKuo6grcKZ8bHy7IXX1XJj2X/ BgRVhVgMHAnDPFIkXtP+SiarkUaLjGzCz7XkUn4XAGDskBNfbizFqYUQCaL2FdbW3DeZqNIa nSzKAZK7Dm9+0VVSRZXP89w71Y7JUV56xL/PlOE+YKKFdEw+gQjQi0e+DZILAtFjJLoCrkEX w4LluMhYX/X8XP6/C3xW0yOZhvHYyn72sV4yJ1uyc/qz3OY32CRy+bwPzAMAkhdwcORA3JPb kPTlimhQqVgvca8m+MQ/JFZ6D+K7QPyvEv7bQ7M+IzFmTkOCwCJ3xqOD6GjX3aphk8Sr0dq3 4Awlf5xFDAG8dn8Uuutb7naGBd/fEv6t8dfkNyzj6yvc4jpVxwARAQABtElBZGhlbWVydmFs IFphbmVsbGEgTmV0dG8gKExpbmFybyBWUE4gS2V5KSA8YWRoZW1lcnZhbC56YW5lbGxhQGxp bmFyby5vcmc+iQI3BBMBCAAhBQJXFRpKAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKqx7BSnlIjv0e8P/1YOYoNkvJ+AJcNUaM5a2SA9oAKjSJ/M/EN4Id5Ow41ZJS4lUA0apSXW NjQg3VeVc2RiHab2LIB4MxdJhaWTuzfLkYnBeoy4u6njYcaoSwf3g9dSsvsl3mhtuzm6aXFH /Qsauav77enJh99tI4T+58rp0EuLhDsQbnBic/ukYNv7sQV8dy9KxA54yLnYUFqH6pfH8Lly sTVAMyi5Fg5O5/hVV+Z0Kpr+ZocC1YFJkTsNLAW5EIYSP9ftniqaVsim7MNmodv/zqK0IyDB GLLH1kjhvb5+6ySGlWbMTomt/or/uvMgulz0bRS+LUyOmlfXDdT+t38VPKBBVwFMarNuREU2 69M3a3jdTfScboDd2ck1u7l+QbaGoHZQ8ZNUrzgObltjohiIsazqkgYDQzXIMrD9H19E+8fw kCNUlXxjEgH/Kg8DlpoYJXSJCX0fjMWfXywL6ZXc2xyG/hbl5hvsLNmqDpLpc1CfKcA0BkK+ k8R57fr91mTCppSwwKJYO9T+8J+o4ho/CJnK/jBy1pWKMYJPvvrpdBCWq3MfzVpXYdahRKHI ypk8m4QlRlbOXWJ3TDd/SKNfSSrWgwRSg7XCjSlR7PNzNFXTULLB34sZhjrN6Q8NQZsZnMNs TX8nlGOVrKolnQPjKCLwCyu8PhllU8OwbSMKskcD1PSkG6h3r0AquQINBFcVGkoBEACgAdbR Ck+fsfOVwT8zowMiL3l9a2DP3Eeak23ifdZG+8Avb/SImpv0UMSbRfnw/N81IWwlbjkjbGTu oT37iZHLRwYUFmA8fZX0wNDNKQUUTjN6XalJmvhdz9l71H3WnE0wneEM5ahu5V1L1utUWTyh VUwzX1lwJeV3vyrNgI1kYOaeuNVvq7npNR6t6XxEpqPsNc6O77I12XELic2+36YibyqlTJIQ V1SZEbIy26AbC2zH9WqaKyGyQnr/IPbTJ2Lv0dM3RaXoVf+CeK7gB2B+w1hZummD21c1Laua +VIMPCUQ+EM8W9EtX+0iJXxI+wsztLT6vltQcm+5Q7tY+HFUucizJkAOAz98YFucwKefbkTp eKvCfCwiM1bGatZEFFKIlvJ2QNMQNiUrqJBlW9nZp/k7pbG3oStOjvawD9ZbP9e0fnlWJIsj 6c7pX354Yi7kxIk/6gREidHLLqEb/otuwt1aoMPg97iUgDV5mlNef77lWE8vxmlY0FBWIXuZ yv0XYxf1WF6dRizwFFbxvUZzIJp3spAao7jLsQj1DbD2s5+S1BW09A0mI/1DjB6EhNN+4bDB SJCOv/ReK3tFJXuj/HbyDrOdoMt8aIFbe7YFLEExHpSk+HgN05Lg5TyTro8oW7TSMTk+8a5M kzaH4UGXTTBDP/g5cfL3RFPl79ubXwARAQABiQIfBBgBCAAJBQJXFRpKAhsMAAoJEKqx7BSn lIjvI/8P/jg0jl4Tbvg3B5kT6PxJOXHYu9OoyaHLcay6Cd+ZrOd1VQQCbOcgLFbf4Yr+rE9l mYsY67AUgq2QKmVVbn9pjvGsEaz8UmfDnz5epUhDxC6yRRvY4hreMXZhPZ1pbMa6A0a/WOSt AgFj5V6Z4dXGTM/lNManr0HjXxbUYv2WfbNt3/07Db9T+GZkpUotC6iknsTA4rJi6u2ls0W9 1UIvW4o01vb4nZRCj4rni0g6eWoQCGoVDk/xFfy7ZliR5B+3Z3EWRJcQskip/QAHjbLa3pml xAZ484fVxgeESOoaeC9TiBIp0NfH8akWOI0HpBCiBD5xaCTvR7ujUWMvhsX2n881r/hNlR9g fcE6q00qHSPAEgGr1bnFv74/1vbKtjeXLCcRKk3Ulw0bY1OoDxWQr86T2fZGJ/HIZuVVBf3+ gaYJF92GXFynHnea14nFFuFgOni0Mi1zDxYH/8yGGBXvo14KWd8JOW0NJPaCDFJkdS5hu0VY 7vJwKcyHJGxsCLU+Et0mryX8qZwqibJIzu7kUJQdQDljbRPDFd/xmGUFCQiQAncSilYOcxNU EMVCXPAQTteqkvA+gNqSaK1NM9tY0eQ4iJpo+aoX8HAcn4sZzt2pfUB9vQMTBJ2d4+m/qO6+ cFTAceXmIoFsN8+gFN3i8Is3u12u8xGudcBPvpoy4OoG Subject: Re: [PATCH 25/28] elf: Implement tail merging of strings in ldconfig Message-ID: Date: Thu, 22 Oct 2020 18:08:45 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-14.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 21:08:50 -0000 On 01/10/2020 13:34, Florian Weimer via Libc-alpha wrote: > This simplifies the string table construction in elf/cache.c > because there is no more need to keep track of offsets explicitly; > the string table implementation does this internally. > > This change slightly reduces the size of the cache on disk. The > file format does not change as a result. The strings are > null-terminated, without explicit length, so tail merging is > transparent to readers. LGTM, thanks. > --- > elf/Makefile | 3 +- > elf/cache.c | 84 ++++++++++++++++++++++++++++------------------------ > 2 files changed, 48 insertions(+), 39 deletions(-) > > diff --git a/elf/Makefile b/elf/Makefile > index ad50a3e16e..5ad8df7da3 100644 > --- a/elf/Makefile > +++ b/elf/Makefile > @@ -118,7 +118,8 @@ others-static += ldconfig > others += ldconfig > install-rootsbin += ldconfig > > -ldconfig-modules := cache readlib xmalloc xstrdup chroot_canon static-stubs > +ldconfig-modules := cache readlib xmalloc xstrdup chroot_canon static-stubs \ > + stringtable > extra-objs += $(ldconfig-modules:=.o) > others-extras = $(ldconfig-modules) > endif Ok. > diff --git a/elf/cache.c b/elf/cache.c > index 3a02a4070a..eda3da98a7 100644 > --- a/elf/cache.c > +++ b/elf/cache.c > @@ -35,11 +35,15 @@ > #include > #include > #include > +#include > + > +/* Used to store library names, paths, and other strings. */ > +struct stringtable strings; Maybe static here? > > struct cache_entry > { > - char *lib; /* Library name. */ > - char *path; /* Path to find library. */ > + struct stringtable_entry *lib; /* Library name. */ > + struct stringtable_entry *path; /* Path to find library. */ > int flags; /* Flags to indicate kind of library. */ > unsigned int osversion; /* Required OS version. */ > uint64_t hwcap; /* Important hardware capabilities. */ Ok. > @@ -300,7 +304,7 @@ static int > compare (const struct cache_entry *e1, const struct cache_entry *e2) > { > /* We need to swap entries here to get the correct sort order. */ > - int res = _dl_cache_libcmp (e2->lib, e1->lib); > + int res = _dl_cache_libcmp (e2->lib->string, e1->lib->string); > if (res == 0) > { > if (e1->flags < e2->flags) Ok. > @@ -369,26 +373,24 @@ save_cache (const char *cache_name) > { > /* The cache entries are sorted already, save them in this order. */ > > - /* Count the length of all strings. */ > - /* The old format doesn't contain hwcap entries and doesn't contain > - libraries in subdirectories with hwcaps entries. Count therefore > - also all entries with hwcap == 0. */ > - size_t total_strlen = 0; > struct cache_entry *entry; > /* Number of cache entries. */ > int cache_entry_count = 0; > - /* Number of normal cache entries. */ > + /* The old format doesn't contain hwcap entries and doesn't contain > + libraries in subdirectories with hwcaps entries. Count therefore > + also all entries with hwcap == 0. */ > int cache_entry_old_count = 0; > > for (entry = entries; entry != NULL; entry = entry->next) > { > - /* Account the final NULs. */ > - total_strlen += strlen (entry->lib) + strlen (entry->path) + 2; > ++cache_entry_count; > if (entry->hwcap == 0) > ++cache_entry_old_count; > } > > + struct stringtable_finalized strings_finalized; > + stringtable_finalize (&strings, &strings_finalized); > + > /* Create the on disk cache structure. */ > struct cache_file *file_entries = NULL; > size_t file_entries_size = 0; Ok. > @@ -432,7 +434,7 @@ save_cache (const char *cache_name) > sizeof CACHE_VERSION - 1); > > file_entries_new->nlibs = cache_entry_count; > - file_entries_new->len_strings = total_strlen; > + file_entries_new->len_strings = strings_finalized.size; > file_entries_new->flags = cache_file_new_flags_endian; > } > Ok. > @@ -449,20 +451,20 @@ save_cache (const char *cache_name) > str_offset = 0; > > /* An array for all strings. */ > - char *strings = xmalloc (total_strlen); > - char *str = strings; > int idx_old; > int idx_new; > > for (idx_old = 0, idx_new = 0, entry = entries; entry != NULL; > entry = entry->next, ++idx_new) > { > - /* First the library. */ > if (opt_format != 2 && entry->hwcap == 0) > { > file_entries->libs[idx_old].flags = entry->flags; > /* XXX: Actually we can optimize here and remove duplicates. */ > file_entries->libs[idx_old].key = str_offset + pad; > + file_entries->libs[idx_new].key = str_offset + entry->lib->offset; > + file_entries->libs[idx_new].value > + = str_offset + entry->path->offset; > } > if (opt_format != 0) > { Ok. > @@ -473,20 +475,12 @@ save_cache (const char *cache_name) > file_entries_new->libs[idx_new].flags = entry->flags; > file_entries_new->libs[idx_new].osversion = entry->osversion; > file_entries_new->libs[idx_new].hwcap = entry->hwcap; > - file_entries_new->libs[idx_new].key = str_offset; > + file_entries_new->libs[idx_new].key > + = str_offset + entry->lib->offset; > + file_entries_new->libs[idx_new].value > + = str_offset + entry->path->offset; > } > > - size_t len = strlen (entry->lib) + 1; > - str = mempcpy (str, entry->lib, len); > - str_offset += len; > - /* Then the path. */ > - if (opt_format != 2 && entry->hwcap == 0) > - file_entries->libs[idx_old].value = str_offset + pad; > - if (opt_format != 0) > - file_entries_new->libs[idx_new].value = str_offset; > - len = strlen (entry->path) + 1; > - str = mempcpy (str, entry->path, len); > - str_offset += len; > /* Ignore entries with hwcap for old format. */ > if (entry->hwcap == 0) > ++idx_old; Ok. > @@ -511,7 +505,7 @@ save_cache (const char *cache_name) > extension_offset += pad; > extension_offset += file_entries_new_size; > } > - extension_offset += total_strlen; > + extension_offset += strings_finalized.size; > extension_offset = roundup (extension_offset, 4); /* Provide alignment. */ > if (opt_format != 0) > file_entries_new->extension_offset = extension_offset; Ok. > @@ -551,7 +545,8 @@ save_cache (const char *cache_name) > error (EXIT_FAILURE, errno, _("Writing of cache data failed")); > } > > - if (write (fd, strings, total_strlen) != (ssize_t) total_strlen) > + if (write (fd, strings_finalized.strings, strings_finalized.size) > + != (ssize_t) strings_finalized.size) > error (EXIT_FAILURE, errno, _("Writing of cache data failed")); > > if (opt_format != 0) Ok. > @@ -580,7 +575,7 @@ save_cache (const char *cache_name) > /* Free all allocated memory. */ > free (file_entries_new); > free (file_entries); > - free (strings); > + free (strings_finalized.strings); > > while (entries) > { Ok. > @@ -596,14 +591,27 @@ void > add_to_cache (const char *path, const char *lib, int flags, > unsigned int osversion, uint64_t hwcap) > { > - size_t liblen = strlen (lib) + 1; > - size_t len = liblen + strlen (path) + 1; > - struct cache_entry *new_entry > - = xmalloc (sizeof (struct cache_entry) + liblen + len); > - > - new_entry->lib = memcpy ((char *) (new_entry + 1), lib, liblen); > - new_entry->path = new_entry->lib + liblen; > - snprintf (new_entry->path, len, "%s/%s", path, lib); > + struct cache_entry *new_entry = xmalloc (sizeof (*new_entry)); > + > + struct stringtable_entry *path_interned; > + { > + /* Use a small, on-stack buffer in most cases. */ > + char buf[200]; > + int ret = snprintf (buf, sizeof (buf), "%s/%s", path, lib); > + if (ret < 0 || ret >= sizeof (buf) - 1) > + { > + char *p; > + if (asprintf (&p, "%s/%s", path, lib) < 0) > + error (EXIT_FAILURE, errno, _("Could not create library path")); > + path_interned = stringtable_intern (&strings, p); > + free (p); > + } > + else > + path_interned = stringtable_intern (&strings, buf); > + } > + > + new_entry->lib = stringtable_intern (&strings, lib); > + new_entry->path = path_interned; > new_entry->flags = flags; > new_entry->osversion = osversion; > new_entry->hwcap = hwcap; > Ok. Is this small string optimization really worth instead of just using asprintf?