* Enable linknamespace testing for libdl and libcrypt
@ 2016-11-16 19:06 Joseph Myers
2016-11-16 19:47 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Myers @ 2016-11-16 19:06 UTC (permalink / raw)
To: libc-alpha
When I set up linknamespace testing, the lists of libraries that might
contain functions from various standards were based on the -l options
POSIX says may be required to find certain functions with the c99
utility.
glibc has some POSIX functions in the libdl and libcrypt libraries,
not mentioned in the definition of the c99 utility (so an
implementation of that utility using glibc would need to use -ldl
-lcrypt automatically). This patch adds those libraries to the ones
considered in linknamespace testing for relevant standards. (The
crypt functions are XSI only, present in XPG3 and above; the libdl
ones were added in UNIX98, then moved from XSI to BASE in the 2008
edition of POSIX.)
I intend to commit this once the fix for bug 20829 (which I tested in
conjunction with this patch), crypt namespace issues, is in.
2016-11-16 Joseph Myers <joseph@codesourcery.com>
* conform/Makefile (linknamespace-libs): Rename to
linknamespace-libs-thr.
(linknamespace-libs-posix): New variable.
(linknamespace-libs-xsi): Likewise.
(linknamespace-libs-XPG3): Include libcrypt.a.
(linknamespace-libs-XPG4): Use $(linknamespace-libs-XPG3).
(linknamespace-libs-POSIX): Use $(linknamespace-libs-thr).
(linknamespace-libs-UNIX98): Use $(linknamespace-libs-xsi).
(linknamespace-libs-XOPEN2K): Likewise.
(linknamespace-libs-XOPEN2K8): Likewise.
(linknamespace-libs-POSIX2008): Use $(linknamespace-libs-posix).
diff --git a/conform/Makefile b/conform/Makefile
index 7883624..5862f70 100644
--- a/conform/Makefile
+++ b/conform/Makefile
@@ -209,18 +209,23 @@ $(linknamespace-symlists-tests): $(objpfx)symlist-%: list-header-symbols.pl
$(evaluate-test)
linknamespace-libs-isoc = $(common-objpfx)libc.a $(common-objpfx)math/libm.a
-linknamespace-libs = $(linknamespace-libs-isoc) \
- $(common-objpfx)rt/librt.a $(static-thread-library)
+linknamespace-libs-thr = $(linknamespace-libs-isoc) \
+ $(common-objpfx)rt/librt.a $(static-thread-library)
+linknamespace-libs-posix = $(linknamespace-libs-thr) \
+ $(common-objpfx)dlfcn/libdl.a
+linknamespace-libs-xsi = $(linknamespace-libs-posix) \
+ $(common-objpfx)crypt/libcrypt.a
linknamespace-libs-ISO = $(linknamespace-libs-isoc)
linknamespace-libs-ISO99 = $(linknamespace-libs-isoc)
linknamespace-libs-ISO11 = $(linknamespace-libs-isoc)
-linknamespace-libs-XPG3 = $(linknamespace-libs-isoc)
-linknamespace-libs-XPG4 = $(linknamespace-libs-isoc)
-linknamespace-libs-POSIX = $(linknamespace-libs)
-linknamespace-libs-UNIX98 = $(linknamespace-libs)
-linknamespace-libs-XOPEN2K = $(linknamespace-libs)
-linknamespace-libs-POSIX2008 = $(linknamespace-libs)
-linknamespace-libs-XOPEN2K8 = $(linknamespace-libs)
+linknamespace-libs-XPG3 = $(linknamespace-libs-isoc) \
+ $(common-objpfx)crypt/libcrypt.a
+linknamespace-libs-XPG4 = $(linknamespace-libs-XPG3)
+linknamespace-libs-POSIX = $(linknamespace-libs-thr)
+linknamespace-libs-UNIX98 = $(linknamespace-libs-xsi)
+linknamespace-libs-XOPEN2K = $(linknamespace-libs-xsi)
+linknamespace-libs-POSIX2008 = $(linknamespace-libs-posix)
+linknamespace-libs-XOPEN2K8 = $(linknamespace-libs-xsi)
$(linknamespace-symlist-stdlibs-tests): $(objpfx)symlist-stdlibs-%: \
$(linknamespace-libs)
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-16 19:06 Enable linknamespace testing for libdl and libcrypt Joseph Myers
@ 2016-11-16 19:47 ` Florian Weimer
2016-11-16 22:45 ` Joseph Myers
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer @ 2016-11-16 19:47 UTC (permalink / raw)
To: Joseph Myers; +Cc: libc-alpha
On 11/16/2016 08:06 PM, Joseph Myers wrote:
> +linknamespace-libs-XPG3 = $(linknamespace-libs-isoc) \
> + $(common-objpfx)crypt/libcrypt.a
Does the linknamespace test perform a full link, and fails if it
encounters undefined symbols?
I don't think anyone has the requisite static NSS libraries for linking
libcrypt statically with --enable-nss-crypt. So you may have to disable
this for $(nss-crypt) configurations.
Carlos will submit a patch to do this for elf/tst-linkall-static. We
weren't aware of this issue until this new test went in.
Thanks,
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-16 19:47 ` Florian Weimer
@ 2016-11-16 22:45 ` Joseph Myers
2016-11-17 9:08 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Myers @ 2016-11-16 22:45 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
On Wed, 16 Nov 2016, Florian Weimer wrote:
> On 11/16/2016 08:06 PM, Joseph Myers wrote:
> > +linknamespace-libs-XPG3 = $(linknamespace-libs-isoc) \
> > + $(common-objpfx)crypt/libcrypt.a
>
> Does the linknamespace test perform a full link, and fails if it encounters
> undefined symbols?
No, it has its own emulation of linker logic, and ignores failure to find
a definition of a symbol (for example, libgcc symbols - it doesn't look
for libgcc.a / libgcc_eh.a, so it won't detect namespace issues there).
(If an undefined symbol breaks namespace rules, that will still result in
a test failure unless whitelisted in linknamespace.pl, whether or not a
definition of that symbol was found.)
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-16 22:45 ` Joseph Myers
@ 2016-11-17 9:08 ` Florian Weimer
2016-11-17 14:18 ` Joseph Myers
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer @ 2016-11-17 9:08 UTC (permalink / raw)
To: Joseph Myers; +Cc: libc-alpha
On 11/16/2016 11:45 PM, Joseph Myers wrote:
> On Wed, 16 Nov 2016, Florian Weimer wrote:
>
>> On 11/16/2016 08:06 PM, Joseph Myers wrote:
>>> +linknamespace-libs-XPG3 = $(linknamespace-libs-isoc) \
>>> + $(common-objpfx)crypt/libcrypt.a
>>
>> Does the linknamespace test perform a full link, and fails if it encounters
>> undefined symbols?
>
> No, it has its own emulation of linker logic, and ignores failure to find
> a definition of a symbol (for example, libgcc symbols - it doesn't look
> for libgcc.a / libgcc_eh.a, so it won't detect namespace issues there).
> (If an undefined symbol breaks namespace rules, that will still result in
> a test failure unless whitelisted in linknamespace.pl, whether or not a
> definition of that symbol was found.)
Understood. Then the linknamespace test fails thusly:
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOWHASH_Begin
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOWHASH_Destroy
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOWHASH_End
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOWHASH_NewContext
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOWHASH_Update
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOW_Init
[initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
[libcrypt.a(md5-crypt.o)] NSSLOW_Shutdown
This is, of course, completely correct and expected. I don't know what
to do about it. XFAIL it?
I'm trying to get rid of NSS-based crypt downstream, but it's going to
take a while (it's already optional in Fedora).
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-17 9:08 ` Florian Weimer
@ 2016-11-17 14:18 ` Joseph Myers
2016-11-17 14:23 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Myers @ 2016-11-17 14:18 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
On Thu, 17 Nov 2016, Florian Weimer wrote:
> Understood. Then the linknamespace test fails thusly:
>
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_Begin
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_Destroy
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_End
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_NewContext
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_Update
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOW_Init
> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
> [libcrypt.a(md5-crypt.o)] NSSLOW_Shutdown
>
> This is, of course, completely correct and expected. I don't know what to do
> about it. XFAIL it?
Assuming there are no implementation-namespace exports of those functions
that libcrypt could use instead, that indicates whitelisting in
linknamespace.pl with a comment referencing a bug filed for nss-crypt
namespace issues.
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-17 14:18 ` Joseph Myers
@ 2016-11-17 14:23 ` Florian Weimer
2016-11-17 14:31 ` Joseph Myers
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer @ 2016-11-17 14:23 UTC (permalink / raw)
To: Joseph Myers; +Cc: libc-alpha
On 11/17/2016 03:17 PM, Joseph Myers wrote:
> On Thu, 17 Nov 2016, Florian Weimer wrote:
>
>> Understood. Then the linknamespace test fails thusly:
>>
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_Begin
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_Destroy
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_End
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_NewContext
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOWHASH_Update
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOW_Init
>> [initial] crypt -> [libcrypt.a(crypt-entry.o)] __md5_crypt ->
>> [libcrypt.a(md5-crypt.o)] NSSLOW_Shutdown
>>
>> This is, of course, completely correct and expected. I don't know what to do
>> about it. XFAIL it?
>
> Assuming there are no implementation-namespace exports of those functions
> that libcrypt could use instead, that indicates whitelisting in
> linknamespace.pl with a comment referencing a bug filed for nss-crypt
> namespace issues.
Well, this is only relevant if we actually had a libfreebl.a which
defines the functions above. I don't think such a thing exists. Fedora
and downstreams don't have it, and neither does Debian jessie.
So it's really a “does it make a sound?”-type question.
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-17 14:23 ` Florian Weimer
@ 2016-11-17 14:31 ` Joseph Myers
2016-11-18 14:36 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Myers @ 2016-11-17 14:31 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
On Thu, 17 Nov 2016, Florian Weimer wrote:
> > Assuming there are no implementation-namespace exports of those functions
> > that libcrypt could use instead, that indicates whitelisting in
> > linknamespace.pl with a comment referencing a bug filed for nss-crypt
> > namespace issues.
>
> Well, this is only relevant if we actually had a libfreebl.a which defines the
> functions above. I don't think such a thing exists. Fedora and downstreams
> don't have it, and neither does Debian jessie.
As a cross-library namespace issue I'd expect it's just as relevant when
shared libcrypt is referencing these functions from another shared
library. That is, this configuration involves a namespace bug, which is
hard to fix so the names in question are whitelisted.
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-17 14:31 ` Joseph Myers
@ 2016-11-18 14:36 ` Florian Weimer
2016-11-18 18:07 ` Joseph Myers
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer @ 2016-11-18 14:36 UTC (permalink / raw)
To: Joseph Myers; +Cc: libc-alpha
On 11/17/2016 03:31 PM, Joseph Myers wrote:
> On Thu, 17 Nov 2016, Florian Weimer wrote:
>
>>> Assuming there are no implementation-namespace exports of those functions
>>> that libcrypt could use instead, that indicates whitelisting in
>>> linknamespace.pl with a comment referencing a bug filed for nss-crypt
>>> namespace issues.
>>
>> Well, this is only relevant if we actually had a libfreebl.a which defines the
>> functions above. I don't think such a thing exists. Fedora and downstreams
>> don't have it, and neither does Debian jessie.
>
> As a cross-library namespace issue I'd expect it's just as relevant when
> shared libcrypt is referencing these functions from another shared
> library.
This is correct. The library with the NSSLOW_* names is just a stub,
and it attempts to dlopen the real thing (without RTLD_GLOBAL), to
conserve the symbol footprint. But that library uses quite a few
functions which may have been interposed by the application, and
RTLD_LOCAL does not guard against that.
Note that moving these libraries into the implementation namespace is
counterproductive because we don't want non-toolchain libraries to be
located there because then we don't own that space anymore and cannot
argue that names in it are safe to use for us. But such low-level
libraries (libidn will be in the same boat soon) could certainly use
glibc functionality under names in the implementation namespace without
ill effect.
For the symbols they define, we need to make sure (at the distribution
level) that these symbols are reasonably prefixed in some way. Names
like “mutex“, “buffer”, ”yyin” are really not a good idea.
> That is, this configuration involves a namespace bug,
Agreed.
> which is hard to fix so the names in question are whitelisted.
Sorry, I don't understand this part.
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-18 14:36 ` Florian Weimer
@ 2016-11-18 18:07 ` Joseph Myers
2016-11-21 10:42 ` Florian Weimer
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Myers @ 2016-11-18 18:07 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
On Fri, 18 Nov 2016, Florian Weimer wrote:
> > That is, this configuration involves a namespace bug,
>
> Agreed.
>
> > which is hard to fix so the names in question are whitelisted.
>
> Sorry, I don't understand this part.
The whitelist in conform/linknamespace.pl covers both false positives and
genuine but hard-to-fix bugs. So these names should be added there with
an appropriate comment.
# The following whitelisted symbols are also allowed for now.
#
# * Bug 17576: stdin, stdout, stderr only reserved with external
# linkage when stdio.h included (and possibly not then), not
# generally.
#
# * Bug 18442: re_syntax_options wrongly brought in by regcomp and
# used by re_comp.
#
# * False positive: matherr only used conditionally. matherrf/matherrl are used
# by IA64 too for the same reason.
#
@whitelist = qw(stdin stdout stderr re_syntax_options matherr matherrf
matherrl);
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Enable linknamespace testing for libdl and libcrypt
2016-11-18 18:07 ` Joseph Myers
@ 2016-11-21 10:42 ` Florian Weimer
0 siblings, 0 replies; 10+ messages in thread
From: Florian Weimer @ 2016-11-21 10:42 UTC (permalink / raw)
To: Joseph Myers; +Cc: libc-alpha
On 11/18/2016 07:07 PM, Joseph Myers wrote:
> On Fri, 18 Nov 2016, Florian Weimer wrote:
>
>>> That is, this configuration involves a namespace bug,
>>
>> Agreed.
>>
>>> which is hard to fix so the names in question are whitelisted.
>>
>> Sorry, I don't understand this part.
>
> The whitelist in conform/linknamespace.pl covers both false positives and
> genuine but hard-to-fix bugs. So these names should be added there with
> an appropriate comment.
>
> # The following whitelisted symbols are also allowed for now.
> #
> # * Bug 17576: stdin, stdout, stderr only reserved with external
> # linkage when stdio.h included (and possibly not then), not
> # generally.
> #
> # * Bug 18442: re_syntax_options wrongly brought in by regcomp and
> # used by re_comp.
> #
> # * False positive: matherr only used conditionally. matherrf/matherrl are used
> # by IA64 too for the same reason.
> #
> @whitelist = qw(stdin stdout stderr re_syntax_options matherr matherrf
> matherrl);
Okay, this approach looks fine to me.
Thanks,
Florian
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2016-11-21 10:42 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-16 19:06 Enable linknamespace testing for libdl and libcrypt Joseph Myers
2016-11-16 19:47 ` Florian Weimer
2016-11-16 22:45 ` Joseph Myers
2016-11-17 9:08 ` Florian Weimer
2016-11-17 14:18 ` Joseph Myers
2016-11-17 14:23 ` Florian Weimer
2016-11-17 14:31 ` Joseph Myers
2016-11-18 14:36 ` Florian Weimer
2016-11-18 18:07 ` Joseph Myers
2016-11-21 10:42 ` Florian Weimer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).