Intention of "headers-not-in-tirpc"?

Intention of "headers-not-in-tirpc"?

[Thorsten Kukuk]

Hi,

one question:

in sunrpc/Makefile we have three variables for header files:

headers
headers-in-tirpc
headers-not-in-tirpc

What is the reason for the differentiation between this?

$(headers) is always installed, that's ok.
$(headers-in-tirpc) and $(headers-not-in-tirpc) are only installed,
if $(link-obsolete-rpc) is yes. I would have expected, that
$(headers-not-in-tirpc) will be installed if link-obsolete-rpc is no, too.

Is this a bug, that $(headers-not-in-tirpc) is not added? Or is this
differentiation obsolete? 

The reason is, that the headers-not-in-tirpc list is incorrect, and the
question is, what would be the correct fix?

Another problem is:
from resolv/Makefile:
ifeq (,$(filter sunrpc,$(subdirs)))
# The netdb.h we install does '#include <rpc/netdb.h>', so one must exist.
# If sunrpc/ is built in this configuration, it installs a real <rpc/netdb.h>.
# If that's not going to happen, install our dummy file.
headers += rpc/netdb.h
endif

The test does not seem to work, since sunrpc seems to be always
include sunrpc. But if not, installing an empty rpc/netdb.h is a bug,
this breaks existing code and prevents tirpc from installing an own
version.

  Thanks,
    Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: Intention of "headers-not-in-tirpc"?

[Roland McGrath]

Don't read too much into the existing state of the code.  There hasn't
been much use of the (default) --disable-obsolete-rpc configuration,
and the code for it was all written a few years ago when the state of
TI-RPC was different from what it is today.  

Now we are finally getting serious about the effort with active
coordination between TI-RPC maintainers and glibc maintainers (which
never really happened before, unfortunately).  So we will gladly
reorganize things on the libc side however makes sense for both a
smooth transition and reasonable maintenance burdens throughout.

I think we should just start from scratch with working out a
"requirements" list.  That is, compared to stock glibc trunk today
configured with --enable-obsolete-rpc, exactly how do we want things
to be different in the end state.  That includes which things libc
should install and which things TI-RPC should install, and so forth.
Once we've reached consensus across both projects about that, then
we can dig into the implementation details of the libc build.

As to rpc/netdb.h in particular, I have a few thoughts that might
help.

The stub rpc/netdb.h (resolv/rpc/netdb.h) exists for configurations
that do not include sunrpc at all (and never have, so they don't have
any ABI compatibility for it either).  (The only such configuration
supported today is arm-nacl.)  Nobody thought about how it would
interact with --disable-obsolete-rpc for Linux or Hurd configurations.

The only reason the stub is needed in any configuration is because
libc's netdb.h (resolv/netdb.h) does #include <rpc/netdb.h> (when not
in a strict POSIX mode).  I originally did that way back when (1995)
because, at the time, the <netdb.h> from BIND 4.9.3 included the
declarations of 'struct rpcent' and friends (under '#ifdef sun',
believe it or not), and I think the SunOS 4 (aka Solaris 1) version
had them too.  However, according to documentation I can find, as far
back as Solaris 7 (1998) and probably earlier, the documented header
for these things is <rpc/rpcent.h>.  That's what the BSDs have today,
and their <netdb.h> does not declare any rpcent stuff.

So I propose we change the glibc API to follow suit.  That is, make
<netdb.h> stop including <rpc/netdb.h> and rename <rpc/netdb.h> to
<rpc/rpcent.h>.  (We might leave <rpc/netdb.h> around as a wrapper
that gives a #warning to prefer <rpc/rpcent.h> instead if that seems
useful.)  Then it becomes straightforward that a glibc configuration
that supports the old sunrpc API (like today's libc configured with
--enable-obsolete-rpc) installs <rpc/rpcent.h>, while a glibc
configuration that does not support the sunrpc API (like the arm-nacl
configuration, or other builds with --disable-obsolete-rpc or a future
equivalent) does not install any <rpc/*> headers at all.

This would be an API change that might well break some packages,
albeit with the trivial fix of adding '#include <rpc/rpcent.h>' (but
needing configury to keep building on old glibc systems that don't
have that file).  So we'd need to get thorough consensus from the
glibc community, and consensus might well be to do it as a staged
deprecation rather than fixing <netdb.h> outright in one release cycle.
(Personally, I'm fine with changing this without advance notice.)

There's still the separate question about NSS support for rpcent
functions.  But we can address that after cleaning up the <netdb.h>
situation.


Thanks,
Roland

Re: Intention of "headers-not-in-tirpc"?

[Thorsten Kukuk]

On Fri, Mar 18, Roland McGrath wrote:

> I think we should just start from scratch with working out a
> "requirements" list.  That is, compared to stock glibc trunk today
> configured with --enable-obsolete-rpc, exactly how do we want things
> to be different in the end state.  That includes which things libc
> should install and which things TI-RPC should install, and so forth.
> Once we've reached consensus across both projects about that, then
> we can dig into the implementation details of the libc build.

I think glibc should install as less as possible. In the end, I think
it should only provide this functions, which requires NSS:

getrpcbyname.c, getrpcbyname_r.c, getrpcbynumber.c, getrpcbynumber_r.c,
getrpcent.c, getrpcent_r.c

Since this is code, which only includes other glibc functions, but
does not contain any RPC releated, and are all combined together in
a single header file (rpc/netdb.h), I don't see there a real issue
to seperate them.

A bigger problem are the publickey functions:
netname.c and publickey.c

Here, NSS support is required for NIS+, I'm not sure about NFS.
NIS+ is dead and the current TIRPC functions, which only use local
files, seems to work for everybody, so maybe this ones and the publickey
entry in /etc/nsswitch.conf could be removed. And since the prototypes
are spread over different header files, it would be difficult to let
them in glibc.

Third difference are the rpcsvc/*.x files. tirpc does not have them.
But all projects I'm are of, ship their own version anyways. If really
somebody needs one, we can add them to tirpc.

> As to rpc/netdb.h in particular, I have a few thoughts that might
> help.
[...]

> So I propose we change the glibc API to follow suit.  That is, make
> <netdb.h> stop including <rpc/netdb.h> and rename <rpc/netdb.h> to
> <rpc/rpcent.h>.  (We might leave <rpc/netdb.h> around as a wrapper
> that gives a #warning to prefer <rpc/rpcent.h> instead if that seems
> useful.)  Then it becomes straightforward that a glibc configuration
> that supports the old sunrpc API (like today's libc configured with
> --enable-obsolete-rpc) installs <rpc/rpcent.h>, while a glibc
> configuration that does not support the sunrpc API (like the arm-nacl
> configuration, or other builds with --disable-obsolete-rpc or a future
> equivalent) does not install any <rpc/*> headers at all.

Hm, I'm not really sure if I like this. The needed changes to the
applications are much bigger, since you always have to introduce
a configure check. So no quick workaround.

I would propose the following:
- Don't include rpc/netdb.h from <netdb.h>
- glibc stays with rpc/netdb.h as today with --disable-obsolete-rpc
- tirpc <rpc/rpcent.h> includes <rpc/netdb.h> on glibc systems, else
  it will provide the prototypes for rpcent itself.

I think that's the less painful solution for everybody and needs
only a small amount of code changes.

> There's still the separate question about NSS support for rpcent
> functions.  But we can address that after cleaning up the <netdb.h>
> situation.

I think NSS support for rpcent and <netdb.h> belong together. If
we remove rpcent* support from glibc, we don't need a rpc/netdb.h anymore
at all and should not include that from <netdb.h>.

  Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: Intention of "headers-not-in-tirpc"?

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

On 19 Mar 2016 13:31, Thorsten Kukuk wrote:
> On Fri, Mar 18, Roland McGrath wrote:
> > I think we should just start from scratch with working out a
> > "requirements" list.  That is, compared to stock glibc trunk today
> > configured with --enable-obsolete-rpc, exactly how do we want things
> > to be different in the end state.  That includes which things libc
> > should install and which things TI-RPC should install, and so forth.
> > Once we've reached consensus across both projects about that, then
> > we can dig into the implementation details of the libc build.
> 
> I think glibc should install as less as possible. In the end, I think
> it should only provide this functions, which requires NSS:
> 
> getrpcbyname.c, getrpcbyname_r.c, getrpcbynumber.c, getrpcbynumber_r.c,
> getrpcent.c, getrpcent_r.c
> 
> Since this is code, which only includes other glibc functions, but
> does not contain any RPC releated, and are all combined together in
> a single header file (rpc/netdb.h), I don't see there a real issue
> to seperate them.

why ?  these have 0 value outside of rpc related code, and libtirpc
already has implementations for them.  why should glibc continue to
waste space on them ?  the NSS indirection is not a compelling reason
imo.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Intention of "headers-not-in-tirpc"?+

[Thorsten Kukuk]

On Mon, Mar 21, Mike Frysinger wrote:

> why ?  these have 0 value outside of rpc related code, and libtirpc
> already has implementations for them.  why should glibc continue to
> waste space on them ?  the NSS indirection is not a compelling reason
> imo.

NSS is the reason and problem.
Why should every other library waste time and resources to re-implement
the glibc NSS interface and stay in sync (and this is needed to use the
same NSS modules) only that glibc saves a few bytes? And yes, we really 
speak only about a few bytes here.

It's ok that you are for removing it from glibc, but please, then provide
a doable alternative solution.


  Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: Intention of "headers-not-in-tirpc"?+

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

On 21 Mar 2016 07:41, Thorsten Kukuk wrote:
> On Mon, Mar 21, Mike Frysinger wrote:
> > why ?  these have 0 value outside of rpc related code, and libtirpc
> > already has implementations for them.  why should glibc continue to
> > waste space on them ?  the NSS indirection is not a compelling reason
> > imo.
> 
> NSS is the reason and problem.
> Why should every other library waste time and resources to re-implement
> the glibc NSS interface and stay in sync (and this is needed to use the
> same NSS modules) only that glibc saves a few bytes? And yes, we really 
> speak only about a few bytes here.
> 
> It's ok that you are for removing it from glibc, but please, then provide
> a doable alternative solution.

again, why is that our problem ?  and do you have any actual users here ?
we've already said RPC code is dead to us and it needs to go.  just because
we happened to provide a (rarely used) nss indirection is not interesting.
if you think it's a killer feature, then get it implemented in the library
that's replacing it (i.e. libtirpc).

alternative C libraries (e.g. uClibc/musl/dietlibc) have yet to see any
requests for these things.  can you cite equiv functionality in other OS's
either (like Solaris) ?
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Intention of "headers-not-in-tirpc"?+

[Zack Weinberg]

On Mon, Mar 21, 2016 at 7:10 PM, Mike Frysinger <vapier@gentoo.org> wrote:
> On 21 Mar 2016 07:41, Thorsten Kukuk wrote:
>>
>> NSS is the reason and problem.
>> Why should every other library waste time and resources to re-implement
>> the glibc NSS interface and stay in sync (and this is needed to use the
>> same NSS modules) only that glibc saves a few bytes? And yes, we really
>> speak only about a few bytes here.
>>
>> It's ok that you are for removing it from glibc, but please, then provide
>> a doable alternative solution.
>
> again, why is that our problem ?  and do you have any actual users here ?

I recall Thorsten cited some users (doing nontrivial things with rpc:
in /etc/nsswitch.conf, that is) 'way back in the first message that
started this conversation, or nearly so.

But regardless, it seems to me, Mike, that you are making the perfect
the enemy of the good.  Thorsten is the person doing all the work
here.  Thorsten says leaving the <rpc/netdb.h> interfaces in glibc
makes it easier to phase out all the *rest* of the sunrpc code in
glibc.  Can't that be good enough for now?  We can always come back to
this in a few release cycles, after the libtirpc transition for
everything else is complete.

zw

Re: Intention of "headers-not-in-tirpc"?+

[Thorsten Kukuk]

On Mon, Mar 21, Mike Frysinger wrote:

> again, why is that our problem ?  and do you have any actual users here ?

It would be really helpful, if you would read what I wrote and not only go
immeaditly in your "No" and "I don't care" mode.
As I already wrote, from customer configurations I know that a lot of
companies are using a central /etc/rpc database.

> we've already said RPC code is dead to us and it needs to go.  

Goodbye NFS? Sorry, but only because not everything is using RPC, it does
not mean it is dead. There are a few, but very important consumers of it.

> we happened to provide a (rarely used) nss indirection is not interesting.

Pleaes prove that it is rarely used.

> alternative C libraries (e.g. uClibc/musl/dietlibc) have yet to see any
> requests for these things.  can you cite equiv functionality in other OS's
> either (like Solaris) ?

Please don't compare such features with special, very small implementations
where the goal was to provide only the minimum basic libc functionality.
And of course Solaris has this. And FreeBSD has it, too. Looks like, you 
did never take a closer look at this topic.

Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)