deprecated sunrpc and rpc/netdb.h

deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

Hi,

I looked at the "open issues" when using tirpc and deprecating
sunrpc in glibc.

Roland McGrath brought already up the topic of the getrpcent
functions and the problem with NSS (between, we have the same
problem with the publickey functions, but it looks like nobody
is really using them today anymore).

After fixing some more libtirpc issues (I hope Steve Dickson
will accept them in the next days), I ended up with only two
main problems, for which you need to touch the applications
source code:
1. link everything additional against libtirpc. That's not
   a big issue
2. getrpcent(), or better, rpc/netdb.h.

That's the real problem. glibc is installing a dummy rpc/netdb.h.
This breaks nearly every application using rpc out there and needs
code changes. Since glibc installs a dummy rpc/netdb.h, it's not
even possible for tirpc, to provide their own version.

If there is a decission, that the getrpcent family will stay
in glibc because of NSS, we should change the glibc code to
not deprecate this functions and to not install a dummy
netdb.h.

Else I have no idea how to solve the rpc/netdb.h issue.

Any thoughts? I can try to make a patch, which does not
deprecate the getrpcent functions and not replace rpc/netdb.h
with a dummy version.

  Thorsten


-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: deprecated sunrpc and rpc/netdb.h

[Andreas Schwab]

Thorsten Kukuk <kukuk@suse.de> writes:

> That's the real problem. glibc is installing a dummy rpc/netdb.h.
> This breaks nearly every application using rpc out there and needs
> code changes. Since glibc installs a dummy rpc/netdb.h, it's not
> even possible for tirpc, to provide their own version.
>
> If there is a decission, that the getrpcent family will stay
> in glibc because of NSS, we should change the glibc code to
> not deprecate this functions and to not install a dummy
> netdb.h.

IMHO the <rpc/netdb.h> interface should stay in glibc.  It is
independent from the rest of the RPC interface and has nothing to do
with sunrpc vs. tirpc.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: deprecated sunrpc and rpc/netdb.h

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

On 16 Mar 2016 11:41, Andreas Schwab wrote:
> Thorsten Kukuk <kukuk@suse.de> writes:
> > That's the real problem. glibc is installing a dummy rpc/netdb.h.
> > This breaks nearly every application using rpc out there and needs
> > code changes. Since glibc installs a dummy rpc/netdb.h, it's not
> > even possible for tirpc, to provide their own version.
> >
> > If there is a decission, that the getrpcent family will stay
> > in glibc because of NSS, we should change the glibc code to
> > not deprecate this functions and to not install a dummy
> > netdb.h.
> 
> IMHO the <rpc/netdb.h> interface should stay in glibc.  It is
> independent from the rest of the RPC interface and has nothing to do
> with sunrpc vs. tirpc.

netdb.h provides funcs to map RPC names<->ports and is only useful
to rpc related code.  if we purge sunrpc code from glibc (which we
want), then purging the netdb code too makes sense.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

On Wed, Mar 16, Mike Frysinger wrote:

> netdb.h provides funcs to map RPC names<->ports and is only useful
> to rpc related code.  if we purge sunrpc code from glibc (which we
> want), then purging the netdb code too makes sense.
> -mike

Correct, but how do you want to handle NSS?
There is a "rpc" entry in /etc/nsswitch.conf and it doesn't
make much sense to copy the whole NSS code into tirpc and
similar libraries.
And how do you want to solve the current problem of 
source code breakage with the dummy rpc/netdb.h?

  Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: deprecated sunrpc and rpc/netdb.h

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

On 16 Mar 2016 15:33, Thorsten Kukuk wrote:
> On Wed, Mar 16, Mike Frysinger wrote:
> > netdb.h provides funcs to map RPC names<->ports and is only useful
> > to rpc related code.  if we purge sunrpc code from glibc (which we
> > want), then purging the netdb code too makes sense.
> 
> Correct, but how do you want to handle NSS?
> There is a "rpc" entry in /etc/nsswitch.conf and it doesn't
> make much sense to copy the whole NSS code into tirpc and
> similar libraries.

drop it from glibc too and make it an ignored keyword.  whether tirpc
wants to support that indirection is up to that project.

> And how do you want to solve the current problem of 
> source code breakage with the dummy rpc/netdb.h?

i'm not familiar with the details you're referring to.  if glibc is
built with --disable-obsolete-rpc, then it shouldn't install this
header at all.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

On Wed, Mar 16, Mike Frysinger wrote:

> On 16 Mar 2016 15:33, Thorsten Kukuk wrote:
> > On Wed, Mar 16, Mike Frysinger wrote:
> > > netdb.h provides funcs to map RPC names<->ports and is only useful
> > > to rpc related code.  if we purge sunrpc code from glibc (which we
> > > want), then purging the netdb code too makes sense.
> > 
> > Correct, but how do you want to handle NSS?
> > There is a "rpc" entry in /etc/nsswitch.conf and it doesn't
> > make much sense to copy the whole NSS code into tirpc and
> > similar libraries.
> 
> drop it from glibc too and make it an ignored keyword.  whether tirpc
> wants to support that indirection is up to that project.

Sorry, but it is completly unrealistic, that an another
project duplicates the glibc NSS infrastructure to support
this. Don't forget, that another project would to closely
follow everything glibc is doing, since there is only one
NSS plugin, which has support for the glibc and tirpc part.
And I'm not sure if this will really work, if this plugins
are always loaded twice ...

> > And how do you want to solve the current problem of 
> > source code breakage with the dummy rpc/netdb.h?
> 
> i'm not familiar with the details you're referring to.  if glibc is
> built with --disable-obsolete-rpc, then it shouldn't install this
> header at all.

Correct, but this would break everything which includes netdb.h,
which is a glibc header file. I assume for this reason, the dummy
rpc/netdb.h was introduced, but it does not solve the problem.

If you don't want to touch every single piece of code there
outside (and this would be the only other option I currently
see, which would be completly incompatible to any other Unix), 
the best solution I currently see is, that glibc continues 
to ship rpc/netdb.h and the corresponding rpcent functions.
Especially, as this function don't contain any SunRPC code 
anymore, this are pure glibc macros.

  Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: deprecated sunrpc and rpc/netdb.h

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 3009 bytes --]

On 17 Mar 2016 11:48, Thorsten Kukuk wrote:
> On Wed, Mar 16, Mike Frysinger wrote:
> > On 16 Mar 2016 15:33, Thorsten Kukuk wrote:
> > > On Wed, Mar 16, Mike Frysinger wrote:
> > > > netdb.h provides funcs to map RPC names<->ports and is only useful
> > > > to rpc related code.  if we purge sunrpc code from glibc (which we
> > > > want), then purging the netdb code too makes sense.
> > > 
> > > Correct, but how do you want to handle NSS?
> > > There is a "rpc" entry in /etc/nsswitch.conf and it doesn't
> > > make much sense to copy the whole NSS code into tirpc and
> > > similar libraries.
> > 
> > drop it from glibc too and make it an ignored keyword.  whether tirpc
> > wants to support that indirection is up to that project.
> 
> Sorry, but it is completly unrealistic, that an another
> project duplicates the glibc NSS infrastructure to support
> this. Don't forget, that another project would to closely
> follow everything glibc is doing, since there is only one
> NSS plugin, which has support for the glibc and tirpc part.
> And I'm not sure if this will really work, if this plugins
> are always loaded twice ...

i'm not suggesting they duplicate it.  maybe they don't provide
alternatives at all.  why does that matter to us ?  it's not like
other projects are really clamoring for this.

> > > And how do you want to solve the current problem of 
> > > source code breakage with the dummy rpc/netdb.h?
> > 
> > i'm not familiar with the details you're referring to.  if glibc is
> > built with --disable-obsolete-rpc, then it shouldn't install this
> > header at all.
> 
> Correct, but this would break everything which includes netdb.h,
> which is a glibc header file. I assume for this reason, the dummy
> rpc/netdb.h was introduced, but it does not solve the problem.

the only reason rpc/netdb.h exists is because glibc imported rpc
code from sun.  getrpcent & friends is not in POSIX or any other
standard.  the fact that glibc installs it today making it a "glibc
header" is kind of a pointless statement -- by that logic, every
single rpc/ header glibc installs is a "glibc header".

> If you don't want to touch every single piece of code there
> outside (and this would be the only other option I currently
> see, which would be completly incompatible to any other Unix), 
> the best solution I currently see is, that glibc continues 
> to ship rpc/netdb.h and the corresponding rpcent functions.
> Especially, as this function don't contain any SunRPC code 
> anymore, this are pure glibc macros.

this only impacts projects that use sunrpc, and that's dwindling in
numbers today.  any project that wants to keep using sunrpc has to
be updated to detect/use tirpc *regardless* of rpc/netdb.h.  if the
tirpc project includes netdb.h, then it's a solved problem.

other than fixing glibc to not install rpc/netdb.h (or any other
rpc related header) when rpc is disabled, i'm not seeing what the
fuss is about.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

On Thu, Mar 17, Mike Frysinger wrote:

> other than fixing glibc to not install rpc/netdb.h (or any other
> rpc related header) when rpc is disabled, i'm not seeing what the
> fuss is about.

The fuss is about allowing a smooth transition from glibc
sunrpc to an external tirpc.
Why do you think all Linux distributions do compile glibc with
--enable-obsolete-rpc?
Do you really think that this will change if there is not
a smooth transition path?
Even if you think RPC is dead and not in wide use, I'm
not aware of any Linux distribution which is able to
work without RPC. And for this reason, a switch to something
else will never happen if it is a huge amount of work for
them and they will additional suffer from missing features 
afterwards.
Why should they do it?

  Thorsten



-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: deprecated sunrpc and rpc/netdb.h

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1587 bytes --]

On 17 Mar 2016 16:01, Thorsten Kukuk wrote:
> On Thu, Mar 17, Mike Frysinger wrote:
> > other than fixing glibc to not install rpc/netdb.h (or any other
> > rpc related header) when rpc is disabled, i'm not seeing what the
> > fuss is about.
> 
> The fuss is about allowing a smooth transition from glibc
> sunrpc to an external tirpc.
> Why do you think all Linux distributions do compile glibc with
> --enable-obsolete-rpc?
> Do you really think that this will change if there is not
> a smooth transition path?
> Even if you think RPC is dead and not in wide use, I'm
> not aware of any Linux distribution which is able to
> work without RPC. And for this reason, a switch to something
> else will never happen if it is a huge amount of work for
> them and they will additional suffer from missing features 
> afterwards.
> Why should they do it?

the smooth transition plan is for tirpc to provide the full API.  if
glibc turns off the exported symbols (which --disable-obsolete-rpc
does), then there is no "smooth" transition -- your code fails to link
entirely if you aren't supporting tirpc.  stub headers do not make it
any easier ... if anything, it makes it harder.

i'm fully aware of the level of pain seen in distros as i've sent/made
many patches to packages that are in Gentoo.

and yes, Gentoo works just fine w/out rpc.  this is also easy to find
out by using alternative C libraries like uClibc & musl which have long
allowed you to disable RPC (or never included it in the first place).

i think you're overstating the impact.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: deprecated sunrpc and rpc/netdb.h

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

On 17 Mar 2016 14:11, Mike Frysinger wrote:
> On 17 Mar 2016 16:01, Thorsten Kukuk wrote:
> > On Thu, Mar 17, Mike Frysinger wrote:
> > > other than fixing glibc to not install rpc/netdb.h (or any other
> > > rpc related header) when rpc is disabled, i'm not seeing what the
> > > fuss is about.
> > 
> > The fuss is about allowing a smooth transition from glibc
> > sunrpc to an external tirpc.
> > Why do you think all Linux distributions do compile glibc with
> > --enable-obsolete-rpc?
> > Do you really think that this will change if there is not
> > a smooth transition path?
> > Even if you think RPC is dead and not in wide use, I'm
> > not aware of any Linux distribution which is able to
> > work without RPC. And for this reason, a switch to something
> > else will never happen if it is a huge amount of work for
> > them and they will additional suffer from missing features 
> > afterwards.
> > Why should they do it?
> 
> the smooth transition plan is for tirpc to provide the full API.  if
> glibc turns off the exported symbols (which --disable-obsolete-rpc
> does), then there is no "smooth" transition -- your code fails to link
> entirely if you aren't supporting tirpc.  stub headers do not make it
> any easier ... if anything, it makes it harder.
> 
> i'm fully aware of the level of pain seen in distros as i've sent/made
> many patches to packages that are in Gentoo.
> 
> and yes, Gentoo works just fine w/out rpc.  this is also easy to find
> out by using alternative C libraries like uClibc & musl which have long
> allowed you to disable RPC (or never included it in the first place).
> 
> i think you're overstating the impact.

to clarify: the *only* reason Gentoo didn't leave RPC disabled in glibc
is purely because libtirpc isn't a full replacement.  as soon as that
situation changes, RPC is being disabled in Gentoo's glibc.

also: nothing is stopping you from *today* sending fixes to projects to
have them use libtirpc.  that'll work regardless of the glibc status.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

On Thu, Mar 17, Mike Frysinger wrote:

> > the smooth transition plan is for tirpc to provide the full API.  if

tirpc should do this. At least I'm not aware about any missing ones.

> > glibc turns off the exported symbols (which --disable-obsolete-rpc
> > does), then there is no "smooth" transition -- your code fails to link
> > entirely if you aren't supporting tirpc.  stub headers do not make it
> > any easier ... if anything, it makes it harder.

The problem is, that glibc neither fully disables rpc nor
that the NSS functionality can be provided by third party
implementation.
But as long as you refuse to dicuss any realistic solution 
to solve this, nothing will change.

> > i'm fully aware of the level of pain seen in distros as i've sent/made
> > many patches to packages that are in Gentoo.

Why didn't you fix the original problem?
With a correct solution of rpc/netdb.h in glibc, I only need
to link against libtirpc. I haven't found any package yet,
which needs other code changes. They may exist, but not in
the base system I'm using.

> to clarify: the *only* reason Gentoo didn't leave RPC disabled in glibc
> is purely because libtirpc isn't a full replacement.  as soon as that
> situation changes, RPC is being disabled in Gentoo's glibc.

I asked several times here on the mailing list, what this problems
should be. Until no, nobody did come up with any example, including
you.

> also: nothing is stopping you from *today* sending fixes to projects to
> have them use libtirpc.  that'll work regardless of the glibc status.

A more solution based discussion would be much more helpfull.

As I wrote: with the current rpc/netdb.h situation with glibc,
it is impossible to find an always working solution with the
same functionality as today.

  Thorsten


-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: deprecated sunrpc and rpc/netdb.h

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On 17 Mar 2016 19:41, Thorsten Kukuk wrote:
> On Thu, Mar 17, Mike Frysinger wrote:
> > > the smooth transition plan is for tirpc to provide the full API.  if
> 
> tirpc should do this. At least I'm not aware about any missing ones.

i've posted in the past (years ago at this point) about some of the parts
libtirpc was missing.

> > > glibc turns off the exported symbols (which --disable-obsolete-rpc
> > > does), then there is no "smooth" transition -- your code fails to link
> > > entirely if you aren't supporting tirpc.  stub headers do not make it
> > > any easier ... if anything, it makes it harder.
> 
> The problem is, that glibc neither fully disables rpc nor
> that the NSS functionality can be provided by third party
> implementation.
> But as long as you refuse to dicuss any realistic solution 
> to solve this, nothing will change.

i have no idea what you're talking about.  i've already stated a realistic
solution: glibc shouldn't install any rpc/ headers when it's disabled as
all headers/symbols are provided by libtirpc.  you on the other hand have
not said why that's a problem.  vague references to "it's not a smooth
transition" makes no sense as i already said.
-mike

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: deprecated sunrpc and rpc/netdb.h

[Joseph Myers]

On Thu, 17 Mar 2016, Mike Frysinger wrote:

> to clarify: the *only* reason Gentoo didn't leave RPC disabled in glibc
> is purely because libtirpc isn't a full replacement.  as soon as that
> situation changes, RPC is being disabled in Gentoo's glibc.

When it's a full replacement, we should also work out how to stop as much 
as possible of the deprecated code from being built at all for future 
architecture ports (so that for ports whose lowest symbol version is 
GLIBC_2.24 or whatever, the symbols don't exist in the glibc ABI even as 
compat symbols and --enable-obsolete-rpc isn't allowed at all).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: deprecated sunrpc and rpc/netdb.h

[Roland McGrath]

> When it's a full replacement, we should also work out how to stop as much 
> as possible of the deprecated code from being built at all for future 
> architecture ports (so that for ports whose lowest symbol version is 
> GLIBC_2.24 or whatever, the symbols don't exist in the glibc ABI even as 
> compat symbols and --enable-obsolete-rpc isn't allowed at all).

Heartily agreed!  I started to take a stab at this quite a while ago, but
gave up quickly.  I think it will be substantially simpler if we've first
gotten all YP/NIS/NIS+-related code out of libc entirely (when TI-RPC or
another project depending on it supplies libnss_{nis,nisplus,compat} and
libnsl so no libc configuration needs to do so even for ABI compatibility).


Thanks,
Roland

Re: deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

On Fri, Mar 18, Roland McGrath wrote:

> Heartily agreed!  I started to take a stab at this quite a while ago, but
> gave up quickly.  I think it will be substantially simpler if we've first
> gotten all YP/NIS/NIS+-related code out of libc entirely (when TI-RPC or
> another project depending on it supplies libnss_{nis,nisplus,compat} and
> libnsl so no libc configuration needs to do so even for ABI compatibility).

We have that already (they can be compiled against glibc sunrpc
or libtirpc):

libnsl replacing libnsl from glibc/nis:
git: https://github.com/thkukuk/libnsl
tar archives: http://www.linux-nis.org/download/libnsl/

libnss_compat replacing glibc/nis/nss_compat. There are no changes
or functional differences, it's only standalone:
git: https://github.com/thkukuk/libnss_compat
tar archives: http://www.linux-nis.org/download/libnss_compat/

libnss_nis replacing glibc/nis/nss_nis. This version is now
using the new NIS protocol if compiled and linked against
above libnsl and libtirpc:
git: https://github.com/thkukuk/libnss_nis
tar archives: http://www.linux-nis.org/download/libnss_nis/

libnss_nisplus replacing glibc/nis/nss_nisplus. This required
much more changes than for the rest and I thought. It should
work, but I couldn't test it against a NIS+ server.
git: https://github.com/thkukuk/libnss_nisplus
tar archives: http://www.linux-nis.org/download/libnss_nisplus/

  Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)

Re: deprecated sunrpc and rpc/netdb.h

[Roland McGrath]

> We have that already (they can be compiled against glibc sunrpc
> or libtirpc):

Great!  How production-ready do you (or really, distro maintainers) think
this is?

I propose that we start by disabling or removing this code from glibc
(before getting serious about sunrpc itself).  If the replacements are
sufficiently production-ready, we can just remove it all now.  If not,
then we'll have to start with a --disable-nis configure switch to
disable building it all.

Re: deprecated sunrpc and rpc/netdb.h

[Thorsten Kukuk]

On Fri, Mar 18, Roland McGrath wrote:

> > We have that already (they can be compiled against glibc sunrpc
> > or libtirpc):
> 
> Great!  How production-ready do you (or really, distro maintainers) think
> this is?

It is production ready, mostly it is based on the glibc code
and only adjusted to compile and link against libtirpc instead.
And it passes our NIS testsuite.

Only problem could be libnss_nisplus, since here bigger changes
were necessary and I cannot test them. But I doubt that there
is really somebody using NIS+ today.

  Thorsten

-- 
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)