Can getaddrinfo() be extended to return the record TTL

Can getaddrinfo() be extended to return the record TTL

[David Howells]

Hi Florian,

The Linux kernel has an upcall to get name -> address mappings for network
filesystems.  The userspace side (key.dns_resolver.c in keyutils) uses
getaddrinfo() to perform the lookup as this gives a greater range of sources
than, say, using res_send(), giving access to NIS, YP, LDAP, files or
whatever, but getaddrinfo() does not include any expiry information for the
information.

The userspace side of the upcall *should* be setting the expiry time on the
record - but it can't as the C library doesn't give us that (and, indeed, it's
not available from all sources).

The userspace upcall program *does* do it for AFSDB records for the afs
filesystem since those are looked up directly in the DNS and the TTL record is
thus available.


Anyway, looking at getaddrinfo(), it looks like the addrinfo struct should be
extensible.  Perhaps a flag, say AI_EXTENDED, could be provided that indicates
that is replaced with a larger struct that has additional fields such as
expiry time and a flag, AI_HAS_EXPIRY that indicates if the expiry time is
actually set.

This looks like it should work as the result records appear to be in a list
rather than being an array.

Another possible additional field would be a source indicator of some kind
that says where the data came from (e.g. 0=file, 1=dns, 2=yp, 3=ldap, ...).


If it's not possible to extend getaddrinfo() in this way, would it be possible
to provide a getaddrinfo2() that returns this information through an
additional parameter?

Thanks,
David

Re: Can getaddrinfo() be extended to return the record TTL

[Florian Weimer]

* David Howells:

> The Linux kernel has an upcall to get name -> address mappings for
> network filesystems.  The userspace side (key.dns_resolver.c in
> keyutils) uses getaddrinfo() to perform the lookup as this gives a
> greater range of sources than, say, using res_send(), giving access to
> NIS, YP, LDAP, files or whatever, but getaddrinfo() does not include
> any expiry information for the information.
>
> The userspace side of the upcall *should* be setting the expiry time
> on the record - but it can't as the C library doesn't give us that
> (and, indeed, it's not available from all sources).

Is this merely a correctness thing, or does this enable some
user-visible functionality?

> Anyway, looking at getaddrinfo(), it looks like the addrinfo struct should be
> extensible.  Perhaps a flag, say AI_EXTENDED, could be provided that indicates
> that is replaced with a larger struct that has additional fields such as
> expiry time and a flag, AI_HAS_EXPIRY that indicates if the expiry time is
> actually set.

I think we would add accessor functions for that, along with an internal
flag, so that gain some flexibility in the representation.

In previous discussions, it was suggested just to increase the size of
struct addrinfo, but this is not something we can do in glibc.

> This looks like it should work as the result records appear to be in a list
> rather than being an array.

Due to the existence of freeaddrinfo, we can use the pointer itself as a
hash table key as a last resort.

> Another possible additional field would be a source indicator of some kind
> that says where the data came from (e.g. 0=file, 1=dns, 2=yp, 3=ldap, ...).

Interesting.  It probably would have to be the name of the service
module (not a number).

Beyond the TTL, the security status was brought up as a suggestion.

> If it's not possible to extend getaddrinfo() in this way, would it be possible
> to provide a getaddrinfo2() that returns this information through an
> additional parameter?

That definitely would be a new ABI.

New ABIs have a lead time of around three years before productization.
Is this something you could work with?

Thanks,
Florian

Re: Can getaddrinfo() be extended to return the record TTL

[David Howells]

Florian Weimer <fweimer@redhat.com> wrote:

> > The userspace side of the upcall *should* be setting the expiry time
> > on the record - but it can't as the C library doesn't give us that
> > (and, indeed, it's not available from all sources).
> 
> Is this merely a correctness thing, or does this enable some
> user-visible functionality?

The kernel's DNS cache's ordinary records currently don't have their expiry
set as I don't have anything to set it to - with the result that the entries
in the cache never expire and have to be manually invalidated if you want them
to be reread.

I can, and probably should, set the default timeout to something reasonably
small and finite (say 10 mins) in the absence of such data, but it would be
better, I think, to set the TTL from the record (if available).

So, yes, this is actually causing a problem.

> > Another possible additional field would be a source indicator of some kind
> > that says where the data came from (e.g. 0=file, 1=dns, 2=yp, 3=ldap, ...).
> 
> Interesting.  It probably would have to be the name of the service
> module (not a number).

That would be fine.

> Beyond the TTL, the security status was brought up as a suggestion.

Interesting.  What does that convey?

> New ABIs have a lead time of around three years before productization.
> Is this something you could work with?

Well, I could put a better/configurable default in the code as a stopgap, so
there is a workaround.

Thanks,
David

Re: Can getaddrinfo() be extended to return the record TTL

[Carlos O'Donell]

On 8/2/19 3:33 PM, David Howells wrote:
> Well, I could put a better/configurable default in the code as a stopgap, so
> there is a workaround.

You'll need it.

For Fedora you're looking at 6-8 months before you see a release with the
fix in it (unless you use Rawhide or the fix is ABI neutral).

Then in RHEL you may have to wait around 3 years before there is a rebase to
an upstream branch.

Other distributions pickup the changes faster or slower depending on which
release you use.

-- 
Cheers,
Carlos.

Re: Can getaddrinfo() be extended to return the record TTL

[Eric Wong]

Florian Weimer <fweimer@redhat.com> wrote:
> * David Howells:
> > If it's not possible to extend getaddrinfo() in this way, would it be possible
> > to provide a getaddrinfo2() that returns this information through an
> > additional parameter?
> 
> That definitely would be a new ABI.
> 
> New ABIs have a lead time of around three years before productization.
> Is this something you could work with?

Fwiw, gaicb has an "int __glibc_reserved[5]" field; so it's
probably possible to extend the already-glibc-specific getaddrinfo_a(3)
to return the TTL.

I was looking into getting the TTL the other week for something
else, but settled on using a dedicated thread to call
getaddrinfo(3) every few seconds and publishing updates to other
threads via URCU.

Re: Can getaddrinfo() be extended to return the record TTL

[Florian Weimer]

* David Howells:

> Florian Weimer <fweimer@redhat.com> wrote:
>
>> > The userspace side of the upcall *should* be setting the expiry time
>> > on the record - but it can't as the C library doesn't give us that
>> > (and, indeed, it's not available from all sources).
>> 
>> Is this merely a correctness thing, or does this enable some
>> user-visible functionality?
>
> The kernel's DNS cache's ordinary records currently don't have their expiry
> set as I don't have anything to set it to - with the result that the entries
> in the cache never expire and have to be manually invalidated if you want them
> to be reread.
>
> I can, and probably should, set the default timeout to something reasonably
> small and finite (say 10 mins) in the absence of such data, but it would be
> better, I think, to set the TTL from the record (if available).
>
> So, yes, this is actually causing a problem.

I actually meant whether it enables some user-visible functionality,
like failover or service migration.

It's tricky to make this work in practice using DNS even if you know the
TTL, which is why I'm asking. 8-)

>> > Another possible additional field would be a source indicator of some kind
>> > that says where the data came from (e.g. 0=file, 1=dns, 2=yp, 3=ldap, ...).
>> 
>> Interesting.  It probably would have to be the name of the service
>> module (not a number).
>
> That would be fine.
>
>> Beyond the TTL, the security status was brought up as a suggestion.
>
> Interesting.  What does that convey?

Status of the AD bit in the reply, indicating whether the (trusted)
recursive DNS resolver has performed DNSSEC validation or not.

Thanks,
Florian

Re: Can getaddrinfo() be extended to return the record TTL

[Florian Weimer]

* Eric Wong:

> Florian Weimer <fweimer@redhat.com> wrote:
>> * David Howells:
>> > If it's not possible to extend getaddrinfo() in this way, would it be possible
>> > to provide a getaddrinfo2() that returns this information through an
>> > additional parameter?
>> 
>> That definitely would be a new ABI.
>> 
>> New ABIs have a lead time of around three years before productization.
>> Is this something you could work with?
>
> Fwiw, gaicb has an "int __glibc_reserved[5]" field; so it's
> probably possible to extend the already-glibc-specific getaddrinfo_a(3)
> to return the TTL.
>
> I was looking into getting the TTL the other week for something
> else, but settled on using a dedicated thread to call
> getaddrinfo(3) every few seconds and publishing updates to other
> threads via URCU.

You could perform the request directly over DNS and if you get back the
same addresses as via getaddrinfo, you can use the TTL you see on DNS.

It's not really nice as far as a programming interface goes, but it
should give you accurate TTL information in most cases.

Thanks,
Florian

Re: Can getaddrinfo() be extended to return the record TTL

[David Howells]

Florian Weimer <fweimer@redhat.com> wrote:

> > Florian Weimer <fweimer@redhat.com> wrote:
> >
> >> > The userspace side of the upcall *should* be setting the expiry time
> >> > on the record - but it can't as the C library doesn't give us that
> >> > (and, indeed, it's not available from all sources).
> >> 
> >> Is this merely a correctness thing, or does this enable some
> >> user-visible functionality?
> >
> > The kernel's DNS cache's ordinary records currently don't have their expiry
> > set as I don't have anything to set it to - with the result that the entries
> > in the cache never expire and have to be manually invalidated if you want them
> > to be reread.
> >
> > I can, and probably should, set the default timeout to something reasonably
> > small and finite (say 10 mins) in the absence of such data, but it would be
> > better, I think, to set the TTL from the record (if available).
> >
> > So, yes, this is actually causing a problem.
> 
> I actually meant whether it enables some user-visible functionality,
> like failover or service migration.
> 
> It's tricky to make this work in practice using DNS even if you know the
> TTL, which is why I'm asking. 8-)

Ah, sorry, I never got around to replying.  The problem is that the address
mapping records in the kernel used by a variety of network filesystems (NFS,
AFS, CIFS, etc.) don't expire because I don't expire because I don't have any
expiration information to apply.

David

Re: Can getaddrinfo() be extended to return the record TTL

[Jeff Layton]

On Fri, 2019-08-02 at 17:46 -0400, Carlos O'Donell wrote:
> On 8/2/19 3:33 PM, David Howells wrote:
> > Well, I could put a better/configurable default in the code as a stopgap, so
> > there is a workaround.
> 
> You'll need it.
> 
> For Fedora you're looking at 6-8 months before you see a release with the
> fix in it (unless you use Rawhide or the fix is ABI neutral).
> 
> Then in RHEL you may have to wait around 3 years before there is a rebase to
> an upstream branch.
> 
> Other distributions pickup the changes faster or slower depending on which
> release you use.
> 

Yes. We absolutely will need to deal with new kernel + old userspace
(and vice versa) for a long time, so that needs to be a consideration.

A reasonable default (60s? 5 mins? 12h?) and maybe a tunable of some
sort (sysctl or module param) for that would probably be fine.

You could also printk on the first downcall when userland didn't send a
TTL field, so that the admin knows that he might want to update his
userland eventually.
-- 
Jeff Layton <jlayton@redhat.com>