public inbox for libc-help@sourceware.org
 help / color / mirror / Atom feed
* How to compile and use 'nscd'
@ 2022-07-24  7:57 이병욱
  2022-07-24 13:50 ` Carlos O'Donell
  0 siblings, 1 reply; 6+ messages in thread
From: 이병욱 @ 2022-07-24  7:57 UTC (permalink / raw)
  To: libc-help

I found nscd's dafault ttl should be 15 seconds
​
because CACHE_PRUNE_INTERVAL is declared as 15.
​
But it seems hard to modify it and compile nscd and use the customed nscd in my Centos 7.9.
​
I want to see how nscd works when CACHE_PRUNE_INTERVAL is 1 because I try to use DNS round robin at below environments.
​
- dns A record www.server.com with 10.1.1.1, 10.1.1.2 with dns ttl 1 seconds.
- client server with customed nscd, send a request to www.server.com
- two backend server indicated by 10.1.1.1, 10.1.1.2
​
Is it possible to use the customed nscd?
​
Sincerely,
Nimdrak

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: How to compile and use 'nscd'
  2022-07-24  7:57 How to compile and use 'nscd' 이병욱
@ 2022-07-24 13:50 ` Carlos O'Donell
  2022-07-24 14:15   ` 이병욱
  0 siblings, 1 reply; 6+ messages in thread
From: Carlos O'Donell @ 2022-07-24 13:50 UTC (permalink / raw)
  To: 이병욱; +Cc: libc-help

On Sun, Jul 24, 2022 at 3:57 AM 이병욱 via Libc-help
<libc-help@sourceware.org> wrote:
>
> I found nscd's dafault ttl should be 15 seconds
>
> because CACHE_PRUNE_INTERVAL is declared as 15.
>
> But it seems hard to modify it and compile nscd and use the customed nscd in my Centos 7.9.
>
> I want to see how nscd works when CACHE_PRUNE_INTERVAL is 1 because I try to use DNS round robin at below environments.
>
> - dns A record www.server.com with 10.1.1.1, 10.1.1.2 with dns ttl 1 seconds.
> - client server with customed nscd, send a request to www.server.com
> - two backend server indicated by 10.1.1.1, 10.1.1.2
>
> Is it possible to use the customed nscd?

It is technically possible to use a customized nscd. However, using a
customized nscd can impact your support through your distribution.

The design goals behind nscd are to cache results locally in as simple
a manner as possible.

The internal hosts cache in nscd is aware of TTL, but the CentOS 7 era
nscd may still have some issues regarding refreshing values based on
TTL.

The value of CACHE_PRUNE_INTERVAL is defined in nscd/nscd.h, and
rebuilding an nscd for your distribution is most easily accomplished
by rebuilding the distribution rpm with a patch added.

The CACHE_PRUNE_INTERVAL drives the loop that prunes the cache, but
other effects prune the cache also, which for the hosts cache should
include the entries ttl.

I suggest evaluating other caching options before changing nscd
though, and you may find it beneficial to look at sssd.

Cheers,
Carlos.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: How to compile and use 'nscd'
  2022-07-24 13:50 ` Carlos O'Donell
@ 2022-07-24 14:15   ` 이병욱
  2022-07-29 16:41     ` Carlos O'Donell
  0 siblings, 1 reply; 6+ messages in thread
From: 이병욱 @ 2022-07-24 14:15 UTC (permalink / raw)
  To: Carlos O'Donell; +Cc: libc-help

Thank you for your reply.
 
If you know, would you tell me why `CACHE_PRUNE_INTERVAL` is defined as 15? It is hard to find the reason anywhere.
 
If it was intended for mitigating the load of the client server with nscd, I think it is a little large, considering the current server performance.
 
I found at JVM environment (JVM has its own DNS caching), It seems that TTL value close to 0, doesn't have an impact to a client server performance, to an extent.
 
I think if we choose a DNS round robin for distributing the traffic load, it is inevitable to assign a DNS ttl close to 0. Otherwise, it can make biased traffic to servers. 
 
Although I make a A record TTL value below 15, this small TTL value become meaningless because the client make a dns query at least every 15 seconds, which is `CACHE_PRUNE_INTERVAL`.
 
Is there a design philosophy about `15`?
 
 
-----Original Message-----
From: "Carlos O'Donell"<carlos@systemhalted.org>
To: "이병욱"<nimdrak@naver.com>;
Cc: "libc-help"<libc-help@sourceware.org>;
Sent: 2022-07-24 (일) 22:50:37 (GMT+09:00)
Subject: Re: How to compile and use 'nscd'
 
On Sun, Jul 24, 2022 at 3:57 AM 이병욱 via Libc-help
<libc-help@sourceware.org> wrote:
>
> I found nscd's dafault ttl should be 15 seconds
>
> because CACHE_PRUNE_INTERVAL is declared as 15.
>
> But it seems hard to modify it and compile nscd and use the customed nscd in my Centos 7.9.
>
> I want to see how nscd works when CACHE_PRUNE_INTERVAL is 1 because I try to use DNS round robin at below environments.
>
> - dns A record www.server.com with 10.1.1.1, 10.1.1.2 with dns ttl 1 seconds.
> - client server with customed nscd, send a request to www.server.com
> - two backend server indicated by 10.1.1.1, 10.1.1.2
>
> Is it possible to use the customed nscd?

It is technically possible to use a customized nscd. However, using a
customized nscd can impact your support through your distribution.

The design goals behind nscd are to cache results locally in as simple
a manner as possible.

The internal hosts cache in nscd is aware of TTL, but the CentOS 7 era
nscd may still have some issues regarding refreshing values based on
TTL.

The value of CACHE_PRUNE_INTERVAL is defined in nscd/nscd.h, and
rebuilding an nscd for your distribution is most easily accomplished
by rebuilding the distribution rpm with a patch added.

The CACHE_PRUNE_INTERVAL drives the loop that prunes the cache, but
other effects prune the cache also, which for the hosts cache should
include the entries ttl.

I suggest evaluating other caching options before changing nscd
though, and you may find it beneficial to look at sssd.

Cheers,
Carlos.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: How to compile and use 'nscd'
  2022-07-24 14:15   ` 이병욱
@ 2022-07-29 16:41     ` Carlos O'Donell
  2022-07-31  9:18       ` 이병욱
  2022-07-31  9:20       ` 이병욱
  0 siblings, 2 replies; 6+ messages in thread
From: Carlos O'Donell @ 2022-07-29 16:41 UTC (permalink / raw)
  To: 이병욱, Carlos O'Donell; +Cc: libc-help

On 7/24/22 10:15, 이병욱 via Libc-help wrote:
> Thank you for your reply.
> 
> If you know, would you tell me why `CACHE_PRUNE_INTERVAL` is defined
> as 15? It is hard to find the reason anywhere.

I don't think there is a strong justification for 15s.

It could be added as a tunable to nscd.conf.

> If it was intended for mitigating the load of the client server with
> nscd, I think it is a little large, considering the current server
> performance.

It limits the CPU usage on the client by pruning the cache at certain
intervals.

The cache entries themselves, depending on the cache, will have TTL
information that can time out on a shorter interval.
 
> I found at JVM environment (JVM has its own DNS caching), It seems
> that TTL value close to 0, doesn't have an impact to a client server
> performance, to an extent.
> 
> I think if we choose a DNS round robin for distributing the traffic
> load, it is inevitable to assign a DNS ttl close to 0. Otherwise, it
> can make biased traffic to servers.
> 
> Although I make a A record TTL value below 15, this small TTL value
> become meaningless because the client make a dns query at least every
> 15 seconds, which is `CACHE_PRUNE_INTERVAL`.

Why does it become meaningless?

The CACHE_PRUNE_INTERVAL is the interval in which the pruning thread looks
for entries with expired lifetime. It is not the interval at which entries
are expired, the actual expiry is determined by TTL values.
 
> Is there a design philosophy about `15`?

None that we have documented.

-- 
Cheers,
Carlos.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: How to compile and use 'nscd'
  2022-07-29 16:41     ` Carlos O'Donell
@ 2022-07-31  9:18       ` 이병욱
  2022-07-31  9:20       ` 이병욱
  1 sibling, 0 replies; 6+ messages in thread
From: 이병욱 @ 2022-07-31  9:18 UTC (permalink / raw)
  To: Carlos O'Donell, Carlos O'Donell; +Cc: libc-help

Thank you for your replies!

Then how about making `CACHE_PRUNE_INTERVAL` tunable at nscd.conf?
 
I think it is too large considering the current server specification.
 
It seems `CACHE_PRUNE_INTERVAL` first appeared at glibc-2.0.99 which was before 2000. It passed about 20 years.
 
I think this large `CACHE_PRUNE_INTERVAL` can make dns caching meaningless, especially when using dns round robin.
 
At dns round robin with CACHE_PRUNE_INTERVAL, it makes unequally distributed traffic to servers. It makes inefficient servers usage.
(I did some experiments for checking this, I posted this https://stackoverflow.com/questions/73181902/could-we-use-dns-round-robin-with-nscds-dns-cache)
 
How about making `CACHE_PRUNE_INTERVAL` tunable at nscd.conf or making it smaller? 

Maybe it mitigate the load of CPU at 2000. But If `CACHE_PRUNE_INTERVAL` become smaller, I think it is not that much burdened at current CPU.
(I didn't do this kind of experiments. But If you think it is needed, I will did it.) 

I think it is more reasonable for user to be able to tune `CACHE_PRUNE_INTERVAL` considering user's machine enviroment.
 
Sincerely,
Nimdrak.
 
 
-----Original Message-----
From: "Carlos O'Donell"<carlos@redhat.com>
To: "이병욱"<nimdrak@naver.com>; "Carlos O'Donell"<carlos@systemhalted.org>;
Cc: "libc-help"<libc-help@sourceware.org>;
Sent: 2022-07-30 (토) 01:41:48 (GMT+09:00)
Subject: Re: How to compile and use 'nscd'
 
On 7/24/22 10:15, 이병욱 via Libc-help wrote:
> Thank you for your reply.
>
> If you know, would you tell me why `CACHE_PRUNE_INTERVAL` is defined
> as 15? It is hard to find the reason anywhere.

I don't think there is a strong justification for 15s.

It could be added as a tunable to nscd.conf.

> If it was intended for mitigating the load of the client server with
> nscd, I think it is a little large, considering the current server
> performance.

It limits the CPU usage on the client by pruning the cache at certain
intervals.

The cache entries themselves, depending on the cache, will have TTL
information that can time out on a shorter interval.

> I found at JVM environment (JVM has its own DNS caching), It seems
> that TTL value close to 0, doesn't have an impact to a client server
> performance, to an extent.
>
> I think if we choose a DNS round robin for distributing the traffic
> load, it is inevitable to assign a DNS ttl close to 0. Otherwise, it
> can make biased traffic to servers.
>
> Although I make a A record TTL value below 15, this small TTL value
> become meaningless because the client make a dns query at least every
> 15 seconds, which is `CACHE_PRUNE_INTERVAL`.

Why does it become meaningless?

The CACHE_PRUNE_INTERVAL is the interval in which the pruning thread looks
for entries with expired lifetime. It is not the interval at which entries
are expired, the actual expiry is determined by TTL values.

> Is there a design philosophy about `15`?

None that we have documented.

--
Cheers,
Carlos.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: How to compile and use 'nscd' 
  2022-07-29 16:41     ` Carlos O'Donell
  2022-07-31  9:18       ` 이병욱
@ 2022-07-31  9:20       ` 이병욱
  1 sibling, 0 replies; 6+ messages in thread
From: 이병욱 @ 2022-07-31  9:20 UTC (permalink / raw)
  To: Carlos O'Donell, Carlos O'Donell; +Cc: libc-help

Thank you for your replies!

Then how about making `CACHE_PRUNE_INTERVAL` tunable at nscd.conf?
 
I think it is too large considering the current server specification.
 
It seems `CACHE_PRUNE_INTERVAL` first appeared at glibc-2.0.99 which was before 2000. It passed about 20 years.
 
I think this large `CACHE_PRUNE_INTERVAL` can make DNS caching meaningless, especially when using dns round robin.
 
At DNS round robin with CACHE_PRUNE_INTERVAL, it makes unequally distributed traffic to servers. It makes inefficient servers usage.
(I did some experiments for checking this, I posted this https://stackoverflow.com/questions/73181902/could-we-use-dns-round-robin-with-nscds-dns-cache)
 
How about making `CACHE_PRUNE_INTERVAL` tunable at nscd.conf or making it smaller? 

Maybe it mitigates the load of CPU at 2000. But If `CACHE_PRUNE_INTERVAL` become smaller, I think it is not that much burdened at current CPU.
(I didn't do this kind of experiments. But If you think it is needed, I will do it.) 

I think it is more reasonable for user to be able to tune `CACHE_PRUNE_INTERVAL` considering user's machine environment.
 
Sincerely,
Nimdrak.
-----Original Message-----
From: "Carlos O'Donell"<carlos@redhat.com>
To: "이병욱"<nimdrak@naver.com>; "Carlos O'Donell"<carlos@systemhalted.org>;
Cc: "libc-help"<libc-help@sourceware.org>;
Sent: 2022-07-30 (토) 01:41:48 (GMT+09:00)
Subject: Re: How to compile and use 'nscd'
 
On 7/24/22 10:15, 이병욱 via Libc-help wrote:
> Thank you for your reply.
>
> If you know, would you tell me why `CACHE_PRUNE_INTERVAL` is defined
> as 15? It is hard to find the reason anywhere.

I don't think there is a strong justification for 15s.

It could be added as a tunable to nscd.conf.

> If it was intended for mitigating the load of the client server with
> nscd, I think it is a little large, considering the current server
> performance.

It limits the CPU usage on the client by pruning the cache at certain
intervals.

The cache entries themselves, depending on the cache, will have TTL
information that can time out on a shorter interval.

> I found at JVM environment (JVM has its own DNS caching), It seems
> that TTL value close to 0, doesn't have an impact to a client server
> performance, to an extent.
>
> I think if we choose a DNS round robin for distributing the traffic
> load, it is inevitable to assign a DNS ttl close to 0. Otherwise, it
> can make biased traffic to servers.
>
> Although I make a A record TTL value below 15, this small TTL value
> become meaningless because the client make a dns query at least every
> 15 seconds, which is `CACHE_PRUNE_INTERVAL`.

Why does it become meaningless?

The CACHE_PRUNE_INTERVAL is the interval in which the pruning thread looks
for entries with expired lifetime. It is not the interval at which entries
are expired, the actual expiry is determined by TTL values.

> Is there a design philosophy about `15`?

None that we have documented.

--
Cheers,
Carlos.


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-07-31  9:20 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-24  7:57 How to compile and use 'nscd' 이병욱
2022-07-24 13:50 ` Carlos O'Donell
2022-07-24 14:15   ` 이병욱
2022-07-29 16:41     ` Carlos O'Donell
2022-07-31  9:18       ` 이병욱
2022-07-31  9:20       ` 이병욱

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).