TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Cary Lewis]

These constants are usually defined in  netinet/tcp.h but are not defined
anywhere in cygwin.

As a result there does not seem to be a way to enable the socket level keep
alives in cygwin.

One result of this is curl's --keepalive-time option does not work, but it
would affect any attempt to enable socket keep alives.

Windows' sockets do support keep alives, and indeed the win32 version of
curl works correctly.

Can someone explain why the TCP keep alive constants are not defined?

Is it possible to add this functionality?

Where in the cygwin source code does the actual posix socket to winsock
translation take place?

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Ken Brown]

On 6/7/2020 9:20 AM, Cary Lewis via Cygwin wrote:
> These constants are usually defined in  netinet/tcp.h but are not defined
> anywhere in cygwin.
 >
> As a result there does not seem to be a way to enable the socket level keep
> alives in cygwin.
> 
> One result of this is curl's --keepalive-time option does not work, but it
> would affect any attempt to enable socket keep alives.
> 
> Windows' sockets do support keep alives, and indeed the win32 version of
> curl works correctly.
> 
> Can someone explain why the TCP keep alive constants are not defined

I'm not an expert on sockets, but I'll just make a couple of comments until 
Corinna has a chance to give a definitive reply.

Cygwin's netinet/tcp.h was derived from the FreeBSD version of the file dated 
2000/01/09.  It has hardly been touched since it was imported into Cygwin in 
2002.  FreeBSD didn't add keepalive support until 2012.

> Is it possible to add this functionality?

If you're asking someone else to do it, the question is whether one of the 
Cygwin developers has the time and expertise to do it.  If you're volunteering 
to do it, patches are always welcome.

> Where in the cygwin source code does the actual posix socket to winsock
> translation take place?

Cygwin's socket support is in the fhandler_socket* files.

Ken

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Cary Lewis]

Thanks very much for the reply - I will wait for Corinna's take on this as
well - the patch shouldn't be too hard.

In fact the source code for curl in the lib/connect.c file has code to
enable the winsock options, so hopefully that code could be helpful. I
imagine there must be a translation between the POSIX sockets and winsock
in the fhandler_socket.

I will attempt to download the cygwin source code - I've never built it, so
I don't know how hard it will be.

I am surprised that no one else has ever needed the low level TCP keep
alives to work.



On Sun, Jun 7, 2020 at 2:21 PM Ken Brown via Cygwin <cygwin@cygwin.com>
wrote:

> On 6/7/2020 9:20 AM, Cary Lewis via Cygwin wrote:
> > These constants are usually defined in  netinet/tcp.h but are not defined
> > anywhere in cygwin.
>  >
> > As a result there does not seem to be a way to enable the socket level
> keep
> > alives in cygwin.
> >
> > One result of this is curl's --keepalive-time option does not work, but
> it
> > would affect any attempt to enable socket keep alives.
> >
> > Windows' sockets do support keep alives, and indeed the win32 version of
> > curl works correctly.
> >
> > Can someone explain why the TCP keep alive constants are not defined
>
> I'm not an expert on sockets, but I'll just make a couple of comments
> until
> Corinna has a chance to give a definitive reply.
>
> Cygwin's netinet/tcp.h was derived from the FreeBSD version of the file
> dated
> 2000/01/09.  It has hardly been touched since it was imported into Cygwin
> in
> 2002.  FreeBSD didn't add keepalive support until 2012.
>
> > Is it possible to add this functionality?
>
> If you're asking someone else to do it, the question is whether one of the
> Cygwin developers has the time and expertise to do it.  If you're
> volunteering
> to do it, patches are always welcome.
>
> > Where in the cygwin source code does the actual posix socket to winsock
> > translation take place?
>
> Cygwin's socket support is in the fhandler_socket* files.
>
> Ken
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Corinna Vinschen]

On Jun  7 14:56, Cary Lewis via Cygwin wrote:
> Thanks very much for the reply - I will wait for Corinna's take on this as
> well - the patch shouldn't be too hard.

It's not that easy, but not too hard either:

- Windows 10 1709 and later actually support the TCP_KEEPIDLE,
  TCP_KEEPINTVL and TCP_KEEPCNT options natively, so all is fine here.

- Older OS versions require the use of a single call
  WSAIoctl(SIO_KEEPALIVE_VALS), while TCP_KEEPIDLE, TCP_KEEPINTVL and
  TCP_KEEPCNT are three distinct calls, with TCP_KEEPCNT being no-op on
  pre W10 1709.

> In fact the source code for curl in the lib/connect.c file has code to
> enable the winsock options, so hopefully that code could be helpful. I

But, here's the question:

Even if neither one of SIO_KEEPALIVE_VALS, TCP_KEEPIDLE, TCP_KEEPINTVL
and TCP_KEEPCNT are available, curl *still* uses SO_KEEPALIVE to enable
TCP keep-alive.  Given TCP keep-alive has default settings on Windows,
too (2 hours, 1 sec, 10 times), and given SO_KEEPALIVE is supported on
Cygwin as well, why isn't that sufficient?

Keep in mind that none of the above WSAIoctl/setsockopt options are
portable, only SO_KEEPALIVE is.  So why should that work on, say,
Solaris, but not on Windows?


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Cary Lewis]

Thanks for the reply. The answer to your question is that the 2 hour keep
alive was not sufficient for a particular use case I encountered.

I was trying to use curl under cygwin to access a very slow REST endpoint
that was taking up to 8 minutes to generate download before any data flowed
back to the client. This caused the server to abort the socket.

Accessing the endpoint in chrome or firefox revealed that they set a
keepalive to 45 seconds, which kept the server happy.

Attempting to set --keepalive-time=45 in cygwin's curl didn't work, and
wireshark revealed that no keepalives were being sent.

I will attempt to patch cygwin, I got the build to work. Can you point me
in the right direction, in terms of where the socket calls get mapped to
the winsock calls?

On a separate note, what are your thoughts on the new windows WSL2 release?
Do you think this will have long term implications for cygwin?


On Tue, Jun 30, 2020 at 7:15 AM Corinna Vinschen <corinna-cygwin@cygwin.com>
wrote:

> On Jun  7 14:56, Cary Lewis via Cygwin wrote:
> > Thanks very much for the reply - I will wait for Corinna's take on this
> as
> > well - the patch shouldn't be too hard.
>
> It's not that easy, but not too hard either:
>
> - Windows 10 1709 and later actually support the TCP_KEEPIDLE,
>   TCP_KEEPINTVL and TCP_KEEPCNT options natively, so all is fine here.
>
> - Older OS versions require the use of a single call
>   WSAIoctl(SIO_KEEPALIVE_VALS), while TCP_KEEPIDLE, TCP_KEEPINTVL and
>   TCP_KEEPCNT are three distinct calls, with TCP_KEEPCNT being no-op on
>   pre W10 1709.
>
> > In fact the source code for curl in the lib/connect.c file has code to
> > enable the winsock options, so hopefully that code could be helpful. I
>
> But, here's the question:
>
> Even if neither one of SIO_KEEPALIVE_VALS, TCP_KEEPIDLE, TCP_KEEPINTVL
> and TCP_KEEPCNT are available, curl *still* uses SO_KEEPALIVE to enable
> TCP keep-alive.  Given TCP keep-alive has default settings on Windows,
> too (2 hours, 1 sec, 10 times), and given SO_KEEPALIVE is supported on
> Cygwin as well, why isn't that sufficient?
>
> Keep in mind that none of the above WSAIoctl/setsockopt options are
> portable, only SO_KEEPALIVE is.  So why should that work on, say,
> Solaris, but not on Windows?
>
>
> Corinna
>
> --
> Corinna Vinschen
> Cygwin Maintainer
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Corinna Vinschen]

On Jun 30 09:46, Cary Lewis via Cygwin wrote:
> Thanks for the reply. The answer to your question is that the 2 hour keep
> alive was not sufficient for a particular use case I encountered.
> 
> I was trying to use curl under cygwin to access a very slow REST endpoint
> that was taking up to 8 minutes to generate download before any data flowed
> back to the client. This caused the server to abort the socket.
> 
> Accessing the endpoint in chrome or firefox revealed that they set a
> keepalive to 45 seconds, which kept the server happy.
> 
> Attempting to set --keepalive-time=45 in cygwin's curl didn't work, and
> wireshark revealed that no keepalives were being sent.
> 
> I will attempt to patch cygwin, I got the build to work. Can you point me
> in the right direction, in terms of where the socket calls get mapped to
> the winsock calls?

Actually, while I'm usually happy to take contributions, you don't have
to dig into that yourself.  I already have a few local patches in the
loop changing some of the affected code.  I have a good idea what's
required to add the keep-alive socket options to that code, so just lay
back and stay tuned for now.

> On a separate note, what are your thoughts on the new windows WSL2 release?
> Do you think this will have long term implications for cygwin?

Same as WSL, I guess ¯\_(ツ)_/¯


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Corinna Vinschen]

On Jun 30 18:53, Corinna Vinschen wrote:
> On Jun 30 09:46, Cary Lewis via Cygwin wrote:
> > Thanks for the reply. The answer to your question is that the 2 hour keep
> > alive was not sufficient for a particular use case I encountered.
> > 
> > I was trying to use curl under cygwin to access a very slow REST endpoint
> > that was taking up to 8 minutes to generate download before any data flowed
> > back to the client. This caused the server to abort the socket.
> > 
> > Accessing the endpoint in chrome or firefox revealed that they set a
> > keepalive to 45 seconds, which kept the server happy.
> > 
> > Attempting to set --keepalive-time=45 in cygwin's curl didn't work, and
> > wireshark revealed that no keepalives were being sent.
> > 
> > I will attempt to patch cygwin, I got the build to work. Can you point me
> > in the right direction, in terms of where the socket calls get mapped to
> > the winsock calls?
> 
> Actually, while I'm usually happy to take contributions, you don't have
> to dig into that yourself.  I already have a few local patches in the
> loop changing some of the affected code.  I have a good idea what's
> required to add the keep-alive socket options to that code, so just lay
> back and stay tuned for now.

Ok, so I added support for a couple more IPPROTO_TCP socket options.
First of all I fixed TCP_MAXSEG which was using the BSD value, rather
than the WinSock value.  Then I added TCP_FASTOPEN, TCP_KEEPIDLE,
TCP_KEEPCNT, TCP_KEEPINTVL, TCP_QUICKACK and TCP_USER_TIMEOUT:

- TCP_FASTOPEN is supported since W10 1607, it's just faked on older
  systems.

- TCP_KEEPIDLE, TCP_KEEPCNT, TCP_KEEPINTVL are using the options
  of the same name since W10 1709, WSAIoctl(SIO_KEEPALIVE_VALS)
  on older systems.

  But here's a problem: Older systems didn't allow to change
  TCP_KEEPCNT.  It is always fixed to 10.  Mulling over that problem in
  the shower, I came up with the following solution:

  The max keep-alive timeout is TCP_KEEPIDLE + TCP_KEEPCNT * TCP_KEEPINTVL.
  This should stay the same from a user space perspective.  So the current
  code tweaks the TCP_KEEPINTVL given to WinSock so that

  TCP_KEEPCNT * user space TCP_KEEPINTVL == 10 * WinSock TCP_KEEPINTVL

  Example: user space TCP_KEEPCNT 4, TCP_KEEPINTVL 5   (4 * 5 == 20)
  ==>      WinSock    TCP_KEEPCNT 10, TCP_KEEPINTVL 2  (10 * 2 == 20)

  I hope that makes sense.

- TCP_USER_TIMEOUT is supported with msec granularity since W10 1607
  (called TCP_MAXRTMS), with 1 secs granularity on older systems
  (called TCP_MAXRT).  Use the latter on older systems under the expected
  loss of precision.

- TCP_QUICKACK is supposedly supported on Windows as a socket option
  but it's still not clear if the net got that right so far.  However,
  there's WSAIoctl(SIO_TCP_SET_ACK_FREQUENCY) doing the same.

I uploaded developer snapshots to https://cygwin.com/snapshots/,
please test.

For testing, you'll need at least the DLL, plus the changed headers
cygwin/socket.h and netinet/tcp.h from the complete tar file
cygwin-20200701.tar.xz. Or, just take the DLL and fetch the headers
right from the git repo.


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Cary Lewis]

That's amazing, thanks. We'll have to try to recompile curl under cygwin to
confirm that it keeps up the constants, and then does the right posix calls.

I will grab the files, and try to have this tested and report back to you.

Take care,

Cary Lewis

On Wed, Jul 1, 2020 at 3:51 PM Corinna Vinschen <corinna-cygwin@cygwin.com>
wrote:

> On Jun 30 18:53, Corinna Vinschen wrote:
> > On Jun 30 09:46, Cary Lewis via Cygwin wrote:
> > > Thanks for the reply. The answer to your question is that the 2 hour
> keep
> > > alive was not sufficient for a particular use case I encountered.
> > >
> > > I was trying to use curl under cygwin to access a very slow REST
> endpoint
> > > that was taking up to 8 minutes to generate download before any data
> flowed
> > > back to the client. This caused the server to abort the socket.
> > >
> > > Accessing the endpoint in chrome or firefox revealed that they set a
> > > keepalive to 45 seconds, which kept the server happy.
> > >
> > > Attempting to set --keepalive-time=45 in cygwin's curl didn't work, and
> > > wireshark revealed that no keepalives were being sent.
> > >
> > > I will attempt to patch cygwin, I got the build to work. Can you point
> me
> > > in the right direction, in terms of where the socket calls get mapped
> to
> > > the winsock calls?
> >
> > Actually, while I'm usually happy to take contributions, you don't have
> > to dig into that yourself.  I already have a few local patches in the
> > loop changing some of the affected code.  I have a good idea what's
> > required to add the keep-alive socket options to that code, so just lay
> > back and stay tuned for now.
>
> Ok, so I added support for a couple more IPPROTO_TCP socket options.
> First of all I fixed TCP_MAXSEG which was using the BSD value, rather
> than the WinSock value.  Then I added TCP_FASTOPEN, TCP_KEEPIDLE,
> TCP_KEEPCNT, TCP_KEEPINTVL, TCP_QUICKACK and TCP_USER_TIMEOUT:
>
> - TCP_FASTOPEN is supported since W10 1607, it's just faked on older
>   systems.
>
> - TCP_KEEPIDLE, TCP_KEEPCNT, TCP_KEEPINTVL are using the options
>   of the same name since W10 1709, WSAIoctl(SIO_KEEPALIVE_VALS)
>   on older systems.
>
>   But here's a problem: Older systems didn't allow to change
>   TCP_KEEPCNT.  It is always fixed to 10.  Mulling over that problem in
>   the shower, I came up with the following solution:
>
>   The max keep-alive timeout is TCP_KEEPIDLE + TCP_KEEPCNT * TCP_KEEPINTVL.
>   This should stay the same from a user space perspective.  So the current
>   code tweaks the TCP_KEEPINTVL given to WinSock so that
>
>   TCP_KEEPCNT * user space TCP_KEEPINTVL == 10 * WinSock TCP_KEEPINTVL
>
>   Example: user space TCP_KEEPCNT 4, TCP_KEEPINTVL 5   (4 * 5 == 20)
>   ==>      WinSock    TCP_KEEPCNT 10, TCP_KEEPINTVL 2  (10 * 2 == 20)
>
>   I hope that makes sense.
>
> - TCP_USER_TIMEOUT is supported with msec granularity since W10 1607
>   (called TCP_MAXRTMS), with 1 secs granularity on older systems
>   (called TCP_MAXRT).  Use the latter on older systems under the expected
>   loss of precision.
>
> - TCP_QUICKACK is supposedly supported on Windows as a socket option
>   but it's still not clear if the net got that right so far.  However,
>   there's WSAIoctl(SIO_TCP_SET_ACK_FREQUENCY) doing the same.
>
> I uploaded developer snapshots to https://cygwin.com/snapshots/,
> please test.
>
> For testing, you'll need at least the DLL, plus the changed headers
> cygwin/socket.h and netinet/tcp.h from the complete tar file
> cygwin-20200701.tar.xz. Or, just take the DLL and fetch the headers
> right from the git repo.
>
>
> Thanks,
> Corinna
>
> --
> Corinna Vinschen
> Cygwin Maintainer
> --
> Problem reports:      https://cygwin.com/problems.html
> FAQ:                  https://cygwin.com/faq/
> Documentation:        https://cygwin.com/docs.html
> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>

Re: TCP_KEEPINVTL and TCP_KEEPIDLE - Socket Keep Alives not working

[Cary Lewis]

Corinna, thank you.

I have tested and confirmed that the patched cywin1.dll, and include
files /usr/include/netinet/tcp.h and /usr/include/cygwin/socket.h,  work as
expected after I re-compile and link curl (specifically lib/connect.c).

I look forward to seeing this patch included in a future release, and
hopefully that will include rebuilt versions of curl, and other network
utilities, such as wget, nc, etc.

It's always a pleasure working with you.




On Thu, Jul 2, 2020 at 10:52 AM Cary Lewis <cary.lewis@gmail.com> wrote:

> That's amazing, thanks. We'll have to try to recompile curl under cygwin
> to confirm that it keeps up the constants, and then does the right posix
> calls.
>
> I will grab the files, and try to have this tested and report back to you.
>
> Take care,
>
> Cary Lewis
>
> On Wed, Jul 1, 2020 at 3:51 PM Corinna Vinschen <corinna-cygwin@cygwin.com>
> wrote:
>
>> On Jun 30 18:53, Corinna Vinschen wrote:
>> > On Jun 30 09:46, Cary Lewis via Cygwin wrote:
>> > > Thanks for the reply. The answer to your question is that the 2 hour
>> keep
>> > > alive was not sufficient for a particular use case I encountered.
>> > >
>> > > I was trying to use curl under cygwin to access a very slow REST
>> endpoint
>> > > that was taking up to 8 minutes to generate download before any data
>> flowed
>> > > back to the client. This caused the server to abort the socket.
>> > >
>> > > Accessing the endpoint in chrome or firefox revealed that they set a
>> > > keepalive to 45 seconds, which kept the server happy.
>> > >
>> > > Attempting to set --keepalive-time=45 in cygwin's curl didn't work,
>> and
>> > > wireshark revealed that no keepalives were being sent.
>> > >
>> > > I will attempt to patch cygwin, I got the build to work. Can you
>> point me
>> > > in the right direction, in terms of where the socket calls get mapped
>> to
>> > > the winsock calls?
>> >
>> > Actually, while I'm usually happy to take contributions, you don't have
>> > to dig into that yourself.  I already have a few local patches in the
>> > loop changing some of the affected code.  I have a good idea what's
>> > required to add the keep-alive socket options to that code, so just lay
>> > back and stay tuned for now.
>>
>> Ok, so I added support for a couple more IPPROTO_TCP socket options.
>> First of all I fixed TCP_MAXSEG which was using the BSD value, rather
>> than the WinSock value.  Then I added TCP_FASTOPEN, TCP_KEEPIDLE,
>> TCP_KEEPCNT, TCP_KEEPINTVL, TCP_QUICKACK and TCP_USER_TIMEOUT:
>>
>> - TCP_FASTOPEN is supported since W10 1607, it's just faked on older
>>   systems.
>>
>> - TCP_KEEPIDLE, TCP_KEEPCNT, TCP_KEEPINTVL are using the options
>>   of the same name since W10 1709, WSAIoctl(SIO_KEEPALIVE_VALS)
>>   on older systems.
>>
>>   But here's a problem: Older systems didn't allow to change
>>   TCP_KEEPCNT.  It is always fixed to 10.  Mulling over that problem in
>>   the shower, I came up with the following solution:
>>
>>   The max keep-alive timeout is TCP_KEEPIDLE + TCP_KEEPCNT *
>> TCP_KEEPINTVL.
>>   This should stay the same from a user space perspective.  So the current
>>   code tweaks the TCP_KEEPINTVL given to WinSock so that
>>
>>   TCP_KEEPCNT * user space TCP_KEEPINTVL == 10 * WinSock TCP_KEEPINTVL
>>
>>   Example: user space TCP_KEEPCNT 4, TCP_KEEPINTVL 5   (4 * 5 == 20)
>>   ==>      WinSock    TCP_KEEPCNT 10, TCP_KEEPINTVL 2  (10 * 2 == 20)
>>
>>   I hope that makes sense.
>>
>> - TCP_USER_TIMEOUT is supported with msec granularity since W10 1607
>>   (called TCP_MAXRTMS), with 1 secs granularity on older systems
>>   (called TCP_MAXRT).  Use the latter on older systems under the expected
>>   loss of precision.
>>
>> - TCP_QUICKACK is supposedly supported on Windows as a socket option
>>   but it's still not clear if the net got that right so far.  However,
>>   there's WSAIoctl(SIO_TCP_SET_ACK_FREQUENCY) doing the same.
>>
>> I uploaded developer snapshots to https://cygwin.com/snapshots/,
>> please test.
>>
>> For testing, you'll need at least the DLL, plus the changed headers
>> cygwin/socket.h and netinet/tcp.h from the complete tar file
>> cygwin-20200701.tar.xz. Or, just take the DLL and fetch the headers
>> right from the git repo.
>>
>>
>> Thanks,
>> Corinna
>>
>> --
>> Corinna Vinschen
>> Cygwin Maintainer
>> --
>> Problem reports:      https://cygwin.com/problems.html
>> FAQ:                  https://cygwin.com/faq/
>> Documentation:        https://cygwin.com/docs.html
>> Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple
>>
>