TCP_CORK (aka TCP_NOPUSH) does not work

TCP_CORK (aka TCP_NOPUSH) does not work

[Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin]

Hi,

Consider the following code:

$ cat cork.c
#include <arpa/inet.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>

#if defined(TCP_NOPUSH)  &&  !defined(TCP_CORK)
#  define TCP_CORK  TCP_NOPUSH
#endif


int main(int argc, const char* argv[])
{
    union {
        struct sockaddr_in in;
        struct sockaddr    sa;
    } addr;
    int sock, cork = 1;

    memset(&addr, 0, sizeof(addr));
    if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0)
        perror("socket");
    addr.in.sin_family = AF_INET;
    addr.in.sin_addr.s_addr = inet_addr(argv[1]);
    addr.in.sin_port = htons((unsigned short) atoi(argv[2]));
    if (connect(sock, &addr.sa, sizeof(addr.in)) < 0)
        perror("connect");
    if (setsockopt(sock, IPPROTO_TCP, TCP_CORK, (char*) &cork, sizeof(cork)) != 0)
        perror("cork");
    return 0;
}

When compiled and run under Cygwin, the last syscall, setsockopt(), returns an error, Protocol not available:

gcc cork.c
./a.exe 8.8.8.8 443
cork: Protocol not available

The same code works under Linux just fine.  I straced both.

gcc cork.c
./a.out 8.8.8.8 443

Any ideas?  Is TCP_NOPUSH (which is a BSDism, BTW) not actually usable on Cygwin?  If not, why is it in the header file <netinet/tcp.h>?

TIA!


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: TCP_CORK (aka TCP_NOPUSH) does not work

[Brian Inglis]

On 2019-07-30 15:30, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin wrote:
> Consider the following code:
> $ cat cork.c
> #include <arpa/inet.h>
> #include <netinet/in.h>
> #include <netinet/tcp.h>
> #include <unistd.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #include <sys/socket.h>
> #include <sys/types.h>
> #if defined(TCP_NOPUSH)  &&  !defined(TCP_CORK)
> #  define TCP_CORK  TCP_NOPUSH
> #endif

For POSIX only non-Nagle TCP_NODELAY is required:
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/netinet_tcp.h.html
and even then says:
"The implementation need not allow the value of the option to be set via
setsockopt() or retrieved via getsockopt()."

TCP_CORK is Linux only; TCP_NOPUSH is BSD only; Windows does its own thing:
https://baus.net/on-tcp_cork/
https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-tcp-socket-options

Regular SO options on Windows:
https://docs.microsoft.com/en-us/windows/win32/winsock/sol-socket-socket-options

You can abuse Nagle to get similar behaviour cross-platform:
https://stackoverflow.com/a/22118709

> int main(int argc, const char* argv[])
> {
>     union {
>         struct sockaddr_in in;
>         struct sockaddr    sa;
>     } addr;
>     int sock, cork = 1;
> 
>     memset(&addr, 0, sizeof(addr));
>     if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0)
>         perror("socket");
>     addr.in.sin_family = AF_INET;
>     addr.in.sin_addr.s_addr = inet_addr(argv[1]);
>     addr.in.sin_port = htons((unsigned short) atoi(argv[2]));
>     if (connect(sock, &addr.sa, sizeof(addr.in)) < 0)
>         perror("connect");
>     if (setsockopt(sock, IPPROTO_TCP, TCP_CORK, (char*) &cork, sizeof(cork)) != 0)
>         perror("cork");
>     return 0;
> }
> When compiled and run under Cygwin, the last syscall, setsockopt(), returns
> an error, Protocol not available:
> gcc cork.c
> ./a.exe 8.8.8.8 443
> cork: Protocol not available

This is error ENOPROTOOPT, where the message is misleading, but as suggested by
the name, means that the socket option is not supported for that protocol;
getsockopt(3p/3posix) says: "The option is not supported by the protocol."
getsockopt(2) for Linux and FreeBSD say: "The option is unknown at the level
indicated." and given the POSIX statement above, that error should be treated as
a warning to do something different.

> The same code works under Linux just fine.  I straced both.
> gcc cork.c
> ./a.out 8.8.8.8 443
> Any ideas?  Is TCP_NOPUSH (which is a BSDism, BTW) not actually usable on
Cygwin?  If not, why is it in the header file <netinet/tcp.h>?

If a socket option is defined, perhaps for compatibility, it should either be
used or ignored, rather than giving an error.
If you are not going to support a socket option, and generate an error, it would
be better to not define the option and generate the error at compile time,
instead of failing at run time.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: TCP_CORK (aka TCP_NOPUSH) does not work

[Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin]

> Why not just write a patch to fix this and send it to the cygwin-patches mailing list?

I had first to make sure that the feature is indeed unsupported (and I wasn't 100% certain because it was included in the header).
Actually, nobody confirmed to me that I am not "seeing things", only taught me what I should be doing.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: TCP_CORK (aka TCP_NOPUSH) does not work

[Ken Brown]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 614 bytes --]

On 7/31/2019 5:00 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin wrote:
>> Linux and BSD options are not available, whatever the .h #defines.
> 
> Unsupported options should not be shown as features in the headers.

Why not just write a patch to fix this and send it to the cygwin-patches 
mailing list?  It would take much less time than continuing this discussion.

Ken
\x03B‹KCB”\x1c›Ø›\x19[H\x1c™\^[ܝ\x1cΈ\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÜ\x1c›Ø›\x19[\Ëš\x1d^[[\x03B‘TNˆ\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ˜\KÃB‘^[ØÝ[Y[\x18]\x1a[ÛŽˆ\b\b\b\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÙ^[ØÜËš\x1d^[[\x03B•[œÝXœØÜšX™H\x1a[™›Îˆ\b\b\b\b\b\x1a\x1d\x1d\x1c\x0e‹ËØÞYÝÚ[‹˜ÛÛKÛ[\vÈÝ[œÝXœØÜšX™K\Ú[\^[\x19CBƒB

Re: TCP_CORK (aka TCP_NOPUSH) does not work

[Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin]

> Linux and BSD options are not available, whatever the .h #defines.

Unsupported options should not be shown as features in the headers.

> Many header and library files are shared with newlib, which is used by Cygwin

If you checked before responding with a large digress into unrelated details,
you could have seen that the file in question is Cygwin's own, and all the history of mods is right there:

https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=blame;f=winsup/cygwin/include/netinet/tcp.h;h=becbec54f800722a2efff2ab4676a71752189d74;hb=HEAD



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: TCP_CORK (aka TCP_NOPUSH) does not work

[Brian Inglis]

On 2019-07-30 22:48, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin wrote:
>> Regular SO options on Windows:
> 
> I did not ask about Windows options.  I asked about a feature that is
> defined in Cygwin yet appears not functional.

You are running Windows sockets under Cygwin emulation.
Linux and BSD options are not available, whatever the .h #defines.

>> You can abuse Nagle to get similar behaviour cross-platform:
> This is not the same thing!

This is your only option.

>> ENOPROTOOPT (109)... "The option is not supported by the protocol."
> 
> Indeed the textual description does not match, and should be changed in Cygwin.  Although, I'm sure it was copied over from Linux, which does not define it properly (probably, historically), either:
> 
> #define ENOPROTOOPT     92      /* Protocol not available */
> 
> BTW, Windows does that correctly:
> 
> WSAENOPROTOOPT (10042) = "Bad protocol option"

>> If a socket option is defined, perhaps for compatibility, it should either be
>> used or ignored, rather than giving an error.
>> If you are not going to support a socket option, and generate an error, it would
>> be better to not define the option and generate the error at compile time,
>> instead of failing at run time.
> Who are you suggesting this to?

Anyone who wants to volunteer to remove non-functional socket options: that's
how Cygwin works.
Input from folks who use and maintain this stuff would be good, before any of us
jump in and delete stuff.
Many header and library files are shared with newlib, which is used by Cygwin,
RTEMS, other embedded non-Linux OSes, many common boards using ARM, Renesas, and
Xilinx chips, most commercial and non-commercial GCC ports and distributions for
non-Linux embedded systems, so conditionalization is often required, unless
newlib provides its own separate version(s) of a header or library file.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: TCP_CORK (aka TCP_NOPUSH) does not work

[Lavrentiev, Anton (NIH/NLM/NCBI) [C] via cygwin]

> Regular SO options on Windows:

I did not ask about Windows options.  I asked about a feature that is defined in Cygwin yet appears not functional.

> You can abuse Nagle to get similar behaviour cross-platform:

This is not the same thing!

> ENOPROTOOPT (109)... "The option is not supported by the protocol."

Indeed the textual description does not match, and should be changed in Cygwin.  Although, I'm sure it was copied over from Linux, which does not define it properly (probably, historically), either:

#define ENOPROTOOPT     92      /* Protocol not available */

BTW, Windows does that correctly:

WSAENOPROTOOPT (10042) = "Bad protocol option"

> If a socket option is defined, perhaps for compatibility, it should either be
> used or ignored, rather than giving an error.
> If you are not going to support a socket option, and generate an error, it would
> be better to not define the option and generate the error at compile time,
> instead of failing at run time.

Who are you suggesting this to?


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple