From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com [IPv6:2a00:1450:4864:20::22f]) by sourceware.org (Postfix) with ESMTPS id C5CB53851C28 for ; Fri, 5 Jun 2020 14:14:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org C5CB53851C28 Received: by mail-lj1-x22f.google.com with SMTP id n24so11894234lji.10 for ; Fri, 05 Jun 2020 07:14:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=G2we/mQous6k7zn2pBS+v9kHtEXYuoNDyg3+65dr2IU=; b=sMpw6HcLj9NfR11jf8Yj2DgdvJzweOOEv231EUAC6VlJcP40ZSjWvYPAIyiOhL7mzM cK1lmH1g5/1R3wrGPvb1+3JhAzCb4bLioLaEbAqpiD+1Q2cK6u7VRGUPBXgHUBo15sil URdso+I688wv7sd8IFRwzSro7Ta0vZZhbKb9DKUcJy/YDkcuhXeQHn39iylpo4nIFtNa OrWPz6BgTTsFBs1hPGj1BXWOhjedd72809oa8dWYTPvqnW1WbKpwBLulCJ0XC7pPkeRl gZAmJPGgKtuVJv8vkETdp8xlTcUfFFositL/8/wjALxTNmvLfeiqlghNc0b4MXDOJMGn 475Q== X-Gm-Message-State: AOAM533KT22pm7fBeQigBiV22huIpNUGnUuB0G1E5/lAvSCAW4XJ+6SO 2l0dk4VTZOxDelC1M/XLFsfypxEvcoiwab/UTJAYCEc93LsEdbodjA== X-Google-Smtp-Source: ABdhPJw0Wl62etZ2hM61AD9Doqah+9KD4qsMImS5rl+ODt6DP4fGpu/bX/P3iX/7QOsS0lL82sTV77Jydwh4jFPE97Y= X-Received: by 2002:a2e:b892:: with SMTP id r18mr5114442ljp.319.1591366444072; Fri, 05 Jun 2020 07:14:04 -0700 (PDT) MIME-Version: 1.0 References: <154000859.3440830.1567799052771.ref@mail.yahoo.com> <154000859.3440830.1567799052771@mail.yahoo.com> <1795784342.3888023.1567906713266@mail.yahoo.com> In-Reply-To: <1795784342.3888023.1567906713266@mail.yahoo.com> From: Cary Lewis Date: Fri, 5 Jun 2020 10:13:52 -0400 Message-ID: Subject: Re: cURL uploads 0 length file for sftp transfer. To: cygwin@cygwin.com X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, HTML_MESSAGE, KAM_LINEPADDING, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2020 14:14:08 -0000 Curl under cygwin - TCP keepalive I need to download a file from a remote server via a https request, and the remote server takes several minutes to prepare the file. Without any keepalives, the remote end will terminate the socket, but it doesn't close it (which I know is weird). Running curl with --keepalive 45 on a linux centos7 machine, enables the transfer to work. With tcpdump, I see the nop packet every 45 seconds. But on cygwin, the --keepalive parameter doesn't seem to have effect. Running wireshark, I don't see any keepalives. Running the request in a browser, chrome or firefox, works fine, and I can see keepalives in wireshark, so clearly they are either setting the socket options correctly, or they are sending their own nop. Running on a vm, vmware workstation / centos8 , on the same machine where I am running curl, I can see the keepalives. I have downloaded and rebuilt curl and that did not resolve the issue. Can someone shed some light on this? On Sat, Sep 7, 2019 at 9:39 PM ALbert via cygwin wrote: > I upgraded to the latest version curl 7.65.3 with libcurl 7.65.3, same > issue. > curl 7.65.3 (i686-pc-cygwin) libcurl/7.65.3 OpenSSL/1.1.1c zlib/1.2.11 > brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.0.4) > libssh/0.8.7/openssl/zlib nghttp2/1.37.0 > Thanks. > > On Friday, September 6, 2019, 3:44:13 PM EDT, ALbert < > albert_76@yahoo.com> wrote: > > We recently upgraded cURL from version:curl 7.50.3 (i686-pc-cygwin) > libcurl/7.50.3 OpenSSL/1.0.2j zlib/1.2.8 libidn/1.29 libpsl/0.14.0 > (+libidn/1.29) libssh2/1.7.0 nghttp2/1.14.0tocurl 7.64.1 (i686-pc-cygwin) > libcurl/7.65.3 OpenSSL/1.1.1c zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 > libpsl/0.21.0 (+libidn2/2.0.4) libssh/0.8.7/openssl/zlib nghttp2/1.37.0 > Since the upgrade, we've seen problems uploading text files to sftp > servers. Here's the call that we make: > curl--trace-ascii trace.txt -k sftp://:@/file.txt-T > file.txt > > On initial call, a 0-length file is uploaded, with successful return code. > On immediate rerun, a full file is uploaded (if we wait few minutes, > 0-length file will get uploaded). Below are the traces from initial and > subsequent calls. We also tried using same version curl on a linux box, > without any issues. We also downloaded latest version of curl from curl for > Windows, "curl 7.65.3 (i386-pc-win32) libcurl/7.65.3 OpenSSL/1.1.1c > (Schannel) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.9.0 nghttp2/1.39.2", > and its also working correctly. > Unfortunately, the server we're uploading to belongs to a client, and we > can't see if this is an issue on a server side. However, even if it was, > why would linux curl and windows curl (from curl.haxx.se) would work > correctly? > Any help or pointers would be appreciated. Thank you. > > Here's the trace files: > From initial call:== Info: STATE: INIT => CONNECT handle 0x80068340; line > 1356 (connection #-5000) > == Info: Added connection 0. The cache now contains 1 members > == Info: STATE: CONNECT => WAITRESOLVE handle 0x80068340; line 1397 > (connection #0) > == Info: Trying XXX.XX.XX.XX...== Info: TCP_NODELAY set > == Info: STATE: WAITRESOLVE => WAITCONNECT handle 0x80068340; line 1476 > (connection #0) > == Info: Connected to XXXXXXX ( XXX.XX.XX.XX) port 22 (#0)== Info: STATE: > WAITCONNECT => SENDPROTOCONNECT handle 0x80068340; line 1532 (connection #0) > == Info: Marked for [keep alive]: SSH default > == Info: User: Opus_efg > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_INIT (line 2115) > == Info: SSH 0x80048f90 state change from SSH_INIT to SSH_S_STARTUP (line > 582) > == Info: STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x80068340; line > 1547 (connection #0) > == Info: SSH 0x80048f90 state change from SSH_S_STARTUP to SSH_HOSTKEY > (line 595) > == Info: SSH 0x80048f90 state change from SSH_HOSTKEY to SSH_AUTHLIST > (line 605) > == Info: SSH 0x80048f90 state change from SSH_AUTHLIST to > SSH_AUTH_PKEY_INIT (line 628) > == Info: Authentication using SSH public key file > == Info: SSH 0x80048f90 state change from SSH_AUTH_PKEY_INIT to > SSH_AUTH_KEY_INIT (line 697) > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY_INIT to > SSH_AUTH_KEY (line 743) > == Info: completed keyboard interactive authentication > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY to SSH_AUTH_DONE > (line 761) > == Info: Authentication complete > == Info: SSH 0x80048f90 state change from SSH_AUTH_DONE to SSH_SFTP_INIT > (line 807) > == Info: SSH 0x80048f90 state change from SSH_SFTP_INIT to > SSH_SFTP_REALPATH (line 833) > == Info: SSH CONNECT phase done > == Info: SSH 0x80048f90 state change from SSH_SFTP_REALPATH to SSH_STOP > (line 850) > == Info: STATE: PROTOCONNECT => DO handle 0x80068340; line 1566 > (connection #0) > == Info: DO phase starts > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_QUOTE_INIT > (line 2332) > == Info: SSH 0x80048f90 state change from SSH_SFTP_QUOTE_INIT to > SSH_SFTP_GETINFO (line 868) > == Info: SSH 0x80048f90 state change from SSH_SFTP_GETINFO to > SSH_SFTP_TRANS_INIT (line 1060) > == Info: SSH 0x80048f90 state change from SSH_SFTP_TRANS_INIT to > SSH_SFTP_UPLOAD_INIT (line 1080) > == Info: SSH 0x80048f90 state change from SSH_SFTP_UPLOAD_INIT to > SSH_SFTP_CLOSE (line 1143) > == Info: STATE: DO => DOING handle 0x80068340; line 1608 (connection #0) > == Info: SFTP DONE done > == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line > 1651) > == Info: DO phase is complete > == Info: STATE: DOING => DO_DONE handle 0x80068340; line 1693 (connection > #0) > == Info: STATE: DO_DONE => PERFORM handle 0x80068340; line 1743 > (connection #0) > == Info: STATE: PERFORM => DONE handle 0x80068340; line 1933 (connection > #0) > == Info: multi_done > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_CLOSE (line > 2390) > == Info: SFTP DONE done > == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line > 1651) > == Info: Connection #0 to host XXXXXXX left intact > > > Trace from subsequent call:== Info: STATE: INIT => CONNECT handle > 0x80068340; line 1356 (connection #-5000) > == Info: Added connection 0. The cache now contains 1 members > == Info: STATE: CONNECT => WAITRESOLVE handle 0x80068340; line 1397 > (connection #0) > == Info: Trying XXX.XX.XX.XX...== Info: TCP_NODELAY set > == Info: STATE: WAITRESOLVE => WAITCONNECT handle 0x80068340; line 1476 > (connection #0) > == Info: Connected to XXXXXXX (XXX.XX.XX.XX) port 22 (#0) > == Info: STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x80068340; line > 1532 (connection #0) > == Info: Marked for [keep alive]: SSH default > == Info: User: Opus_efg > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_INIT (line 2115) > == Info: SSH 0x80048f90 state change from SSH_INIT to SSH_S_STARTUP (line > 582) > == Info: STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x80068340; line > 1547 (connection #0) > == Info: SSH 0x80048f90 state change from SSH_S_STARTUP to SSH_HOSTKEY > (line 595) > == Info: SSH 0x80048f90 state change from SSH_HOSTKEY to SSH_AUTHLIST > (line 605) > == Info: SSH 0x80048f90 state change from SSH_AUTHLIST to > SSH_AUTH_PKEY_INIT (line 628) > == Info: Authentication using SSH public key file > == Info: SSH 0x80048f90 state change from SSH_AUTH_PKEY_INIT to > SSH_AUTH_KEY_INIT (line 697) > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY_INIT to > SSH_AUTH_KEY (line 743) > == Info: completed keyboard interactive authentication > == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY to SSH_AUTH_DONE > (line 761) > == Info: Authentication complete > == Info: SSH 0x80048f90 state change from SSH_AUTH_DONE to SSH_SFTP_INIT > (line 807) > == Info: SSH 0x80048f90 state change from SSH_SFTP_INIT to > SSH_SFTP_REALPATH (line 833) > == Info: SSH CONNECT phase done > == Info: SSH 0x80048f90 state change from SSH_SFTP_REALPATH to SSH_STOP > (line 850) > == Info: STATE: PROTOCONNECT => DO handle 0x80068340; line 1566 > (connection #0) > == Info: DO phase starts > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_QUOTE_INIT > (line 2332) > == Info: SSH 0x80048f90 state change from SSH_SFTP_QUOTE_INIT to > SSH_SFTP_GETINFO (line 868) > == Info: SSH 0x80048f90 state change from SSH_SFTP_GETINFO to > SSH_SFTP_TRANS_INIT (line 1060) > == Info: SSH 0x80048f90 state change from SSH_SFTP_TRANS_INIT to > SSH_SFTP_UPLOAD_INIT (line 1080) > == Info: SSH 0x80048f90 state change from SSH_SFTP_UPLOAD_INIT to SSH_STOP > (line 1222) > == Info: DO phase is complete > == Info: STATE: DO => DO_DONE handle 0x80068340; line 1621 (connection #0) > == Info: STATE: DO_DONE => PERFORM handle 0x80068340; line 1743 > (connection #0) > => Send data, 10688 bytes (0x29c0) > SENSITIVE DATA DELETED > == Info: We are completely uploaded and fine > == Info: STATE: PERFORM => DONE handle 0x80068340; line 1933 (connection > #0) > == Info: multi_done > == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_CLOSE (line > 2390) > == Info: SFTP DONE done > == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line > 1651) > == Info: Connection #0 to host XXXXXXX left intact > == Info: Expire cleared (transfer 0x80068340) > > > > > > > > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > >