RE: [soft-mpop-qqm:27201] Re: scp command fails to transfer file and returns 1.

["suzuki.masaomi@fujitsu.com" <suzuki.masaomi@fujitsu.com>]

Cygwin Support Dear Representatives

Thank you very much for your reply.

>This seems to be behaviour change betweeen OpenSSH 8.x and 9.0.

You are right.

Referring to the URL below, the problem is caused by the change in the protocol 
used by the scp command by default in the latest OpenSSH (9.0p1-1).

https://www.openssh.com/releasenotes.html#9.0p1
---------------------------------
This release switches scp(1) from using the legacy scp/rcp protocol
to using the SFTP protocol by default.

Legacy scp/rcp performs wildcard expansion of remote filenames (e.g.
"scp host:* .") through the remote shell. This has the side effect of
requiring double quoting of shell meta-characters in file names
included on scp(1) command-lines, otherwise they could be interpreted
as shell commands on the remote side.

This creates one area of potential incompatibility: scp(1) when using
the SFTP protocol no longer requires this finicky and brittle quoting,
and attempts to use it may cause transfers to fail. We consider the
removal of the need for double-quoting shell characters in file names
to be a benefit and do not intend to introduce bug-compatibility for
legacy scp/rcp in scp(1) when using the SFTP protocol.

Another area of potential incompatibility relates to the use of remote
paths relative to other user's home directories, for example -
"scp host:~user/file /tmp". The SFTP protocol has no native way to
expand a ~user path. However, sftp-server(8) in OpenSSH 8.7 and later
support a protocol extension "expand-path@openssh.com" to support
this.

In case of incompatibility, the scp(1) client may be instructed to use
the legacy scp/rcp using the -O flag.(★)
---------------------------------

According to the above instructions(the ★ part), in the OpenSSH(9.0p1-1),
when runing the scp commond with -O option, the old protocol(legacy scp/rcp) will be used.

Therefore, I runed the scp command wit the -O option and it was successful.
---------------------------------
C:\Users\Administrator>C:\cygwin64\bin\scp.exe -O "/cygdrive//C/test/a" "root"@RJEL74JP:"\"/tmp/a\""
a                                                               100%   36KB   1.3MB/s   00:00

>echo %errorlevel%
0
---------------------------------

Previously, in the old version Cygwin(3.2.0) that we downloaded from the URL (https://www.cygwin.com/), 
the version of OpenSSH is 8.4p1-1, so on our Windows that the Cygwin(3.2.0) was installed,
the following scp command can be successful.
---------------------------------
<Cygwin Installation Directory>\bin\scp.exe "/cygdrive//C/test/a" "root"@ホスト名:"\"/tmp/a\""
---------------------------------

The problem is not related to the version of Cygwin, I'm sorry to have troubled you.


以降は日本語の文章です。
******************************************************************
Cygwinサポート ご担当様

ご協力、どうもありがとうございました。

>This seems to be behaviour change betweeen OpenSSH 8.x and 9.0.

ご認識のとおりです。

下記のURLを参照したところ、今回の現象は、最新のOpenSSH(9.0p1-1)において、
scpコマンドがデフォルトで使用しているプロトコルの変更に起因していることがわかりました。

https://www.openssh.com/releasenotes.html#9.0p1
---------------------------------
This release switches scp(1) from using the legacy scp/rcp protocol
to using the SFTP protocol by default.

Legacy scp/rcp performs wildcard expansion of remote filenames (e.g.
"scp host:* .") through the remote shell. This has the side effect of
requiring double quoting of shell meta-characters in file names
included on scp(1) command-lines, otherwise they could be interpreted
as shell commands on the remote side.

This creates one area of potential incompatibility: scp(1) when using
the SFTP protocol no longer requires this finicky and brittle quoting,
and attempts to use it may cause transfers to fail. We consider the
removal of the need for double-quoting shell characters in file names
to be a benefit and do not intend to introduce bug-compatibility for
legacy scp/rcp in scp(1) when using the SFTP protocol.

Another area of potential incompatibility relates to the use of remote
paths relative to other user's home directories, for example -
"scp host:~user/file /tmp". The SFTP protocol has no native way to
expand a ~user path. However, sftp-server(8) in OpenSSH 8.7 and later
support a protocol extension "expand-path@openssh.com" to support
this.

In case of incompatibility, the scp(1) client may be instructed to use
the legacy scp/rcp using the -O flag.(★)
---------------------------------

上記の★箇所の説明により、OpenSSH(9.0p1-1)では、
-Oオプションを指定するscpコマンドを実行した場合、古いプロトコル(scp/rcp)が
使用されて、今回のscpコマンドが正常に行われると考えられます。

また、実機で-Oオプションを指定するscpコマンドを実行したところ、
正常に行われていることを確認しました。
---------------------------------
C:\Users\Administrator>C:\cygwin64\bin\scp.exe -O "/cygdrive//C/test/a" "root"@RJEL74JP:"\"/tmp/a\""
a                                                               100%   36KB   1.3MB/s   00:00

>echo %errorlevel%
0
---------------------------------
※Cygwin Bash Shellにも正常に行われています。

過去に、当方がURL(https://www.cygwin.com/)からダウンロードした
旧バージョン(3.2.0)のCygwinにおいて、使用しているOpenSSHのバージョンが8.4p1-1 なので、
該当Cygwinが導入したWinodwsにて、下記のscpコマンドが正常に行われています。
---------------------------------
<Cygwinのインストールディレクトリ>\bin\scp.exe "/cygdrive//C/test/a" "root"@ホスト名:"\"/tmp/a\""
---------------------------------

今回の現象は、Cygwinのバージョンと関係していないため、
ご迷惑をおかけて、申し訳ございません。
******************************************************************

> -----Original Message-----
> From: Takashi Yano <takashi.yano@nifty.ne.jp>
> Sent: Tuesday, July 19, 2022 1:54 PM
> To: 'cygwin@cygwin.com' <cygwin@cygwin.com>
> Cc: Suzuki, Masaomi/鈴木 将臣 <suzuki.masaomi@fujitsu.com>;
> 'soft-mpop-qqm@ml.css.fujitsu.com' <soft-mpop-qqm@ml.css.fujitsu.com>
> Subject: Re: [soft-mpop-qqm:27201] Re: scp command fails to transfer file and
> returns 1.
> 
> On Tue, 19 Jul 2022 13:18:55 +0900
> Takashi Yano <takashi.yano@nifty.ne.jp> wrote:
> 
> > On Tue, 19 Jul 2022 02:40:14 +0000
> > > Thank you very much for your reply.
> > >
> > > >it seems your application is calling scp with " around the arguments, so,
> scp try to copy "/tmp/a" instead of just /tmp/a.
> > >
> > > We analyzed our application again, internally, calling scp command is as
> follows.
> > >
> > > 　<Cygwinのインストールディレクトリ>\bin\scp.exe
> "/cygdrive//C/test/a" "root"@ホスト名:"\"/tmp/a\""
> > >
> > > The scp command above runs successfully with the old version
> > > Cygwin(3.2.0), but not with the the latest version Cygwin(3.3.5).
> > >
> > > <result of execution on command prompt>
> > > =============================================
> > > ●the old version Cygwin(3.2.0)
> > > ---------------------------------
> > > C:\Users\Administrator>C:\cygwin64\bin\scp.exe
> "/cygdrive//C/test/a" "root"@RJEL74JP:"\"/tmp/a\""
> > > a                                                               100%
> 36KB   1.3MB/s   00:00
> > >
> > > >echo %errorlevel%
> > > 0
> > > ---------------------------------
> > > 　→Successfully.
> >
> > In my environment, your test case also fails with cygwin 3.2.0.
> >
> > C:\Users\yano>\cygwin64\bin\uname -a
> > CYGWIN_NT-10.0-19044 HP-Z230 3.2.0-340.x86_64 2021-03-29 08:42 UTC
> > x86_64 Cygwin
> >
> > C:\Users\yano>\cygwin64\bin\scp "/home/yano/a"
> "yano"@server:"\"/tmp/a\""
> > /usr/bin/scp: dest open ""/tmp/a"": No such file or directory
> > /usr/bin/scp: failed to upload file /tmp/a to "/tmp/a"
> >
> > The behaviour difference seems not due to cygwin version.
> 
> This seems to be behaviour change betweeen OpenSSH 8.x and 9.0.
> 
> --
> Takashi Yano <takashi.yano@nifty.ne.jp>