From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13]) by sourceware.org (Postfix) with ESMTPS id 06BC3396E857 for ; Thu, 14 May 2020 15:15:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 06BC3396E857 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=SystematicSw.ab.ca Authentication-Results: sourceware.org; spf=none smtp.mailfrom=brian.inglis@systematicsw.ab.ca Received: from [192.168.1.104] ([24.64.172.44]) by shaw.ca with ESMTP id ZFaMjGgUUng7KZFaNjZSEQ; Thu, 14 May 2020 09:15:48 -0600 X-Authority-Analysis: v=2.3 cv=ecemg4MH c=1 sm=1 tr=0 a=kiZT5GMN3KAWqtYcXc+/4Q==:117 a=kiZT5GMN3KAWqtYcXc+/4Q==:17 a=IkcTkHD0fZMA:10 a=w_pzkKWiAAAA:8 a=mDV3o1hIAAAA:8 a=RW_WsGlVAAAA:8 a=yMhMjlubAAAA:8 a=bP3WA5TJxaZjxncXXDYA:9 a=QEXdDO2ut3YA:10 a=MBDeAHZg_MMA:10 a=hov-qUw37tgA:10 a=p3mPwpJxSVAA:10 a=sRI3_1zDfAgwuvI8zelB:22 a=_FVE-zBwftR9WsbkzFJk:22 a=I7iXOxQ6q06iqld3noLY:22 Reply-To: cygwin@cygwin.com Subject: Re: patch command incorrectly capitalizes filenames that live on external USB flash drives References: From: Brian Inglis Autocrypt: addr=Brian.Inglis@SystematicSw.ab.ca; prefer-encrypt=mutual; keydata= mDMEXopx8xYJKwYBBAHaRw8BAQdAnCK0qv/xwUCCZQoA9BHRYpstERrspfT0NkUWQVuoePa0 LkJyaWFuIEluZ2xpcyA8QnJpYW4uSW5nbGlzQFN5c3RlbWF0aWNTdy5hYi5jYT6IlgQTFggA PhYhBMM5/lbU970GBS2bZB62lxu92I8YBQJeinHzAhsDBQkJZgGABQsJCAcCBhUKCQgLAgQW AgMBAh4BAheAAAoJEB62lxu92I8Y0ioBAI8xrggNxziAVmr+Xm6nnyjoujMqWcq3oEhlYGAO WacZAQDFtdDx2koSVSoOmfaOyRTbIWSf9/Cjai29060fsmdsDLg4BF6KcfMSCisGAQQBl1UB BQEBB0Awv8kHI2PaEgViDqzbnoe8B9KMHoBZLS92HdC7ZPh8HQMBCAeIfgQYFggAJhYhBMM5 /lbU970GBS2bZB62lxu92I8YBQJeinHzAhsMBQkJZgGAAAoJEB62lxu92I8YZwUBAJw/74rF IyaSsGI7ewCdCy88Lce/kdwX7zGwid+f8NZ3AQC/ezTFFi5obXnyMxZJN464nPXiggtT9gN5 RSyTY8X+AQ== Organization: Systematic Software To: cygwin@cygwin.com Message-ID: Date: Thu, 14 May 2020 09:15:46 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfP/iNYLnc6LnBUNsCeIh+bx9CuN/UVh7Fl4Q4Iyw6dXig5IlK5hNG7G03Gc8z4HTQq5CKg5qCA7NJtcrcodH8iwm6TKAnD/0egZZiV2fak1kkIZgzUMo MiSLl9CAUnpcKqC3wAuKJ2YNR6vQk0iviwdMsySAsf12U2fRkq6qCPbtXd0vWKIxH7Dq802iLIxYLQ== X-Spam-Status: No, score=-18.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_EXEURI, KAM_LAZY_DOMAIN_SECURITY, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_NONE, TXREP, USB_DRIVES autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2020 15:16:00 -0000 On 2020-05-13 23:07, Jason Gross wrote: > On Wed, May 13, 2020 at 10:32 PM Jason Gross wrote: >> On Wed, May 13, 2020 at 8:31 PM Jason Gross wrote: >>> Brian Inglis wrote: >>>> Marco Atzeri wrote: >>>>> Thomas Wolff wrote: >>>>>> On Tue, Apr 28, 2020 at 3:27 PM Jason Gross wrote: [Please insert comments inline or add at the bottom on this mailing list] >>>>>> Consider the following script in foo.sh: >>>>>> #!/usr/bin/env bash >>>>>> set -ex >>>>>> cd "$1" >>>>>> rm -rf foo >>>>>> mkdir foo >>>>>> cd foo >>>>>> cat > Makefile <>>>>> a >>>>>> b >>>>>> c >>>>>> d >>>>>> e >>>>>> EOF >>>>>> cat > diff <>>>>> diff --git a/Makefile b/Makefile >>>>>> index 9405325..86d2f8c 100644 >>>>>> --- a/Makefile >>>>>> +++ b/Makefile >>>>>> @@ -1,5 +1,5 @@ >>>>>> a >>>>>> b >>>>>> -c >>>>>> +ccc >>>>>> d >>>>>> e >>>>>> EOF >>>>>> patch -p1 -i ./diff >>>>>> ls >>>>>> If I run `./foo.sh /cygdrive/c/`, I get, as expected, >>>>>> + cd /cygdrive/c/ >>>>>> + rm -rf foo >>>>>> + mkdir foo >>>>>> + cd foo >>>>>> + cat >>>>>> + cat >>>>>> + patch -p1 -i ./diff >>>>>> patching file Makefile >>>>>> + ls >>>>>> diff Makefile >>> Sorry for the late reply; I can see replies to my messages at >>> https://cygwin.com/pipermail/cygwin/2020-April/244660.html, but somehow >>> I'm not receiving them in Gmail. I've tried (re?)subscribing to the >>> cygwin mailing list, hopefully this fixes the problem. >>>> You are throwing a puzzle into the mailing list and if you are lucky, >>>> someone may like to solve it.>>>> But perhaps: can you try to minimize your test case, please.>>>> Something like: touch Makefile; ls (if that's it). >>> I think there's some sort of misconception here. touch and cat create >>> correctly capitalized files, and sed -i doesn't change capitalization, >>> even on my FAT32 drive. patch is the only command I've found so far >>> which capitalizes filenames when modifying files. I can try to dig into >>> the source code of patch and figure out a minimal C program that breaks >>> casing on files, but, come on, the fact that patch seems to capitalize >>> the file name of every file it modifies, and no other utility does this, >>> that seems like a pretty minimal test-case to me. And anyway, the cygwin >>> patch sources (version 2.7.4) are impossible to compile, because safe.c >>> can't find sys/resource.h and passing -I/usr/include via CFLAGS hits an >>> internal bug in patch's configure script (search.h: present but cannot >>> be compiled; sys/timeb.h: present but cannot be compiled; fcntl.h: >>> present but cannot be compiled). (I've emailed bug-patch@gnu.org as >>> requested by the configure script, but so far >>> https://lists.gnu.org/archive/html/bug-patch/ isn't showing anything >>> newer than January.) >>>>>> If I instead run `./foo.sh /cygdrive/h/`, I get >>>>>> ``` >>>>>> + cd /cygdrive/h/ >>>>>> + rm -rf foo >>>>>> + mkdir foo >>>>>> + cd foo >>>>>> + cat >>>>>> + cat >>>>>> + patch -p1 -i ./diff >>>>>> patching file Makefile >>>>>> + ls >>>>>> diff MAKEFILE >>>>>> ``` >>>>>> >>>>>> My C drive is an internal SSD (NTFS), my H drive is an external flash >>>>>> drive (FAT32). I installed cygwin with the commands: >>>>>> ``` >>>>>> powershell -Command "(New-Object >>>>>> Net.WebClient).DownloadFile('http://www.cygwin.com/setup-x86_64.exe', >>>>>> 'setup-x86_64.exe')" >>>>>> SET CYGMIRROR=http://mirror.easyname.at/cygwin >>>>>> SET CYGROOT=H:\cygwin64 >>>>>> SET CYGCACHE=%CYGROOT%\var\cache\setup >>>>>> setup-x86_64.exe -qnNdO -R %CYGROOT% -l %CYGCACHE% -s %CYGMIRROR% -P >>>>>> rsync -P patch -P diffutils -P make -P unzip -P m4 -P findutils -P >>>>>> time -P wget -P curl -P git -P >>>>>> mingw64-x86_64-binutils,mingw64-x86_64-gcc-core,mingw64-x86_64-gcc-g++,mingw64-x86_64-pkg-config,mingw64-x86_64-windows_default_manifest >>>>>> -P mingw64-x86_64-headers,mingw64-x86_64-runtime,mingw64-x86_64-pthreads,mingw64-x86_64-zlib >>>>>> -P python3 >>>>>> ``` >>>>>> >>>>>> Running `patch -v` says `GNU patch 2.7.4`. Note that this happens >>>>>> regardless of whether I install cygwin itself on my external flash >>>>>> drive or on my internal HD. >>>>>> >>>>>> This came up when trying to run `opam install findlib` (which fails >>>>>> when the home directory is on an external USB drive). >>>> That might be expected with FAT32, which is normally the default format >>>> for flash drives, for maximum compatibility with microcontrollers, >>>> which may not create VFAT Long File Names when file names are <= 8.3, >>>> so they appear as upper case. >>> This does not explain why `ls` displays "Makefile" as "Makefile" before I >>> run `patch`, but displays the filename as "MAKEFILE" after I run `patch`. >>> Nor does it explain why this happens to patch-modified files, but not to >>> files modified via sed -i. >>>> use a flash driver with NTFS and check the difference >>> Indeed, I can confirm that this issue occurs when it's FAT or FAT32, and >>> does not occur under NTFS nor exFAT. >>>> I doubt it is a patch issue >>> Do you have another utility that you suggest I try that you think will >>> display the same problem as patch? So far, `| tee -a`, `sed -i`, >>> `touch`, `>`, and `>>` all do not display this issue, while `patch` >>> does. >> By the way, when I run the same script on the same flash drive from WSL, it >> works fine, and does not capitalize the filename. So this does seem to be >> a cygwin-patch specific issue... > Also btw, the bug I reported to patch is at > https://lists.gnu.org/archive/html/bug-patch/2020-05/msg00000.html . It's not > clear to me if this is a cygwin issue or a patch issue. It may be neither - it may be a Mingw/Msys2 issue - that could explain why you can not reproduce the issue with other utilities. You appear to have installed and be running x86_64-w64-mingw32-gcc, binutils, and libraries, and not Cygwin gcc, binutils, and libraries. That will not build Cygwin sources: examine your bug report carefully, check your *PATH*, especially for a ".:" entry, and show the output from: $ pwd; which -a cat gcc ls patch tee touch sed To re-/build Cygwin utilities, install cygport, which will install most of the required packages and run it. If you are using a FAT32 file system and create or rename a file with a naturally short name <= 8.3, you will likely end up with an all uppercase name, unless the program uses a case sensitive long filename API to ensure output case. If you use Cygwin rather than Mingw/Msys2 utilities, it may work, or you may have to force use of a long name > 8.3 e.g. Makefile9.temp, then rename to keep the desired case. It's been a while since I've done anything on any FAT file system. You may want to try enabling long paths to see if this solves your problem: https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#enable-long-paths-in-windows-10-version-1607-and-later $ regtool get -pv /proc/registry/HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet\ /Control/FileSystem/LongPathsEnabled 0 $ regtool set -iv /proc/registry/HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet\ /Control/FileSystem/LongPathsEnabled 1 -- Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada This email may be disturbing to some readers as it contains too much technical detail. Reader discretion is advised. [Data in IEC units and prefixes, physical quantities in SI.]