From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by sourceware.org (Postfix, from userid 2155) id 5EA9D3858C54; Wed, 6 Dec 2023 09:38:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5EA9D3858C54 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1701855527; bh=V+7jcaqW0Pi2VvcAaBcpOILu0hGqqY1C7nza9LuQpwM=; h=Date:From:To:Subject:Reply-To:References:In-Reply-To:From; b=s/WH9567x2BsLmcNVo8b+TY3s2R1RW9AUYXM4qjS0wqQLobOxlA0c+5GCw/0SVOhF vXE5DrTy08xO1TdKd4RuFkrFirscZiMffTAEL3B1BhkFSnNTkjrOIdB+c3GTG/AvH7 CAKlI33IUyLuUigLXfko51U/T3JRXDRPHqwtNp+E= Received: by calimero.vinschen.de (Postfix, from userid 500) id E3600A80A6B; Wed, 6 Dec 2023 10:38:44 +0100 (CET) Date: Wed, 6 Dec 2023 10:38:44 +0100 From: Corinna Vinschen To: cygwin@cygwin.com Subject: Re: Sparse file support for SMB by default? Re: Comment about "Cygwin: sparse support: enable automatic sparsifying of files on SSDs", extend feature to VMware/qemu disks? Message-ID: Reply-To: cygwin@cygwin.com Mail-Followup-To: cygwin@cygwin.com References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: On Dec 6 09:49, Cedric Blancher via Cygwin wrote: > On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin > VMware emulates NVME SSD these days, so this should work, yes? It doesn't matter, see below. > Primary concern is how to detect whether sparse file support is ON in > Cygwin for a specific filesystem? Maybe /sbin/mount without options > should have a sparse/nosparse mount flag to reflect what is used? What concern? I explained exactly what happens, and it all revolves around the fact that a filesystem sets the FILE_SUPPORTS_SPARSE_FILES flag or not. This is the only interesting fact. Nothing else matters. If you want to see if a filesystem supports that flag, just use the /usr/lib/csih/getVolInfo tool: $ /usr/lib/csih/getVolInfo . Device Type : 7 Characteristics : 20020 Volume Name : Serial Number : 2292001085 Max Filenamelength : 255 Filesystemname : Flags : 3e72eff FILE_CASE_SENSITIVE_SEARCH : TRUE FILE_CASE_PRESERVED_NAMES : TRUE FILE_UNICODE_ON_DISK : TRUE FILE_PERSISTENT_ACLS : TRUE FILE_FILE_COMPRESSION : TRUE FILE_VOLUME_QUOTAS : TRUE FILE_SUPPORTS_SPARSE_FILES : TRUE FILE_SUPPORTS_REPARSE_POINTS : TRUE FILE_SUPPORTS_REMOTE_STORAGE : FALSE FILE_RETURNS_CLEANUP_RESULT_INFO : TRUE FILE_SUPPORTS_POSIX_UNLINK_RENAME : TRUE FILE_VOLUME_IS_COMPRESSED : FALSE FILE_SUPPORTS_OBJECT_IDS : TRUE FILE_SUPPORTS_ENCRYPTION : TRUE FILE_NAMED_STREAMS : TRUE FILE_READ_ONLY_VOLUME : FALSE FILE_SEQUENTIAL_WRITE_ONCE : FALSE FILE_SUPPORTS_TRANSACTIONS : TRUE FILE_SUPPORTS_HARD_LINKS : TRUE FILE_SUPPORTS_EXTENDED_ATTRIBUTES : TRUE FILE_SUPPORTS_OPEN_BY_FILE_ID : TRUE FILE_SUPPORTS_USN_JOURNAL : TRUE FILE_SUPPORTS_INTEGRITY_STREAMS : FALSE FILE_SUPPORTS_BLOCK_REFCOUNTING : FALSE FILE_SUPPORTS_SPARSE_VDL : FALSE FILE_DAX_VOLUME : FALSE FILE_SUPPORTS_GHOSTING : FALSE > > What Cygwin gets to see is that you want to access a file on some > > disk. It opens the file and fetches volume information. So it finds: > > > > - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES > > > > So we can create sparse files. > > > > - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag. > > > > So we're on a drive having no seek penalty. This is *probably* > > an SSD, but may be some other type of drive. > > What about SMB? SMB supports sparse files, and should certainly use > this by DEFAULT, as it makes many HPC apps operate much faster. > BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a > drive, and not for network filesystem. > > Same applies to the NFSv4.1 filesystem driver, I'm going to pester > Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue > as SMB here. Again, the filesystem doesn't matter. It either sets the FILE_SUPPORTS_SPARSE_FILES flag or not, as simple as that. If it does, you can create sparse files with chattr +S, or you can rely on the lseek/ftruncate/posix_fallocate automatism, or you stamp a hole into the file with fallocate(FALLOC_FL_PUNCH_HOLE). The *only* difference is if you have to use the "sparse" mount option or not. Basically, with 3.4, you always have to set the "sparse" mount option, with 3.5, on local SSDs you don't. I don't see a problem here. Corinna