From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) by sourceware.org (Postfix) with ESMTPS id B8E55385828F for ; Mon, 18 Dec 2023 11:10:32 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B8E55385828F Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org B8E55385828F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::52a ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702897836; cv=none; b=VdGrRcDSsM8yA9V7UQMIMO5cFHxeXOz4XdgcB/q1/xi7rqVXJf24lMvZ6rwGPCzpFoJCNUDbVZBW/NC2CuvGBTMgLKfEpbFqAUEjMhDkufKearoOkplvpP57wGJpHQiR1p/Usk9hycdi96Ya7056e8re7nMnimOh0rJLq3M9dGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1702897836; c=relaxed/simple; bh=Hdlshq17p64jVAQxrMdRIyt7bnSulvwAArWvN7amb/4=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=M9z9xqhErX4i+I9gD/+b+TXW4CpJJ3PDjhEc1jg0xjbIx6aDxM30dnvicdg0+XihiHkDRnF0rlwnjwCSp27QJ/+JCD5GmNajrWLboKcCv43m5WD4XJDl2d0qPYiN0FAJy4fXmk8j9ycVQcgdMWTw1o72OGPYomuNQiOh4f/k6Hs= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-55359dc0290so990490a12.1 for ; Mon, 18 Dec 2023 03:10:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702897831; x=1703502631; darn=cygwin.com; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=w8Gh9v+vN4pM8Q+zRqbMOyUUcYoFQBJBT+XO5vmhGkA=; b=DUNWnAMxvSrvpuDX/ijVC3orLh59lONLguem0KJX4kSfc9oERe/wNHxGXBWF1dolOv vVoqo2jDI3vGAGD7UGGTdp6M6pQWM9ArcaqVwwFmdPqUiedyHruUMfBvUFujluPlzxYI U/Qc098aoPZw2pdoocQb4oMPxEKvMzLNpm3YJhPZX5Bn5KTypDUNsF6zR4kriVhEJKFA X6LrpAvGyvMVro8lM8FH0rBTWRO6VCR0svD8e+ZUu60C/lYDDy4M8Jn1BmXLbNfg6OL5 OpAs/1GxVc3H+YceYQinQlCj7ihKogZK0kKtuQR6pnl9jSYV9sssKGu4CTjbyRcwCeqy py8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702897831; x=1703502631; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=w8Gh9v+vN4pM8Q+zRqbMOyUUcYoFQBJBT+XO5vmhGkA=; b=M1c5ZM/+YBSBdfxsCy/rCkeHqOahs7H6R3pXN9IeiloK55qeAHwhDJ4z4FnnoDykw/ +OzTtXQj7fiBGWwngXPglWF/KiuVGocwsg2XX1KHppX30xT00DemurT2BvaXz2HbUH+t vDwIqaNTgPxzkrdG4q3iIrYriSE/0bAfHHXi8fjlbl/swqgFD6UFjJKIe/eekYtRWs86 hhOlwydxR5tBQWO1huteXxLLf3Pk8ASRi2sfbyXwX4ED2nQntoilEZd1SC/2NSKTLHJJ X2S7Nbssn/QMvG7VO2HbmDmzXM7+D4Fs25Da9M95/khmbztrBj6jpJh8yKB78IYwsXq7 QG2g== X-Gm-Message-State: AOJu0YxNA/UbQOhGdFd53SoG6anXHmvtgn0OjCYoAuFnEL4M7a50Ozw8 Rnb5HcWiOtQH25NGdFrVikhzn13440mEZinsC02dZBet X-Google-Smtp-Source: AGHT+IHvur7oRFkkBLqBQqGZH6nXzmO6cghcHCCeLm+ePZkT2a8fuVoRmMsa9riso7Kwr9lk4IconBQbOtLx9qBA3Q4= X-Received: by 2002:a50:cd97:0:b0:553:d69:9059 with SMTP id p23-20020a50cd97000000b005530d699059mr3246600edi.4.1702897830777; Mon, 18 Dec 2023 03:10:30 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Cedric Blancher Date: Mon, 18 Dec 2023 12:10:00 +0100 Message-ID: Subject: Re: Sparse file support for SMB by default? Re: Comment about "Cygwin: sparse support: enable automatic sparsifying of files on SSDs", extend feature to VMware/qemu disks? To: cygwin@cygwin.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Wed, 6 Dec 2023 at 10:39, Corinna Vinschen via Cygwin wrote: > > On Dec 6 09:49, Cedric Blancher via Cygwin wrote: > > On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin > > VMware emulates NVME SSD these days, so this should work, yes? > > It doesn't matter, see below. > > > Primary concern is how to detect whether sparse file support is ON in > > Cygwin for a specific filesystem? Maybe /sbin/mount without options > > should have a sparse/nosparse mount flag to reflect what is used? > > What concern? I explained exactly what happens, and it all revolves > around the fact that a filesystem sets the FILE_SUPPORTS_SPARSE_FILES > flag or not. This is the only interesting fact. Nothing else > matters. > > If you want to see if a filesystem supports that flag, just use the > /usr/lib/csih/getVolInfo tool: > > $ /usr/lib/csih/getVolInfo . > Device Type : 7 > Characteristics : 20020 > Volume Name : > Serial Number : 2292001085 > Max Filenamelength : 255 > Filesystemname : > Flags : 3e72eff > FILE_CASE_SENSITIVE_SEARCH : TRUE > FILE_CASE_PRESERVED_NAMES : TRUE > FILE_UNICODE_ON_DISK : TRUE > FILE_PERSISTENT_ACLS : TRUE > FILE_FILE_COMPRESSION : TRUE > FILE_VOLUME_QUOTAS : TRUE > FILE_SUPPORTS_SPARSE_FILES : TRUE > FILE_SUPPORTS_REPARSE_POINTS : TRUE > FILE_SUPPORTS_REMOTE_STORAGE : FALSE > FILE_RETURNS_CLEANUP_RESULT_INFO : TRUE > FILE_SUPPORTS_POSIX_UNLINK_RENAME : TRUE > FILE_VOLUME_IS_COMPRESSED : FALSE > FILE_SUPPORTS_OBJECT_IDS : TRUE > FILE_SUPPORTS_ENCRYPTION : TRUE > FILE_NAMED_STREAMS : TRUE > FILE_READ_ONLY_VOLUME : FALSE > FILE_SEQUENTIAL_WRITE_ONCE : FALSE > FILE_SUPPORTS_TRANSACTIONS : TRUE > FILE_SUPPORTS_HARD_LINKS : TRUE > FILE_SUPPORTS_EXTENDED_ATTRIBUTES : TRUE > FILE_SUPPORTS_OPEN_BY_FILE_ID : TRUE > FILE_SUPPORTS_USN_JOURNAL : TRUE > FILE_SUPPORTS_INTEGRITY_STREAMS : FALSE > FILE_SUPPORTS_BLOCK_REFCOUNTING : FALSE > FILE_SUPPORTS_SPARSE_VDL : FALSE > FILE_DAX_VOLUME : FALSE > FILE_SUPPORTS_GHOSTING : FALSE > > > > What Cygwin gets to see is that you want to access a file on some > > > disk. It opens the file and fetches volume information. So it finds: > > > > > > - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES > > > > > > So we can create sparse files. > > > > > > - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag. > > > > > > So we're on a drive having no seek penalty. This is *probably* > > > an SSD, but may be some other type of drive. > > > > What about SMB? SMB supports sparse files, and should certainly use > > this by DEFAULT, as it makes many HPC apps operate much faster. > > BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a > > drive, and not for network filesystem. > > > > Same applies to the NFSv4.1 filesystem driver, I'm going to pester > > Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue > > as SMB here. > > Again, the filesystem doesn't matter. It either sets the > FILE_SUPPORTS_SPARSE_FILES flag or not, as simple as that. > > If it does, you can create sparse files with chattr +S, or you can rely > on the lseek/ftruncate/posix_fallocate automatism, or you stamp a hole > into the file with fallocate(FALLOC_FL_PUNCH_HOLE). > > The *only* difference is if you have to use the "sparse" mount option or > not. > > Basically, with 3.4, you always have to set the "sparse" mount option, > with 3.5, on local SSDs you don't. I don't see a problem here. How can I remount the existing C: mount with the "sparse" option in Cygwin 3.4? I cannot get it working, or I am too stupid. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur