From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) by sourceware.org (Postfix) with ESMTPS id C963A3858D28 for ; Tue, 2 Apr 2024 22:35:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C963A3858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C963A3858D28 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::12b ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712097343; cv=none; b=kFRPq/6vF8dbjzbSY0N5fg5yk/jzM9kWPhgC53+cIHcHeftin4pLwCfBQsOJxCCDCCiL9uFzIz5yMCEIlJhQUYKTsoKtQJJnZySxaiIwBU3eXfGK3EUIP0bj9aRzGlDcc6KHOcARWoszMMwXo7Vl4/n8NekGIdOlXDgmqFv43cE= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1712097343; c=relaxed/simple; bh=PqdJa5uPCmFlpobUK/Xdrav7DcQ3oVnH6ja8mJb//9s=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=fLTG3f2Fd/14oda8+lMQItAfIesYbf9xoV1OCgR5C3Q2mrMCWwq1ZOag4ZcN7ocB/xcdNXk5LxPxDCKI8gDy9duNrc7sPif2wbzXpD2QD1ELMxRe1pBlSE5MZdBk/I2O+wgGmz1xwWvb1RpPvAt0a9kyqGFBT24MqxPRCKN1qrY= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lf1-x12b.google.com with SMTP id 2adb3069b0e04-51381021af1so7890299e87.0 for ; Tue, 02 Apr 2024 15:35:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712097339; x=1712702139; darn=cygwin.com; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=mQV/NmotOyZWkRd+yY1Ur6ekZAN70VRA6oI14xBGtRQ=; b=GPzTpsmSnSMO3Ue0QqJm9BJHHCnSSEBnVxjg3NsPRiCpLIslkMJE0hoaF0Qdj8eXLN w/u37kyM9oAjgMBZeMG4aXFLLEul9faIhlV78oB44Vq5PutcdaT7Ozr/bUoW2JXOJLA5 wgVOsPi7KhPPFHuLpIJAo8aKH/vp1OP1aOkg2PJXl6mEcDp4/3/rE6GkJxFuqnkv0QSt eVUluEhdmDBpVGV5ze9BZaM0T91MBvRv6vFurvIEl7MmJlEM2psptTYBph+/SYdDDi6G +bZomKFNvLzA9Z9cc0IR8Jogi+lxqA3+p0btAllpag2Mh+dpZVA05RgglsStceoNUSRF 6bSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712097339; x=1712702139; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mQV/NmotOyZWkRd+yY1Ur6ekZAN70VRA6oI14xBGtRQ=; b=Wp5ZauAW7ks2/6az0dJ5YbffxBxyaaeyPrS8sujk1w/7q+CDquG51TQLy7jGUBW7mu UIqg1+W1Px777fbgkS0mX8eDq/T+kph6Xk1SdQUqisxADCujKpIKrwFixzoGbccKFH6c gnRv0TS/fn6Kn5iRQlTM5jC6mI24z8G4q/xD1ZF5fFsGDYLuXbqQnHDBSRCXI+UwKS/u unF4C2DzBaZgQQduV/od5DOo6BvdAjzO16/ab09sKdFdOxc9VDZ7AGFynyKEjP0iTlPh T39KiA8rDH1lTTx+PAxYCxcP6a0c7yS5ATuMknl3m3r/YScbSGAhlyasPe8EL9qHE9+k Yihw== X-Gm-Message-State: AOJu0YwdIlakdcDf6YFpRegPDeNNo9IrWfLpPe/bMpgZt0GHfsr4bW1x 7Favjl66aB8FL922pR6cJ45T1UK/dUBu1XeH3iBuyQlBK62LCR20bo+35kUaDPzh89Q4pMakxqG TPbJsfU85aV5+xt+yWHKSOOSOMxIs7x8fkg4= X-Google-Smtp-Source: AGHT+IEpmwYKre2w5V094IMDI8lkP5zRKoDzniaC8Ltecm8bCm5Bv24W0aC6wemj+wvP7ezR88MRQ0la+FOIhXtqQAs= X-Received: by 2002:a05:6512:3f28:b0:515:a9d5:aab6 with SMTP id y40-20020a0565123f2800b00515a9d5aab6mr2797517lfa.18.1712097338678; Tue, 02 Apr 2024 15:35:38 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Martin Wege Date: Wed, 3 Apr 2024 00:35:27 +0200 Message-ID: Subject: Re: Cygwin&Win32 file prefetch, block sizes? To: cygwin@cygwin.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, Apr 2, 2024 at 3:17=E2=80=AFPM Corinna Vinschen via Cygwin wrote: > > On Apr 2 02:04, Martin Wege via Cygwin wrote: > > Hello, > > > > Is there any document which describes how Cygwin and Win32 file > > prefetch and readahead work, and which sizes are used (e.g. always > > read one full page even if only 16 bytes are requested?)? > > I'm not aware of any docs, but again, keep in mind that Cygwin is a > usersapce DLL. We basically do what Windows does for low-level file > access. > > > Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that > > mean the file's block size is really 64k? Is this info per filesystem, > > or hardcoded in Cygwin? > > Hardcoded in Cygwin since 2017, based on a discussion in terms of > file access performance, especially when using stdio.h functions: > > https://cygwin.com/cgit/newlib-cygwin/commit/?id=3D7bef7db5ccd9c OUCH. While I can understand the motivation, FAT32 on multi-GB-devices having 64k block size, and Win32 API on Win95/98/ME/Win7 being optimized to that insane block size, it is absolutely WRONG with today's NTFS and even more so with ReFS. This only works if you stream files, but as soon as you are doing random read/writes the performance is terrible due to cache thrashing. That could explain the many complaints about Cygwin's IO performance. So, what can be done? I'm not a benchmarking guru, so I'd like to propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to the CYGWIN env variable (marked as "experimental"), so the benchmarking guys can do performance testing without recompiling everything, get perf results for Cygwin 3.6, and decide what to do for Cygwin 3.7. Thanks, Martin