From: Mark Geisert <mark@maxrnd.com>
To: cygwin@cygwin.com
Subject: Re: Cygwin&Win32 file prefetch, block sizes?
Date: Tue, 2 Apr 2024 18:10:16 -0700 [thread overview]
Message-ID: <af940815-81af-4ca5-8198-072d09dad23e@maxrnd.com> (raw)
In-Reply-To: <CANH4o6P8cts9TJgpdjR4mi+sj2YvuDa=d49XLcEVvYnRB81KRw@mail.gmail.com>
On 4/2/2024 3:35 PM, Martin Wege via Cygwin wrote:
> On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin
> <cygwin@cygwin.com> wrote:
>>
>> On Apr 2 02:04, Martin Wege via Cygwin wrote:
>>> Hello,
>>>
>>> Is there any document which describes how Cygwin and Win32 file
>>> prefetch and readahead work, and which sizes are used (e.g. always
>>> read one full page even if only 16 bytes are requested?)?
>>
>> I'm not aware of any docs, but again, keep in mind that Cygwin is a
>> usersapce DLL. We basically do what Windows does for low-level file
>> access.
>>
>>> Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that
>>> mean the file's block size is really 64k? Is this info per filesystem,
>>> or hardcoded in Cygwin?
>>
>> Hardcoded in Cygwin since 2017, based on a discussion in terms of
>> file access performance, especially when using stdio.h functions:
>>
>> https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c
>
> OUCH.
>
> While I can understand the motivation, FAT32 on multi-GB-devices
> having 64k block size, and Win32 API on Win95/98/ME/Win7 being
> optimized to that insane block size, it is absolutely WRONG with
> today's NTFS and even more so with ReFS. This only works if you stream
> files, but as soon as you are doing random read/writes the performance
> is terrible due to cache thrashing. That could explain the many
> complaints about Cygwin's IO performance.
No comment.
> So, what can be done? I'm not a benchmarking guru, so I'd like to
> propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to
> the CYGWIN env variable (marked as "experimental"), so the
> benchmarking guys can do performance testing without recompiling
> everything, get perf results for Cygwin 3.6, and decide what to do for
> Cygwin 3.7.
That kind of experiment is what folks who can build their own
cygwin1.dll might do. I doubt we'd want to make a run-time global disk
I/O strategy changer available like this, even temporarily.
What could make sense is enhancing Cygwin's posix_fadvise() to support
POSIX_FADV_RANDOM getting mapped to Windows' FILE_RANDOM_ACCESS flag.
Something like this is currently done for POSIX_FADV_SEQUENTIAL ->
FILE_SEQUENTIAL_ONLY. These are per-filedescriptor adjustments and due
to Windows limitations would apply to a whole file rather than having
the POSIX behavior of being settable for a byte range within a file.
SHTDI, PTC, and all that :-).
..mark
next prev parent reply other threads:[~2024-04-03 1:10 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-02 0:04 Martin Wege
2024-04-02 13:16 ` Corinna Vinschen
2024-04-02 22:35 ` Martin Wege
2024-04-03 1:10 ` Mark Geisert [this message]
2024-04-03 6:44 ` Cedric Blancher
2024-04-04 18:10 ` Brian Inglis
2024-04-03 5:21 ` Cedric Blancher
2024-04-03 8:14 ` Corinna Vinschen
2024-04-03 11:44 ` Corinna Vinschen
2024-04-03 17:42 ` Cedric Blancher
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=af940815-81af-4ca5-8198-072d09dad23e@maxrnd.com \
--to=mark@maxrnd.com \
--cc=cygwin@cygwin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).