Implementing aio_* and lio_* (async i/o) on Cygwin

Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

I've got a proof-of-concept implementation working but haven't integrated 
it into my local Cygwin source tree yet.  This implementation uses a pool 
of pthreads to issue reads and writes asynchronously.  Using threads 
allows to operate transparently with the usual Cygwin-supplied file 
descriptors/handles that are set up for synchronous-only operation.

If I were to use Win32 overlapped i/o, as Corinna suggested as a 
possibility, there would have to be some supporting changes to Cygwin 
internals, I think.  Either file descriptors would have to always permit 
overlapped usage, or open() and/or fcntl() would have to allow some new 
flag to specify overlapped capability when creating a new fd/handle.

There is an O_ASYNC flag in some non-POSIX Unixes but what little 
documentation I can find online makes it sound like an inexact match to 
the need here.  It's used to signal (via SIGIO arrival) that space is 
available for a write, or that data is available for a read.  What it 
would need to do for aio_* semantics is to signal that a write or a read 
has completed.  For disk files one *might* consider these two viewpoints 
equal but I'd like to hear if anybody thinks differently.

I'm also unsure one can add overlapped capability to an existing Win32 
handle.  If you can, then we could have fcntl() add or remove it from a fd 
on the fly.  If you can't add it to an existing handle, then an open() 
would have to ask for the capability when creating an fd/handle.

Wouldn't it be easier (less modification of existing code) to just use the 
thread model?

Any thoughts on any part of this would be appreciated.
Thanks,

..mark

P.S. Google search aio_read(), aio_write() man pages for background.
P.P.S. I could put my PoC code up on my GitHub space for inspection.

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 3122 bytes --]

On Dec 12 01:11, Mark Geisert wrote:
> I've got a proof-of-concept implementation working but haven't integrated it
> into my local Cygwin source tree yet.  This implementation uses a pool of
> pthreads to issue reads and writes asynchronously.  Using threads allows to
> operate transparently with the usual Cygwin-supplied file
> descriptors/handles that are set up for synchronous-only operation.
> 
> If I were to use Win32 overlapped i/o, as Corinna suggested as a
> possibility, there would have to be some supporting changes to Cygwin
> internals, I think.  Either file descriptors would have to always permit
> overlapped usage, or open() and/or fcntl() would have to allow some new flag
> to specify overlapped capability when creating a new fd/handle.

If you use threads, you will have to change your code to use cygthreads
instead of pthreads.  Pthreads are a userspace concept, cygthreads are
used for internal tasks.

However, using async IO, the only change necessary would be to change
prw_open to open an async handle, and pread/pwrite to wait for the
result if the current I/O is NOT async.  It may be necessary to add a
parameter or two to the pread/pwrite fhandler methods, but all the rest
of the functionality could be in the callers.

Please stop thinking in Win32 overlapped I/O.  I have no idea why
Microsoft uses this term.  Under the ntdll hood, a handle is either
synchronous (Keeping track of file position, waiting for read/write to
complete) or asynchronous (Not keeping track of file position, returning
STATUS_PENDING rather than waiting for completion, based on using the
FILE_SYNCHRONOUS_IO_{NON}ALERT.

On async handles, you can wait for an event object to be signalled, or
you can ask the NtReadFile/NtWriteFile functions to call a completion
routine.

> There is an O_ASYNC flag in some non-POSIX Unixes but what little
> documentation

No, if we had to create a new open flag for this functionality it would
be wrong.

> need here.  It's used to signal (via SIGIO arrival) that space is available
> for a write, or that data is available for a read.  What it would need to do
> for aio_* semantics is to signal that a write or a read has completed.

Yes, but with an arbitrary signal.  Alternatively aio can start a
pthread on completion, basically all the stuff sigevent is supposed to
handle.

> I'm also unsure one can add overlapped capability to an existing Win32
> handle.

That's what prw_open is for.

> Wouldn't it be easier (less modification of existing code) to just use the
> thread model?

Not sure.  You will have to change the threading stuff anyway and what's
working in user space is pretty different from how you do it in Cygwin.
Also, wouldn't it be nice to use the methods already provided by the OS
for just such an opportunity?  Changing the pread/pwrite/prw_open
methods to support aio_read/aio_write should be pretty straightforward.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

Hi Corinna,

On Tue, 12 Dec 2017, Corinna Vinschen wrote:
> On Dec 12 01:11, Mark Geisert wrote:
>> I've got a proof-of-concept implementation working but haven't integrated it
>> into my local Cygwin source tree yet.  This implementation uses a pool of
>> pthreads to issue reads and writes asynchronously.  Using threads allows to
>> operate transparently with the usual Cygwin-supplied file
>> descriptors/handles that are set up for synchronous-only operation.
>>
>> If I were to use Win32 overlapped i/o, as Corinna suggested as a
>> possibility, there would have to be some supporting changes to Cygwin
>> internals, I think.  Either file descriptors would have to always permit
>> overlapped usage, or open() and/or fcntl() would have to allow some new flag
>> to specify overlapped capability when creating a new fd/handle.
>
> If you use threads, you will have to change your code to use cygthreads
> instead of pthreads.  Pthreads are a userspace concept, cygthreads are
> used for internal tasks.

Understood in general.  I had originally implemented a libaio that was 
built on top of Cygwin, so there was an "app" viewpoint in my thinking. 
But I'm coming to understand what you're talking about: really doing aio_* 
and lio_* as Cygwin syscalls using the same capabilities within Cygwin 
that other syscalls use.  So cygthreads it is, only if they're needed.

> However, using async IO, the only change necessary would be to change
> prw_open to open an async handle, and pread/pwrite to wait for the
> result if the current I/O is NOT async.  It may be necessary to add a
> parameter or two to the pread/pwrite fhandler methods, but all the rest
> of the functionality could be in the callers.
>
> Please stop thinking in Win32 overlapped I/O.  I have no idea why
> Microsoft uses this term.  Under the ntdll hood, a handle is either
> synchronous (Keeping track of file position, waiting for read/write to
> complete) or asynchronous (Not keeping track of file position, returning
> STATUS_PENDING rather than waiting for completion, based on using the
> FILE_SYNCHRONOUS_IO_{NON}ALERT.
>
> On async handles, you can wait for an event object to be signalled, or
> you can ask the NtReadFile/NtWriteFile functions to call a completion
> routine.

OK.  I understand.

>> There is an O_ASYNC flag in some non-POSIX Unixes but what little
>> documentation
>
> No, if we had to create a new open flag for this functionality it would
> be wrong.
>
>> need here.  It's used to signal (via SIGIO arrival) that space is available
>> for a write, or that data is available for a read.  What it would need to do
>> for aio_* semantics is to signal that a write or a read has completed.
>
> Yes, but with an arbitrary signal.  Alternatively aio can start a
> pthread on completion, basically all the stuff sigevent is supposed to
> handle.
>
>> I'm also unsure one can add overlapped capability to an existing Win32
>> handle.
>
> That's what prw_open is for.

Ah, the light of enlightenment has lit.

>> Wouldn't it be easier (less modification of existing code) to just use the
>> thread model?
>
> Not sure.  You will have to change the threading stuff anyway and what's
> working in user space is pretty different from how you do it in Cygwin.
> Also, wouldn't it be nice to use the methods already provided by the OS
> for just such an opportunity?  Changing the pread/pwrite/prw_open
> methods to support aio_read/aio_write should be pretty straightforward.

Yes, I've got the idea now.  I'll go work on this direction and report 
back when I have something, modulo holidays and stuff.

Right now I have a patch to implement sigtimedwait() since it's similar to 
the already implemented sigwait() and sigwaitinfo().  I also have a patch 
that extends support for getting and setting cygthread and pthread names 
via cygwin_external().  I'll submit these two patches shortly.

Thanks for the patient internals explanations, as always.

..mark

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

On Dec 12 22:48, Mark Geisert wrote:
> Hi Corinna,
> [...]
> Right now I have a patch to implement sigtimedwait() since it's similar to
> the already implemented sigwait() and sigwaitinfo().

Sorry for skipping over everything else, but that's really cool!

> I also have a patch
> that extends support for getting and setting cygthread and pthread names via
> cygwin_external().  I'll submit these two patches shortly.

That sounds a bit weird.  Why would you want to set cygthread names
via cygwin_external?  Those are not exposed to userspace.

And getting and setting pthread names is already implemented via
the Linux-like calls pthread_getname_np / pthread_setname_np...


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

On Wed, 13 Dec 2017, Corinna Vinschen wrote:
> On Dec 12 22:48, Mark Geisert wrote:
>> Hi Corinna,
>> [...]
>> Right now I have a patch to implement sigtimedwait() since it's similar to
>> the already implemented sigwait() and sigwaitinfo().
>
> Sorry for skipping over everything else, but that's really cool!

Thanks :), I needed it for the userspace aio_suspend() and lio_listio().

>> I also have a patch
>> that extends support for getting and setting cygthread and pthread names via
>> cygwin_external().  I'll submit these two patches shortly.
>
> That sounds a bit weird.  Why would you want to set cygthread names
> via cygwin_external?  Those are not exposed to userspace.
>
> And getting and setting pthread names is already implemented via
> the Linux-like calls pthread_getname_np / pthread_setname_np...

The thread name patch to cygwin_internal() [sic] is for the benefit of 
strace, the forthcoming cygmon, and any other utility that might want to 
display thread names of some other process.  The pthread part of the patch 
does use those functions you mentioned; the cygthread part uses the 
existing cygthread::name() and a new ::setname() I supply.

..mark

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

Corinna Vinschen wrote (a while back):
> However, using async IO, the only change necessary would be to change
> prw_open to open an async handle, and pread/pwrite to wait for the
> result if the current I/O is NOT async.  It may be necessary to add a
> parameter or two to the pread/pwrite fhandler methods, but all the rest
> of the functionality could be in the callers.

I see what you're suggesting; seems sly in a nice way :).

A small part I'm missing is in interfacing to the layer above this.  Are the 
aio_* functions supposed to be implemented as "real" syscalls (in syscalls.cc)? 
Or should they be implemented in their own aio.cc (which is where I have them 
ATM) and call pread()/pwrite() to do their dirty work?  I'm unsure how "central" 
a syscall has to be to merit syscalls.cc location.

Regardless of which file the code is in, I was thinking I should copy 
syscalls.cc's pread() contents into aio_read(), for example, to start with. 
Then add extra param(s) as needed to the fhandler method call as you suggested.

The problem with having aio_read() call pread() directly is how to tell pread() 
we want an async read.  That's why I was suggesting O_ASYNC previously, though I 
now understand that's the wrong direction to go.  What is the correct direction?
Thanks much,

..mark

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2085 bytes --]

On Mar 19 22:53, Mark Geisert wrote:
> Corinna Vinschen wrote (a while back):
> > However, using async IO, the only change necessary would be to change
> > prw_open to open an async handle, and pread/pwrite to wait for the
> > result if the current I/O is NOT async.  It may be necessary to add a
> > parameter or two to the pread/pwrite fhandler methods, but all the rest
> > of the functionality could be in the callers.
> 
> I see what you're suggesting; seems sly in a nice way :).
> 
> A small part I'm missing is in interfacing to the layer above this.  Are the
> aio_* functions supposed to be implemented as "real" syscalls (in
> syscalls.cc)? Or should they be implemented in their own aio.cc (which is
> where I have them ATM) and call pread()/pwrite() to do their dirty work?
> I'm unsure how "central" a syscall has to be to merit syscalls.cc location.

syscalls.cc is historical.  Cygwin has grown over a single file to keep
"syscalls" anyway, so having the interfaces in an aio.cc file doesn't
hurt.

> Regardless of which file the code is in, I was thinking I should copy
> syscalls.cc's pread() contents into aio_read(), for example, to start with.
> Then add extra param(s) as needed to the fhandler method call as you
> suggested.

That's definitely ok.  When I was talking about pread/pwrite in our
preliminary discussion on cygwin-patches, I was always talking about the
fhandler_disk_file::pread and fhandler_disk_file::pwrite methods in
fact, not the syscalls of the same name.  I'm sorry I was unclear there :}

> The problem with having aio_read() call pread() directly is how to tell
> pread() we want an async read.  That's why I was suggesting O_ASYNC
> previously, though I now understand that's the wrong direction to go.  What
> is the correct direction?

Looks like you're on the right track, just go ahead.  We can fix the
finer details later.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

Corinna Vinschen wrote:
> On Mar 19 22:53, Mark Geisert wrote:
>> Regardless of which file the code is in, I was thinking I should copy
>> syscalls.cc's pread() contents into aio_read(), for example, to start with.
>> Then add extra param(s) as needed to the fhandler method call as you
>> suggested.
>
> That's definitely ok.  When I was talking about pread/pwrite in our
> preliminary discussion on cygwin-patches, I was always talking about the
> fhandler_disk_file::pread and fhandler_disk_file::pwrite methods in
> fact, not the syscalls of the same name.  I'm sorry I was unclear there :}
[...]
> Looks like you're on the right track, just go ahead.  We can fix the
> finer details later.

OK, I've posted a 3-part patch set to cygwin-patches.  This is still a WIP so it 
doesn't yet include what's been decided above.  It currently queues async ops to 
cygthread worker threads.

I plan to update aio_read() and aio_write() to try to launch async ops inline, 
and only if they fail with ESPIPE would they be queued for worker thread action. 
  (E.g., ops on sockets or other devices without pread/pwrite support.)

I think parts 1 and 2 of the patch set are essentially finished, unless there 
are issues to be corrected.  But part 3 will need some more work done.
Thanks,

..mark

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2006 bytes --]

Hi Mark,

On Mar 28 22:57, Mark Geisert wrote:
> Corinna Vinschen wrote:
> > On Mar 19 22:53, Mark Geisert wrote:
> > > Regardless of which file the code is in, I was thinking I should copy
> > > syscalls.cc's pread() contents into aio_read(), for example, to start with.
> > > Then add extra param(s) as needed to the fhandler method call as you
> > > suggested.
> > 
> > That's definitely ok.  When I was talking about pread/pwrite in our
> > preliminary discussion on cygwin-patches, I was always talking about the
> > fhandler_disk_file::pread and fhandler_disk_file::pwrite methods in
> > fact, not the syscalls of the same name.  I'm sorry I was unclear there :}
> [...]
> > Looks like you're on the right track, just go ahead.  We can fix the
> > finer details later.
> 
> OK, I've posted a 3-part patch set to cygwin-patches.  This is still a WIP
> so it doesn't yet include what's been decided above.  It currently queues
> async ops to cygthread worker threads.
> 
> I plan to update aio_read() and aio_write() to try to launch async ops
> inline, and only if they fail with ESPIPE would they be queued for worker
> thread action.  (E.g., ops on sockets or other devices without pread/pwrite
> support.)
> 
> I think parts 1 and 2 of the patch set are essentially finished, unless
> there are issues to be corrected.  But part 3 will need some more work done.
> Thanks,

Thank you!  I'm a bit low on available time ATM so please be patient.
I'll check your patch as time permits.

A word in terms of your commit messages.  Just providing the fact in the
commit header is a bit low on details.  Don't be shy to improve your
commit messages with some details, what you did, why you did it, what to
look for.

Some testcase (here on cygwin-developers, not as patch) would be
nice, too.


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Implementing aio_* and lio_* (async i/o) on Cygwin

[Mark Geisert]

Hi Corinna,
I appreciate the update.  I'm about to debug a revised implementation that 
takes into account your most recent comments.  So...

On Tue, 3 Apr 2018, Corinna Vinschen wrote:
> Hi Mark,
>
> On Mar 28 22:57, Mark Geisert wrote:
>> Corinna Vinschen wrote:
>>> On Mar 19 22:53, Mark Geisert wrote:
>>>> Regardless of which file the code is in, I was thinking I should copy
>>>> syscalls.cc's pread() contents into aio_read(), for example, to start with.
>>>> Then add extra param(s) as needed to the fhandler method call as you
>>>> suggested.
>>>
>>> That's definitely ok.  When I was talking about pread/pwrite in our
>>> preliminary discussion on cygwin-patches, I was always talking about the
>>> fhandler_disk_file::pread and fhandler_disk_file::pwrite methods in
>>> fact, not the syscalls of the same name.  I'm sorry I was unclear there :}
>> [...]
>>> Looks like you're on the right track, just go ahead.  We can fix the
>>> finer details later.
>>
>> OK, I've posted a 3-part patch set to cygwin-patches.  This is still a WIP
>> so it doesn't yet include what's been decided above.  It currently queues
>> async ops to cygthread worker threads.
>>
>> I plan to update aio_read() and aio_write() to try to launch async ops
>> inline, and only if they fail with ESPIPE would they be queued for worker
>> thread action.  (E.g., ops on sockets or other devices without pread/pwrite
>> support.)
>>
>> I think parts 1 and 2 of the patch set are essentially finished, unless
>> there are issues to be corrected.  But part 3 will need some more work done.
>> Thanks,
>
> Thank you!  I'm a bit low on available time ATM so please be patient.
> I'll check your patch as time permits.

No worries.  Take the "part 3" patch with a large grain of salt if/when 
you get to it.

> A word in terms of your commit messages.  Just providing the fact in the
> commit header is a bit low on details.  Don't be shy to improve your
> commit messages with some details, what you did, why you did it, what to
> look for.

Yes, absolutely.  I neglected to say what each patch part does.  I seem to 
be stumbling over every patch posting convention one by one :-?.  Onward!

> Some testcase (here on cygwin-developers, not as patch) would be
> nice, too.

I do have a couple already.  One is a (fairly large) test app I have that 
times various methods of copying the heap from one process to a child. 
AIO is one of those methods.  I could whittle that down to something 
using only AIO.  And the Linux man page for aio(7) has a sample program 
that can test AIO on something other than disk files.
Thanks & Regards,

..mark

Re: Testing Posix asynchronous I/O (was: Implementing...)

[Mark Geisert]

Mark Geisert wrote:
> On Tue, 3 Apr 2018, Corinna Vinschen wrote:
>> Some testcase (here on cygwin-developers, not as patch) would be
>> nice, too.
>
> I do have a couple already.  One is a (fairly large) test app I have that times
> various methods of copying the heap from one process to a child. AIO is one of
> those methods.  I could whittle that down to something using only AIO.  And the
> Linux man page for aio(7) has a sample program that can test AIO on something
> other than disk files.

In addition to those two test programs, there's iozone.  That supports AIO 
operations but would need to be ported to Cygwin.  IMNSHO iozone needs a -NG 
re-write.  It is 5 source files, no header files, and 1K of its 30K lines are 
#ifdef's.  And what it calls a Windows build is actually a Cygwin (32-bits) 
build.  But it's there if needed, modulo some work porting it.

I'm using the Linux man page aio(7) example to debug the AIO code now for the 
non-diskfile case.

The "heap transfer" program I mentioned earlier, heapxfer, allows me to specify 
heap size and number of simultaneous AIOs.  Simple cases, such as staying within 
AIO_MAX AIOs, work fine.  I recently finished debugging a testcase writing 1GB 
of data to a file using 512 AIOs.  So the first AIO_MAX AIOs launched as inline 
AIOs, while the remainder were queued.  Then as worker threads became available, 
they launched inline AIOs themselves.  Found a couple of nits but it's working now.

Comments or questions welcome.  I'll keep going on the testing.
Cheers,

..mark

Re: Testing Posix asynchronous I/O

[Mark Geisert]

Updating my previous post...

>> On Tue, 3 Apr 2018, Corinna Vinschen wrote:
>>> Some testcase (here on cygwin-developers, not as patch) would be
>>> nice, too.
>>
>> I do have a couple already.  One is a (fairly large) test app I have that times
>> various methods of copying the heap from one process to a child. AIO is one of
>> those methods.  I could whittle that down to something using only AIO.  And the
>> Linux man page for aio(7) has a sample program that can test AIO on something
>> other than disk files.

The man page sample program, which does a single aio_read() on each file/device 
named in args, seems to work for all cases except the specific one demonstrated 
there, which is more than one /dev/stdin.  On Cygwin, satisfying the first 
aio_read() causes the second to be satisfied with no input.  On Linux, the 
program waits for each aio_read() to be satisfied in sequence.

> In addition to those two test programs, there's iozone.  That supports AIO
> operations but would need to be ported to Cygwin.  IMNSHO iozone needs a -NG
> re-write.  It is 5 source files, no header files, and 1K of its 30K lines are
> #ifdef's.  And what it calls a Windows build is actually a Cygwin (32-bits)
> build.  But it's there if needed, modulo some work porting it.

I have ported iozone to 64-bit Cygwin (not re-written) and I can see it will be 
very helpful in stress-testing the AIO code.  At the moment I'm debugging an odd 
strace message after many thousands of I/Os:
       0 [sig] iozone 12248 wait_sig: garbled signal pipe data nb 176, sig 0
which seems to say the code is internally sending "signal 0", but there's no 
obvious way that could be occurring.

> The "heap transfer" program I mentioned earlier, heapxfer, allows me to specify
> heap size and number of simultaneous AIOs.  Simple cases, such as staying within
> AIO_MAX AIOs, work fine.  I recently finished debugging a testcase writing 1GB
> of data to a file using 512 AIOs.  So the first AIO_MAX AIOs launched as inline
> AIOs, while the remainder were queued.  Then as worker threads became available,
> they launched inline AIOs themselves.  Found a couple of nits but it's working now.

Most recently with heapxfer I've been testing aio_write()s of 2047MB in 2047 
AIOs on my 2Core/4Thread system.  Found and fixed another obscure buglet.

I will be AFK June 15..25 but wanted to post status since it's been a while 
since my last posting.  Comments welcome but in any case I'll keep testing.
Thanks & Regards,

..mark

Re: Testing Posix asynchronous I/O

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2973 bytes --]

On Jun 12 01:18, Mark Geisert wrote:
> Updating my previous post...
> 
> > > On Tue, 3 Apr 2018, Corinna Vinschen wrote:
> > > > Some testcase (here on cygwin-developers, not as patch) would be
> > > > nice, too.
> > > 
> > > I do have a couple already.  One is a (fairly large) test app I have that times
> > > various methods of copying the heap from one process to a child. AIO is one of
> > > those methods.  I could whittle that down to something using only AIO.  And the
> > > Linux man page for aio(7) has a sample program that can test AIO on something
> > > other than disk files.
> 
> The man page sample program, which does a single aio_read() on each
> file/device named in args, seems to work for all cases except the specific
> one demonstrated there, which is more than one /dev/stdin.  On Cygwin,
> satisfying the first aio_read() causes the second to be satisfied with no
> input.  On Linux, the program waits for each aio_read() to be satisfied in
> sequence.
> 
> > In addition to those two test programs, there's iozone.  That supports AIO
> > operations but would need to be ported to Cygwin.  IMNSHO iozone needs a -NG
> > re-write.  It is 5 source files, no header files, and 1K of its 30K lines are
> > #ifdef's.  And what it calls a Windows build is actually a Cygwin (32-bits)
> > build.  But it's there if needed, modulo some work porting it.
> 
> I have ported iozone to 64-bit Cygwin (not re-written) and I can see it will
> be very helpful in stress-testing the AIO code.  At the moment I'm debugging
> an odd strace message after many thousands of I/Os:
>       0 [sig] iozone 12248 wait_sig: garbled signal pipe data nb 176, sig 0
> which seems to say the code is internally sending "signal 0", but there's no
> obvious way that could be occurring.

This is weird indeed.

> > The "heap transfer" program I mentioned earlier, heapxfer, allows me to specify
> > heap size and number of simultaneous AIOs.  Simple cases, such as staying within
> > AIO_MAX AIOs, work fine.  I recently finished debugging a testcase writing 1GB
> > of data to a file using 512 AIOs.  So the first AIO_MAX AIOs launched as inline
> > AIOs, while the remainder were queued.  Then as worker threads became available,
> > they launched inline AIOs themselves.  Found a couple of nits but it's working now.
> 
> Most recently with heapxfer I've been testing aio_write()s of 2047MB in 2047
> AIOs on my 2Core/4Thread system.  Found and fixed another obscure buglet.
> 
> I will be AFK June 15..25 but wanted to post status since it's been a while
> since my last posting.  Comments welcome but in any case I'll keep testing.
> Thanks & Regards,

No worries and enjoy your vacation.  Apart from the few nits I really
like the code you provided!


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]