Severe performance degradation of writev

Severe performance degradation of writev

[jojelino]

2008-07-27  Corinna Vinschen  <corinna@vinschen.de>

        * fhandler_socket.cc (fhandler_socket::send_internal): Send 
never more
        then 64K bytes at once.  For blocking sockets, loop until entire 
data
        has been sent or an error occurs.
        (fhandler_socket::sendto): Drop code which sends on 64K bytes.
        (fhandler_socket::sendmsg): Ditto.

This commit added workaround for KB823764. but it has brought another 
performance issue when writev sends <64k of data.
Execute following command shows the problem.
svnrdump dump --incremental http://svn.apache.org/repos/asf/subversion 
subversion
cygwin does split writev request into many WSASendTo call and serf 
library sets TCP_NODELAY for socket it uses, a WSASendTo call 
corresponds to a tcp packet.
You can see that http header is sent being splitted when you executed 
above command. Whereas win32 backend of apr library doesn't exhibit such 
behavior by using send call.
-- 
Regards.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Severe performance degradation of writev

[jojelino]

On 2014-07-07 AM 7:28, jojelino wrote:
> svnrdump dump --incremental http://svn.apache.org/repos/asf/subversion
> subversion
Instead of above the wrong one, this
svnrdump dump --incremental http://svn.apache.org/repos/asf/subversion > 
/dev/null

-- 
Regards.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Severe performance degradation of writev

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 903 bytes --]

On Jul  7 07:28, jojelino wrote:
> 2008-07-27  Corinna Vinschen  <corinna@vinschen.de>
> 
>        * fhandler_socket.cc (fhandler_socket::send_internal): Send never
> more
>        then 64K bytes at once.  For blocking sockets, loop until entire data
>        has been sent or an error occurs.
>        (fhandler_socket::sendto): Drop code which sends on 64K bytes.
>        (fhandler_socket::sendmsg): Ditto.
> 
> This commit added workaround for KB823764. but it has brought another
> performance issue when writev sends <64k of data.

That's why the code contains that FIXME comment.  If you have a good
idea for simple code to split a message into the least number of pieces
to minimize the number of WsaSendTo calls...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Severe performance degradation of writev

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

On Jul  7 12:10, Corinna Vinschen wrote:
> On Jul  7 07:28, jojelino wrote:
> > 2008-07-27  Corinna Vinschen  <corinna@vinschen.de>
> > 
> >        * fhandler_socket.cc (fhandler_socket::send_internal): Send never
> > more
> >        then 64K bytes at once.  For blocking sockets, loop until entire data
> >        has been sent or an error occurs.
> >        (fhandler_socket::sendto): Drop code which sends on 64K bytes.
> >        (fhandler_socket::sendmsg): Ditto.
> > 
> > This commit added workaround for KB823764. but it has brought another
> > performance issue when writev sends <64k of data.
> 
> That's why the code contains that FIXME comment.  If you have a good
> idea for simple code to split a message into the least number of pieces
> to minimize the number of WsaSendTo calls...

I took a stab at the code and I think the new version improves writing
multiple small buffers a lot.  In my testing it still works in other
scenarios, too, but I would be very grateful if somebody could have a
critical look into my code changes as posted in
https://cygwin.com/ml/cygwin-cvs/2014-q3/msg00003.html

I uploaded a new developer snapshot to http://cygwin.com/snapshots/
Please give it a thorough try.


Thanks,
Corinna





-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Severe performance degradation of writev

[David Rothenberger]

Corinna Vinschen wrote:
> On Jul  7 12:10, Corinna Vinschen wrote:
>> On Jul  7 07:28, jojelino wrote:
>>> 2008-07-27  Corinna Vinschen  <corinna@vinschen.de>
>>> 
>>> * fhandler_socket.cc (fhandler_socket::send_internal): Send
>>> never more then 64K bytes at once.  For blocking sockets, loop
>>> until entire data has been sent or an error occurs. 
>>> (fhandler_socket::sendto): Drop code which sends on 64K bytes. 
>>> (fhandler_socket::sendmsg): Ditto.
>>> 
>>> This commit added workaround for KB823764. but it has brought
>>> another performance issue when writev sends <64k of data.
>> 
>> That's why the code contains that FIXME comment.  If you have a
>> good idea for simple code to split a message into the least
>> number of pieces to minimize the number of WsaSendTo calls...
> 
> I took a stab at the code and I think the new version improves
> writing multiple small buffers a lot.  In my testing it still works
> in other scenarios, too, but I would be very grateful if somebody
> could have a critical look into my code changes as posted in 
> https://cygwin.com/ml/cygwin-cvs/2014-q3/msg00003.html
> 
> I uploaded a new developer snapshot to
> http://cygwin.com/snapshots/ Please give it a thorough try.

FWIW, this snapshot fixed a recent performance degradation for me when
doing ssh/rsync transfers within my local network at work. These
transfers had run at about 25 MB/s but recently degraded to about 500
kB/s. The snapshot restored the original performance.


-- 
David Rothenberger  ----  daveroth@acm.org

Issawi's Laws of Progress:
        The Course of Progress:
                Most things get steadily worse.
        The Path of Progress:
                A shortcut is the longest distance between two points.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Severe performance degradation of writev

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 1920 bytes --]

On Jul  9 09:44, David Rothenberger wrote:
> Corinna Vinschen wrote:
> > On Jul  7 12:10, Corinna Vinschen wrote:
> >> On Jul  7 07:28, jojelino wrote:
> >>> 2008-07-27  Corinna Vinschen  <corinna@vinschen.de>
> >>> 
> >>> * fhandler_socket.cc (fhandler_socket::send_internal): Send
> >>> never more then 64K bytes at once.  For blocking sockets, loop
> >>> until entire data has been sent or an error occurs. 
> >>> (fhandler_socket::sendto): Drop code which sends on 64K bytes. 
> >>> (fhandler_socket::sendmsg): Ditto.
> >>> 
> >>> This commit added workaround for KB823764. but it has brought
> >>> another performance issue when writev sends <64k of data.
> >> 
> >> That's why the code contains that FIXME comment.  If you have a
> >> good idea for simple code to split a message into the least
> >> number of pieces to minimize the number of WsaSendTo calls...
> > 
> > I took a stab at the code and I think the new version improves
> > writing multiple small buffers a lot.  In my testing it still works
> > in other scenarios, too, but I would be very grateful if somebody
> > could have a critical look into my code changes as posted in 
> > https://cygwin.com/ml/cygwin-cvs/2014-q3/msg00003.html
> > 
> > I uploaded a new developer snapshot to
> > http://cygwin.com/snapshots/ Please give it a thorough try.
> 
> FWIW, this snapshot fixed a recent performance degradation for me when
> doing ssh/rsync transfers within my local network at work. These
> transfers had run at about 25 MB/s but recently degraded to about 500
> kB/s. The snapshot restored the original performance.

Cool.  Is the result still intact?  It's kind of simple to have
lots of performance if the code just doesn't send everything... :}


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Severe performance degradation of writev

[David Rothenberger]

Corinna Vinschen wrote:
> On Jul  9 09:44, David Rothenberger wrote:
>> Corinna Vinschen wrote:
>>> On Jul  7 12:10, Corinna Vinschen wrote:
>>>> On Jul  7 07:28, jojelino wrote:
>>>>> 2008-07-27  Corinna Vinschen  <corinna@vinschen.de>
>>>>> 
>>>>> * fhandler_socket.cc (fhandler_socket::send_internal):
>>>>> Send never more then 64K bytes at once.  For blocking
>>>>> sockets, loop until entire data has been sent or an error
>>>>> occurs. (fhandler_socket::sendto): Drop code which sends on
>>>>> 64K bytes. (fhandler_socket::sendmsg): Ditto.
>>>>> 
>>>>> This commit added workaround for KB823764. but it has
>>>>> brought another performance issue when writev sends <64k of
>>>>> data.
>>>> 
>>>> That's why the code contains that FIXME comment.  If you have
>>>> a good idea for simple code to split a message into the
>>>> least number of pieces to minimize the number of WsaSendTo
>>>> calls...
>>> 
>>> I took a stab at the code and I think the new version improves 
>>> writing multiple small buffers a lot.  In my testing it still
>>> works in other scenarios, too, but I would be very grateful if
>>> somebody could have a critical look into my code changes as
>>> posted in 
>>> https://cygwin.com/ml/cygwin-cvs/2014-q3/msg00003.html
>>> 
>>> I uploaded a new developer snapshot to 
>>> http://cygwin.com/snapshots/ Please give it a thorough try.
>> 
>> FWIW, this snapshot fixed a recent performance degradation for me
>> when doing ssh/rsync transfers within my local network at work.
>> These transfers had run at about 25 MB/s but recently degraded to
>> about 500 kB/s. The snapshot restored the original performance.
> 
> Cool.  Is the result still intact?  It's kind of simple to have 
> lots of performance if the code just doesn't send everything... :}

Details, details. :)

Yeah, I just tried a transfer and verified the checksum of the
transferred file. It worked fine.


-- 
David Rothenberger  ----  daveroth@acm.org

Hempstone's Question:
        If you have to travel on the Titanic, why not go first class?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Severe performance degradation of writev

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 2290 bytes --]

On Jul  9 10:17, David Rothenberger wrote:
> Corinna Vinschen wrote:
> > On Jul  9 09:44, David Rothenberger wrote:
> >> Corinna Vinschen wrote:
> >>> On Jul  7 12:10, Corinna Vinschen wrote:
> >>>> On Jul  7 07:28, jojelino wrote:
> >>>>> 2008-07-27  Corinna Vinschen  <corinna@vinschen.de>
> >>>>> 
> >>>>> * fhandler_socket.cc (fhandler_socket::send_internal):
> >>>>> Send never more then 64K bytes at once.  For blocking
> >>>>> sockets, loop until entire data has been sent or an error
> >>>>> occurs. (fhandler_socket::sendto): Drop code which sends on
> >>>>> 64K bytes. (fhandler_socket::sendmsg): Ditto.
> >>>>> 
> >>>>> This commit added workaround for KB823764. but it has
> >>>>> brought another performance issue when writev sends <64k of
> >>>>> data.
> >>>> 
> >>>> That's why the code contains that FIXME comment.  If you have
> >>>> a good idea for simple code to split a message into the
> >>>> least number of pieces to minimize the number of WsaSendTo
> >>>> calls...
> >>> 
> >>> I took a stab at the code and I think the new version improves 
> >>> writing multiple small buffers a lot.  In my testing it still
> >>> works in other scenarios, too, but I would be very grateful if
> >>> somebody could have a critical look into my code changes as
> >>> posted in 
> >>> https://cygwin.com/ml/cygwin-cvs/2014-q3/msg00003.html
> >>> 
> >>> I uploaded a new developer snapshot to 
> >>> http://cygwin.com/snapshots/ Please give it a thorough try.
> >> 
> >> FWIW, this snapshot fixed a recent performance degradation for me
> >> when doing ssh/rsync transfers within my local network at work.
> >> These transfers had run at about 25 MB/s but recently degraded to
> >> about 500 kB/s. The snapshot restored the original performance.
> > 
> > Cool.  Is the result still intact?  It's kind of simple to have 
> > lots of performance if the code just doesn't send everything... :}
> 
> Details, details. :)

Yeah, yeah, I know, I'm nitpicking here ;)

> Yeah, I just tried a transfer and verified the checksum of the
> transferred file. It worked fine.

Thanks!


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]