[ECOS] Few questions on lwip stack

[ECOS] Few questions on lwip stack

[Alexander Aganichev]

Hello,

I'm trying to create TCP/IP application on custom LPC2468 based board
which should have fast response time. Tried BSD stack, but found poor
performance, so switched to lwip. Until friday I used previous lwip
version and all was good except of the fact each IP packet had PUSH
flag set which cause some slow down on PC side. At friday I decided to
upgrade eCos tree and all the problems came on :)

I replaced lwip_init() with cyg_lwip_sequential_init(), and started
thoose IP dealing threads with cyg_lwip_thread_new().

Now I have a problem that lwip sometimes ignores IP frames from PC
which cause PC to retransmit frame after two seconds which is not
acceptable. This happens not under heavy load. Never saw anything
similar on previous lwip version. The response time on ICMP is very
good and I don't see any delay or ignorance...  Just wondering if
anyone saw something similar and have ideas where to dig?

Another issue I found, is that lwip sends garbage on Ethernet after it
"cannot allocate pbuf to receive packet". I tested it on my custom
echo test (but I believe it should be reproducible on default echo
test) under the load created from HyperTerminal "send text" function.
This should not happen in my app since it is based on request-response
protocol with the fairly small packets, but someone may run into this.

WBR, Alexander

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] Re: Few questions on lwip stack

[Alexander Aganichev]

On Tue, Nov 16, 2010 at 6:18 PM, Alexander Aganichev
<aaganichev@gmail.com> wrote:
> Now I have a problem that lwip sometimes ignores IP frames from PC
> which cause PC to retransmit frame after two seconds which is not
> acceptable. This happens not under heavy load. Never saw anything
> similar on previous lwip version. The response time on ICMP is very
> good and I don't see any delay or ignorance...  Just wondering if
> anyone saw something similar and have ideas where to dig?

It looks like tcp_tmr() is not executed in sequential mode and this
cause the problem. Can it just be started from tcpip_thread() like
other timers?

WBR, Alexander

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Few questions on lwip stack

[Sergei Gavrikov]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1995 bytes --]

On Wed, 17 Nov 2010, Alexander Aganichev wrote:

> On Tue, Nov 16, 2010 at 6:18 PM, Alexander Aganichev
> <aaganichev@gmail.com> wrote:
>> Now I have a problem that lwip sometimes ignores IP frames from PC
>> which cause PC to retransmit frame after two seconds which is not
>> acceptable. This happens not under heavy load. Never saw anything
>> similar on previous lwip version. The response time on ICMP is very
>> good and I don't see any delay or ignorance...  Just wondering if
>> anyone saw something similar and have ideas where to dig?
>
> It looks like tcp_tmr() is not executed in sequential mode and this
> cause the problem. Can it just be started from tcpip_thread() like
> other timers?

Hi,

Can you provide your ecm file/test? Of curiosity I tried eCos `tcpecho'
test under some load and I did not observe the same fading.

Having no real H/W  I used something virtual

   eCos:     CVS updated
   Host:     Linux
   Target:   pc_rltk8139
   Template: lwip_eth
   Startup:  RAM
   Hardware: QEMU
   Test:     tcpecho
   Run land: GDB (i386-elf-gdb)

   Only changed:

   cdl_option CYGFUN_LWIP_MODE_SIMPLE {
       user_value 0
   };
   cdl_option CYGFUN_LWIP_MODE_SEQUENTIAL {
       user_value 1
   };

   Stress Fu:

   % strings /dev/urandom | nc host port   ;# port was mapped on 7th

I saw l-o-o-o-ng and continuous 'matrix' in terminal. No fading. After
STOP/CONT (I got a resumption of the run as I expected) Well, it was
lazy and not accurate test, it was virtual H/W  (but real S/W) and I
could not reproduce the issue for my environment. However, I guess the
packets could be lost (I was looking only for any long fading and I did
not plan to dive in the dumps of network traffic). Also, I call that
default CDL options (like mine) cannot be suitable for stress conditions
(i.e.  default sizes of net buffers, the stack sizes, etc.).  Seek the
posts near the time of porting new LWIP version, please, I hope that
Simon gave some advices.

Sergei

> WBR, Alexander

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Few questions on lwip stack

[Alexander Aganichev]

On Wed, Nov 17, 2010 at 11:56 PM, Sergei Gavrikov
<sergei.gavrikov@gmail.com> wrote:
> On Wed, 17 Nov 2010, Alexander Aganichev wrote:
>
>> On Tue, Nov 16, 2010 at 6:18 PM, Alexander Aganichev
>> <aaganichev@gmail.com> wrote:
>>>
>>> Now I have a problem that lwip sometimes ignores IP frames from PC
>>> which cause PC to retransmit frame after two seconds which is not
>>> acceptable. This happens not under heavy load. Never saw anything
>>> similar on previous lwip version. The response time on ICMP is very
>>> good and I don't see any delay or ignorance...  Just wondering if
>>> anyone saw something similar and have ideas where to dig?
>>
>> It looks like tcp_tmr() is not executed in sequential mode and this
>> cause the problem. Can it just be started from tcpip_thread() like
>> other timers?
>
> Hi,
>
> Can you provide your ecm file/test? Of curiosity I tried eCos `tcpecho'
> test under some load and I did not observe the same fading.
>
> Having no real H/W  I used something virtual
>
>  eCos:     CVS updated
>  Host:     Linux
>  Target:   pc_rltk8139
>  Template: lwip_eth
>  Startup:  RAM
>  Hardware: QEMU
>  Test:     tcpecho
>  Run land: GDB (i386-elf-gdb)
>
>  Only changed:
>
>  cdl_option CYGFUN_LWIP_MODE_SIMPLE {
>      user_value 0
>  };
>  cdl_option CYGFUN_LWIP_MODE_SEQUENTIAL {
>      user_value 1
>  };
>
>  Stress Fu:
>
>  % strings /dev/urandom | nc host port   ;# port was mapped on 7th
>
> I saw l-o-o-o-ng and continuous 'matrix' in terminal. No fading. After
> STOP/CONT (I got a resumption of the run as I expected) Well, it was
> lazy and not accurate test, it was virtual H/W  (but real S/W) and I
> could not reproduce the issue for my environment. However, I guess the
> packets could be lost (I was looking only for any long fading and I did
> not plan to dive in the dumps of network traffic). Also, I call that
> default CDL options (like mine) cannot be suitable for stress conditions
> (i.e.  default sizes of net buffers, the stack sizes, etc.).

My target board is quite slow on Ethernet and HyperTerminal broke
sending data in multiple small packets, so probably that's why I was
able to reach point where no more pbufs could be allocated.

> Seek the posts near the time of porting new LWIP version, please, I hope that
> Simon gave some advices.

It looks like the root of my problem is in the way I created tasks. I
used POSIX message queues and POSIX threads. After switching I left
POSIX threads that's why timers do not work for me. Tried to create
threads with cyg_lwip_thread_new() but only first thread was
started... Tried to rewrite cyg_lwip_thread_new() to create POSIX
threads instead, but also no luck... The only way I was able to start
all the threads was locking task scheduler before creating and
unlocking after all threads are created. Just thinking what to do
best: continue with modifying cyg_lwip_thread_new() or just get rid of
message queues.

WBR, Alexander

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Few questions on lwip stack

[Alexander Aganichev]

On Thu, Nov 18, 2010 at 2:27 PM, Alexander Aganichev
<aaganichev@gmail.com> wrote:
>> Seek the posts near the time of porting new LWIP version, please, I hope that
>> Simon gave some advices.
>
> It looks like the root of my problem is in the way I created tasks. I
> used POSIX message queues and POSIX threads. After switching I left
> POSIX threads that's why timers do not work for me. Tried to create
> threads with cyg_lwip_thread_new() but only first thread was
> started... Tried to rewrite cyg_lwip_thread_new() to create POSIX
> threads instead, but also no luck... The only way I was able to start
> all the threads was locking task scheduler before creating and
> unlocking after all threads are created. Just thinking what to do
> best: continue with modifying cyg_lwip_thread_new() or just get rid of
> message queues.

Finally I get it working more or less. I rewrote lwip to use POSIX
threads if they enabled (just wondering if anyone wants it to be
published?), increased stack sizes (looks like default too low if you
enable debug output for PHY), increased number of timers (though I do
not use my own, the default number is not enough for two threads that
use LWIP), increased some other buffers.

Now I have another problem. I'm sending about 64K of data and
sometimes time between two consecutive TCP frames reach about 40-60
msec what does not look like any timer I'm aware of. When I look to
the Ethereal output LWIP always response quite fast, but if it need to
send two consecutive frames without acknowledge from the PC it
_sometimes_ takes time. I have 100 Mbit dedicated network, so there's
no influence from other traffic. Can anyone advise if I can change
something to increase speed of transmission?

WBR, Alexander

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Few questions on lwip stack

[Sergei Gavrikov]

On Tue, 23 Nov 2010, Alexander Aganichev wrote:
> I rewrote lwip to use POSIX threads if they enabled (just wondering if
> anyone wants it to be published?), increased stack sizes (looks like
> default too low if you enable debug output for PHY), increased number
> of timers (though I do not use my own, the default number is not
> enough for two threads that use LWIP), increased some other buffers.

Hi,

What does mean 'rewrote lwip'? Does it mean that you have a patch for
current lwip package to use POSIX threads or you worked on CDL config
options only? What were your steps? As we have not some kind of 'eCos
Magazine' it would be fine to share your experience on the list (IMO).

Sergei

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Few questions on lwip stack

[Alexander Aganichev]

On Wed, Nov 24, 2010 at 3:40 PM, Sergei Gavrikov
<sergei.gavrikov@gmail.com> wrote:
> On Tue, 23 Nov 2010, Alexander Aganichev wrote:
>> I rewrote lwip to use POSIX threads if they enabled (just wondering if
>> anyone wants it to be published?), increased stack sizes (looks like
>> default too low if you enable debug output for PHY), increased number
>> of timers (though I do not use my own, the default number is not
>> enough for two threads that use LWIP), increased some other buffers.
>
> Hi,
>
> What does mean 'rewrote lwip'? Does it mean that you have a patch for
> current lwip package to use POSIX threads or you worked on CDL config
> options only? What were your steps? As we have not some kind of 'eCos
> Magazine' it would be fine to share your experience on the list (IMO).

Oops, wrong term. Yes, I patched lwip's layer to use POSIX threads
instead of native threads if eCos configured to use POSIX threads (so,
no explicit CDL option). I'm just not sure if anyone wants it. I can
post it to bugzilla if you want. All other things was done based on
options only. It looks like some options should be better documented
so other people will not spend time trying to figure out why it does
not work for them.

I have few other improvements over eCos which need some CDL code (for
example, LCP2468 require copying transmission buffers to internal
memory if we work with external RAM due to too low speed of transfer)
or review from someone more familiar with eCos internals.

Best regards,
Alexander

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Few questions on lwip stack

[Sergei Gavrikov]

On Wed, 24 Nov 2010, Alexander Aganichev wrote:

> On Wed, Nov 24, 2010 at 3:40 PM, Sergei Gavrikov
> <sergei.gavrikov@gmail.com> wrote:
>> On Tue, 23 Nov 2010, Alexander Aganichev wrote:
>>> I rewrote lwip to use POSIX threads if they enabled (just wondering if
>>> anyone wants it to be published?), increased stack sizes (looks like
>>> default too low if you enable debug output for PHY), increased number
>>> of timers (though I do not use my own, the default number is not
>>> enough for two threads that use LWIP), increased some other buffers.
>>
>> Hi,
>>
>> What does mean 'rewrote lwip'? Does it mean that you have a patch for
>> current lwip package to use POSIX threads or you worked on CDL config
>> options only? What were your steps? As we have not some kind of 'eCos
>> Magazine' it would be fine to share your experience on the list (IMO).
>
> Oops, wrong term. Yes, I patched lwip's layer to use POSIX threads
> instead of native threads if eCos configured to use POSIX threads (so,
> no explicit CDL option). I'm just not sure if anyone wants it. I can
> post it to bugzilla if you want. All other things was done based on
> options only. It looks like some options should be better documented
> so other people will not spend time trying to figure out why it does
> not work for them.

It would be great if you find time to post your workaround (patch and
ecm) in eCos Bugzilla database. Why not? 

> I have few other improvements over eCos which need some CDL code (for
> example, LCP2468 require copying transmission buffers to internal
> memory if we work with external RAM due to too low speed of transfer)
> or review from someone more familiar with eCos internals.

Any enhancements are welcome. You're not trying to change the interrupt
priority for LPC Ethernet driver? AFAIK, default value is non-priority
value.

Yes, by the way, if you have crossed lwIP with POSIX, can you build lwIP
net tests (with POSIX threads?) and run it against eCos Net throughput
tests (tcp_source, tcp_sink, etc.)? As I remember you started with BSD
net stack and POSIX threads. Just wondering: Have you got a performance
boost? Or how much faster lwIP stack by your  subjective experience?

Thank you,
Sergei

> Best regards,
> Alexander
>
> -- 
> Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
> and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss
>

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss