lwIP port status

lwIP port status

[Simon Kallweit]

Hi

It's time for a little status update on the lwIP 1.3.x port. Let me give 
you a little list of what has been done lately:

* CDL should include every configuration option offered by lwIP now, was 
reviewed by John Dallaway and got restructured and cleaned up afterwards.
* Implemented 'Sequential' mode.
* Updated to the latest lwIP sources from CVS again.
* Reviewed test cases.
* Cleanup, cleanup, cleanup.

What's left to do is lots of testing and writing documentation etc. I 
think it would be great to finish the port when lwIP hits the 1.3.1 
release which should not be too far from now. If time permits, I may 
also work a bit on PPP.

For now I would like to shortly discuss my 'Sequential' implementation. 
In 'Sequential' mode, lwIP is configured with NO_SYS == 0 and allows for 
multiple threads to use the stack (netcomm, sockets). The main thread 
(tcpip) is basically used to synchronize all threads accessing the stack 
by sequentially processing incoming messages on a message queue. When a 
packet arrives on an interface, a message pointing to the packet is sent 
to the message queue. This is what is done with the SLIP interface for 
example. SLIP is running in it's own thread, reading from the serial 
device (blocked). When a packet is received, it is passed to the tcpip 
thread by posting a message to the message queue. This way the packet is 
processed in sync with the tcpip thread. For ethernet devices I choose 
another path. The old port implemented a separate ethernet delivery 
thread which would wait on a semaphore for packets to arrive. It would 
wake up when a packet arrives and read it from the interface and send a 
message to the tcpip thread. In my port I got rid of the delivery thread 
and implemented sending/receiving as follows:

When a packet is sent, lwIP assumes that after the call to output the 
packet, the packet is no longer access by the driver and can be 
released. This means that the packet should be sent before the call 
returns -> blocking, or packets would have to be copied and put in a 
queue, which I think does not make much sense. I therefore implemented 
sending to be blocking.

Receiving packets could be done in parallel to the tcpip thread, but 
packets still have to be processed in sync to the tcpip thread. I choose 
a really simple route. When a packet arrives, a callback is registered 
with the tcpip thread, which is called back as soon as ongoing 
processing in the tcpip thread is finished. The callback then reads all 
the pending packets from the ethernet devices and directly processes them.

What do people think of this approach? Is it feasible or completely 
broken? :)

Anybody cares to do some testing of the new lwIP port?

Simon

Re: lwIP port status

[MaxiD]

Simon Kallweit wrote:
> 
> Hi
> 
> It's time for a little status update on the lwIP 1.3.x port. Let me give 
> you a little list of what has been done lately:
> 
> * CDL should include every configuration option offered by lwIP now, was 
> reviewed by John Dallaway and got restructured and cleaned up afterwards.
> * Implemented 'Sequential' mode.
> * Updated to the latest lwIP sources from CVS again.
> * Reviewed test cases.
> * Cleanup, cleanup, cleanup.
> 
> What's left to do is lots of testing and writing documentation etc. I 
> think it would be great to finish the port when lwIP hits the 1.3.1 
> release which should not be too far from now. If time permits, I may 
> also work a bit on PPP.
> 
> For now I would like to shortly discuss my 'Sequential' implementation. 
> In 'Sequential' mode, lwIP is configured with NO_SYS == 0 and allows for 
> multiple threads to use the stack (netcomm, sockets). The main thread 
> (tcpip) is basically used to synchronize all threads accessing the stack 
> by sequentially processing incoming messages on a message queue. When a 
> packet arrives on an interface, a message pointing to the packet is sent 
> to the message queue. This is what is done with the SLIP interface for 
> example. SLIP is running in it's own thread, reading from the serial 
> device (blocked). When a packet is received, it is passed to the tcpip 
> thread by posting a message to the message queue. This way the packet is 
> processed in sync with the tcpip thread. For ethernet devices I choose 
> another path. The old port implemented a separate ethernet delivery 
> thread which would wait on a semaphore for packets to arrive. It would 
> wake up when a packet arrives and read it from the interface and send a 
> message to the tcpip thread. In my port I got rid of the delivery thread 
> and implemented sending/receiving as follows:
> 
> When a packet is sent, lwIP assumes that after the call to output the 
> packet, the packet is no longer access by the driver and can be 
> released. This means that the packet should be sent before the call 
> returns -> blocking, or packets would have to be copied and put in a 
> queue, which I think does not make much sense. I therefore implemented 
> sending to be blocking.
> 
> Receiving packets could be done in parallel to the tcpip thread, but 
> packets still have to be processed in sync to the tcpip thread. I choose 
> a really simple route. When a packet arrives, a callback is registered 
> with the tcpip thread, which is called back as soon as ongoing 
> processing in the tcpip thread is finished. The callback then reads all 
> the pending packets from the ethernet devices and directly processes them.
> 
> What do people think of this approach? Is it feasible or completely 
> broken? :)
> 
> Anybody cares to do some testing of the new lwIP port?
> 
> Simon
> 
> 

Hello Simon,

I get errors with mboxes with older versions so I would test your port.
Would you please send me the link to your port??

Thanks & best regards
Max
-- 
View this message in context: http://www.nabble.com/lwIP-port-status-tp23598717p24603788.html
Sent from the Sourceware - ecos-devel mailing list archive at Nabble.com.

Re: lwIP port status

[Simon Kallweit]

MaxiD wrote:
> Hello Simon,
> 
> I get errors with mboxes with older versions so I would test your port.
> Would you please send me the link to your port??

I put a snapshot up:

http://download.westlicht.ch/lwip-20090722.tar.gz

I'm glad for any feedback. lwip is close to the 1.3.1 release, and I 
think this will be the time to get the new port into the eCos 
repository, so any testing is highly appreciated.

Simon

Re: lwIP port status

[John Dallaway]

Hi Simon

Simon Kallweit wrote:

> lwip is close to the 1.3.1 release, and I think this will be the time
> to get the new port into the eCos repository, so any testing is highly
> appreciated.

lwIP 1.3.1 was finally released today.

In view of the known issues with the older eCos lwIP 1.1.1 port and the
many fixes to the master lwIP sources over the intervening years, I'm
keen to get this committed to eCos CVS as soon as possible.

Having reviewed the lwIP ChangeLog, I think it makes sense to simply
replace the older lwIP port in the eCos CVS trunk (tagging the
repository before the commit). There are known issues with lwIP PPP
which need to be addressed upstream. In the meantime, eCos users wishing
to work with lwIP PPP should be better off hacking on the lwIP 1.3.1
stack where necessary rather than using the older port.

What is the status of your testing of your lwIP 1.3.x port now? Are you
comfortable that it is stable under heavy CPU and network loads?

John Dallaway

Re: lwIP port status

[Simon Kallweit]

John Dallaway schrieb:
> Hi Simon
>
> Simon Kallweit wrote:
>
>   
>> lwip is close to the 1.3.1 release, and I think this will be the time
>> to get the new port into the eCos repository, so any testing is highly
>> appreciated.
>>     
>
> lwIP 1.3.1 was finally released today.
>   

Yes, I've read it on the mailing list.

> In view of the known issues with the older eCos lwIP 1.1.1 port and the
> many fixes to the master lwIP sources over the intervening years, I'm
> keen to get this committed to eCos CVS as soon as possible.
>
> Having reviewed the lwIP ChangeLog, I think it makes sense to simply
> replace the older lwIP port in the eCos CVS trunk (tagging the
> repository before the commit). There are known issues with lwIP PPP
> which need to be addressed upstream. In the meantime, eCos users wishing
> to work with lwIP PPP should be better off hacking on the lwIP 1.3.1
> stack where necessary rather than using the older port.
>   

I think this is the way to go. There is already some hacking on PPP in 
my port, which could lead to some confusion, as it is not identical to 
the lwIP codebase. Also it does only work in a polled environment like 
mine, so it might not be much use of the broad community. I don't know 
how we should proceed with this?

> What is the status of your testing of your lwIP 1.3.x port now? Are you
> comfortable that it is stable under heavy CPU and network loads?
>   

Well, the tests I have been doing were successful. But I could only test 
on the synthetic target as my hardware lacks ethernet support. Some 
broader testing on real hardware would be great.

I'll merge the latest changes from the lwip 1.3.1 release and make a new 
release for review and testing.

Simon

Re: lwIP port status

[John Dallaway]

Hi Simon and all

Simon Kallweit wrote:

> John Dallaway schrieb:
>
>> Having reviewed the lwIP ChangeLog, I think it makes sense to simply
>> replace the older lwIP port in the eCos CVS trunk (tagging the
>> repository before the commit). There are known issues with lwIP PPP
>> which need to be addressed upstream. In the meantime, eCos users wishing
>> to work with lwIP PPP should be better off hacking on the lwIP 1.3.1
>> stack where necessary rather than using the older port.
> 
> I think this is the way to go. There is already some hacking on PPP in
> my port, which could lead to some confusion, as it is not identical to
> the lwIP codebase. Also it does only work in a polled environment like
> mine, so it might not be much use of the broad community. I don't know
> how we should proceed with this?

Perhaps it would be preferable to publish your PPP-related changes as a
patch for the time being and keep the CVS sources as close to the master
lwIP code as possible.

>> What is the status of your testing of your lwIP 1.3.x port now? Are you
>> comfortable that it is stable under heavy CPU and network loads?
> 
> Well, the tests I have been doing were successful. But I could only test
> on the synthetic target as my hardware lacks ethernet support. Some
> broader testing on real hardware would be great.

I can certainly test on an x86 PC target. I think Sergei Gavrikov has
also been experimenting with your port (probably on ARM). Are there any
volunteers to perform a basic sanity check of the eCos port on a
big-endian target?

> I'll merge the latest changes from the lwip 1.3.1 release and make a new
> release for review and testing.

Great.

Thank you

John Dallaway

Re: lwIP port status

[Sergei Gavrikov]

On Wed, Aug 19, 2009 at 08:11:09AM +0100, John Dallaway wrote:
> I can certainly test on an x86 PC target. I think Sergei Gavrikov has
> also been experimenting with your port (probably on ARM). Are there any
> volunteers to perform a basic sanity check of the eCos port on a
> big-endian target?

Hello John, Simon

Unfortunately, I ran in some issues when I tried lwip-20090722 on ARM
http://ecos.sourceware.org/ml/ecos-discuss/2009-07/msg00223.html It was
not possible to build lwip tests because new package had not a few
interface functions are used by io/eth/lwip/current/src/eth_drv.c
lwip_set_addr(), lwip_dhcp_init(), lwip_dsr_stuff(). Old eCos lwip
package has those implementations. Perhaps, I missed something. May be I
have to implement those in HAL? AFAIR I tweaked something in simple.c to
compile lwip_tcpip itself. My "memory sticks" are at home. May be I had
installed/built the package/tests in a wrong way. I used eCos CVS. The
steps were

rm -r $ECOS_REPOSITORY/net/lwip_tcpip
cp -a net/lwip_tcpip $ECOS_REPOSITORY/net
ecosconfig new <target> lwip_eth
...

Well, I did stop my attempts then. Waiting for new Simon stuff. Simon,
if I missed some things, forgive my stupidity. Both, thank you for your
persistence and your efforts!

Regards

Sergei

Re: lwIP port status

[Simon Kallweit]

John Dallaway wrote:
> Hi Simon and all
> 
> Simon Kallweit wrote:
> 
>> John Dallaway schrieb:
>>
>>> Having reviewed the lwIP ChangeLog, I think it makes sense to simply
>>> replace the older lwIP port in the eCos CVS trunk (tagging the
>>> repository before the commit). There are known issues with lwIP PPP
>>> which need to be addressed upstream. In the meantime, eCos users wishing
>>> to work with lwIP PPP should be better off hacking on the lwIP 1.3.1
>>> stack where necessary rather than using the older port.
>> I think this is the way to go. There is already some hacking on PPP in
>> my port, which could lead to some confusion, as it is not identical to
>> the lwIP codebase. Also it does only work in a polled environment like
>> mine, so it might not be much use of the broad community. I don't know
>> how we should proceed with this?
> 
> Perhaps it would be preferable to publish your PPP-related changes as a
> patch for the time being and keep the CVS sources as close to the master
> lwIP code as possible.

That sounds like a good idea. I'll look into it.

I'll do the merge with the newest lwip sources tomorrow and release the 
package.

Simon

Re: lwIP port status

[Simon Kallweit]

Sergei Gavrikov wrote:
> On Wed, Aug 19, 2009 at 08:11:09AM +0100, John Dallaway wrote:
>> I can certainly test on an x86 PC target. I think Sergei Gavrikov has
>> also been experimenting with your port (probably on ARM). Are there any
>> volunteers to perform a basic sanity check of the eCos port on a
>> big-endian target?
> 
> Hello John, Simon
> 
> Unfortunately, I ran in some issues when I tried lwip-20090722 on ARM
> http://ecos.sourceware.org/ml/ecos-discuss/2009-07/msg00223.html It was
> not possible to build lwip tests because new package had not a few
> interface functions are used by io/eth/lwip/current/src/eth_drv.c
> lwip_set_addr(), lwip_dhcp_init(), lwip_dsr_stuff(). Old eCos lwip
> package has those implementations. Perhaps, I missed something. May be I
> have to implement those in HAL? AFAIR I tweaked something in simple.c to
> compile lwip_tcpip itself. My "memory sticks" are at home. May be I had
> installed/built the package/tests in a wrong way. I used eCos CVS. The
> steps were
> 
> rm -r $ECOS_REPOSITORY/net/lwip_tcpip
> cp -a net/lwip_tcpip $ECOS_REPOSITORY/net
> ecosconfig new <target> lwip_eth
> ...
> 
> Well, I did stop my attempts then. Waiting for new Simon stuff. Simon,
> if I missed some things, forgive my stupidity. Both, thank you for your
> persistence and your efforts!

Well, that's my fault. I have also changed the generic ethernet driver 
for my lwip port. I'll publish these sources with the lwip package tomorrow.

Simon