[ECOS] Problem compiling newnet with timing stats

[ECOS] Problem compiling newnet with timing stats

[Paul Randall]

Hi all,

I am having problems compiling the newnet (FreeBSD) template with
CYGDBG_NET_TIMING_STATS turned on in eCos 2.0.  I can compile and run
the original TCP/IP stack (OpenBSD) with timing stats with no problem.
During the make for newnet, I get the following errors:

In file included from
/ecos-c/cygwin/src/ecos2a/ecos/newnet/install/include/sys/param.h:820,
from
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:56:

/ecos-c/cygwin/src/ecos2a/ecos/newnet/install/include/cyg/kernel/kapi.h:360:
warning: function declaration isn't a prototype
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:
In function `cyg_net_malloc':
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:197:
parse error before `start_time'
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:197:
`start_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:197:
(Each undeclared identifier is reported only once
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:197:
for each function it appears in.)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:203:
`end_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:203:
`elapsed_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:
In function `cyg_net_mbuf_alloc':
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:222:
parse error before `start_time'
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:222:
`start_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:229:
`end_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:229:
`elapsed_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:
In function `cyg_net_cluster_alloc':
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:251:
parse error before `start_time'
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:251:
`start_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:253:
`end_time' undeclared (first use in this function)
/ecos-c/cygwin/src/ecos2a/ecos/packages/net/bsd_tcpip/current/src/ecos/support.c:253:
`elapsed_time' undeclared (first use in this function)
make[1]: *** [src/ecos/support.o.d] Error 1
make[1]: Leaving directory
`/src/ecos2a/ecos/newnet/net/bsd_tcpip/current'
make: *** [build] Error 2

It looks like the compile errors are coming from the START_STATS() and
FINISH_STATS() macros in param.h:

#ifdef CYGDBG_NET_TIMING_STATS
#define START_STATS()                                   \
    cyg_uint32 start_time, end_time, elapsed_time;      \
    HAL_CLOCK_READ(&start_time);
#define
FINISH_STATS(stats)
\

HAL_CLOCK_READ(&end_time);
\
    if (end_time < start_time)
{                                                        \
        elapsed_time = (end_time+CYGNUM_KERNEL_COUNTERS_RTC_PERIOD) -
start_time;       \
    } else
{
\
        elapsed_time = end_time -
start_time;                                           \

}
\
    if (stats.min_time == 0)
{                                                          \
        stats.min_time =
0x7FFFFFFF;                                                    \

}
\
    if (elapsed_time <
stats.min_time)                                                  \
        stats.min_time =
elapsed_time;                                                  \
    if (elapsed_time >
stats.max_time)                                                  \
        stats.max_time =
elapsed_time;                                                  \
    stats.total_time +=
elapsed_time;                                                   \
    stats.count++;
#else
#define START_STATS()
#define FINISH_STATS(X)
#endif

I tried moving the "cyg_uint32 start_time, end_time, elapsed_time;" line
before the "#define START_STATS()" and the template compiled ok, but
when I went to run a test, my viper locked up during the ethernet
initialization.

Any suggestions?

Paul Randall
Delta Information Systems


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Problem compiling newnet with timing stats

[Gary Thomas]

On Fri, 2002-06-14 at 15:03, Paul Randall wrote:
> Hi all,
> 
> I am having problems compiling the newnet (FreeBSD) template with
> CYGDBG_NET_TIMING_STATS turned on in eCos 2.0.  I can compile and run
> the original TCP/IP stack (OpenBSD) with timing stats with no problem.
> During the make for newnet, I get the following errors:
  <snip>

Don't forget that the FreeBSD stack is still alpha - some things haven't
been well shaken out and this is obviously one of them.

Exactly what information were you interested in getting?  The particular
statistics that CYGDBG_NET_TIMING_STATS enables are quite low level and
really only deal with some of the internal memory handling of the stack.
They don't provide much in the way of general information about the
networking system as a whole.



-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Problem compiling newnet with timing stats

[Gary Thomas]

On Fri, 2002-06-14 at 15:03, Paul Randall wrote:
> Hi all,
> 
> I am having problems compiling the newnet (FreeBSD) template with
> CYGDBG_NET_TIMING_STATS turned on in eCos 2.0.  I can compile and run
> the original TCP/IP stack (OpenBSD) with timing stats with no problem.
> During the make for newnet, I get the following errors:
> 
 ...
> Any suggestions?

Just by way of completeness, this patch will fix the compile errors:

Index: net/bsd_tcpip/current/ChangeLog
===================================================================
RCS file: /cvs/ecos/ecos-opt/net/net/bsd_tcpip/current/ChangeLog,v
retrieving revision 1.2
diff -u -5 -p -r1.2 ChangeLog
--- net/bsd_tcpip/current/ChangeLog	5 Jun 2002 16:37:52 -0000	1.2
+++ net/bsd_tcpip/current/ChangeLog	14 Jun 2002 21:34:43 -0000
@@ -1,5 +1,10 @@
+2002-06-14  Gary Thomas  <gary@chez-thomas.org>
+
+	* src/ecos/support.c: 
+	Fix compile errors when CYGDBG_NET_TIMING_STATS enabled.
+
 2002-06-05  Gary Thomas  <gary@chez-thomas.org>
 
 	* include/netinet/in.h: Rename 'ip_opts' field (using the same
 	name as the structure is illegal in C++).
 
Index: net/bsd_tcpip/current/src/ecos/support.c
===================================================================
RCS file: /cvs/ecos/ecos-opt/net/net/bsd_tcpip/current/src/ecos/support.c,v
retrieving revision 1.1
diff -u -5 -p -r1.1 support.c
--- net/bsd_tcpip/current/src/ecos/support.c	20 May 2002 22:25:01 -0000	1.1
+++ net/bsd_tcpip/current/src/ecos/support.c	14 Jun 2002 21:34:43 -0000
@@ -191,12 +191,12 @@ show_net_times(void)
 void *
 cyg_net_malloc(u_long size, int type, int flags)
 {
     void *res;
 
-    log(LOG_MDEBUG, "Net malloc[%d] = ", size);
     START_STATS();
+    log(LOG_MDEBUG, "Net malloc[%d] = ", size);
     if (flags & M_NOWAIT) {
         res = cyg_mempool_var_try_alloc(net_mem, size);
     } else {
         res = cyg_mempool_var_alloc(net_mem, size);
     }
@@ -216,12 +216,12 @@ cyg_net_free(caddr_t addr, int type)
 void *
 cyg_net_mbuf_alloc(int type, int flags)
 {
     void *res;    
 
-    log(LOG_MDEBUG, "Alloc mbuf = ");
     START_STATS();
+    log(LOG_MDEBUG, "Alloc mbuf = ");
     mbstat.m_mbufs++;
     if (flags & M_NOWAIT) {
         res = cyg_mempool_fix_try_alloc(net_mbufs);
     } else {
         res = cyg_mempool_fix_alloc(net_mbufs);
@@ -245,12 +245,13 @@ cyg_net_mbuf_free(caddr_t addr, int type
 
 void *
 cyg_net_cluster_alloc(void)
 {
     void *res;
-    log(LOG_MDEBUG, "Allocate cluster = ");
+
     START_STATS();
+    log(LOG_MDEBUG, "Allocate cluster = ");
     res = cyg_mempool_fix_try_alloc(net_clusters);
     FINISH_STATS(stats_cluster_alloc);
     log(LOG_MDEBUG, "%p\n", res);
     return res;
 }


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Problem compiling newnet with timing stats

[Paul Randall]

Gary Thomas wrote:

> On Fri, 2002-06-14 at 15:03, Paul Randall wrote:
> > Hi all,
> >
> > I am having problems compiling the newnet (FreeBSD) template with
> > CYGDBG_NET_TIMING_STATS turned on in eCos 2.0.  I can compile and run
> > the original TCP/IP stack (OpenBSD) with timing stats with no problem.
> > During the make for newnet, I get the following errors:
>   <snip>
>
> Don't forget that the FreeBSD stack is still alpha - some things haven't
> been well shaken out and this is obviously one of them.
>
> Exactly what information were you interested in getting?  The particular
> statistics that CYGDBG_NET_TIMING_STATS enables are quite low level and
> really only deal with some of the internal memory handling of the stack.
> They don't provide much in the way of general information about the
> networking system as a whole.

I have been running some network performance tests with eCos running on the
viper.  Using the FreeBSD stack, I am able to get a throughput of 5.5 Mbps
with 1500 byte UDP packets in one direction from the viper to my host PC.
Using the OpenBSD stack, the rate is 4.7 Mbps.  The throughput is about the
same when running on an isolated 10 Mbps or a 100 Mbps network.

I was hoping that the timing stats would shed some light on where the
performance bottleneck is.  Running timing stats on the OpenBSD stack, I got
the following results from sending 1024 UDP packets:

Net malloc:
  count:     41, min:       5.45, max:      21.79, total:     562.44,
ave:      13.62
Net free:
  count:     12, min:      20.43, max:      28.60, total:     291.43,
ave:      23.15
Mbuf alloc:
  count:   3083, min:      14.98, max:     144.36, total:   56569.52,
ave:      17.70
Mbuf free:
  count:   3015, min:      13.62, max:     360.89, total:   58892.82,
ave:      19.07
Cluster alloc:
  count:     55, min:       6.81, max:      17.70, total:     875.66,
ave:      14.98
Checksum:
  count:   2050, min:       5.45, max:     281.90, total:  167672.61,
ave:      81.71
Net memcpy:
  count:      0, min:       0.00, max:       0.00, total:       0.00, ave:
-1370435568.18
Net memset:
  count:      0, min:       0.00, max:       0.00, total:       0.00, ave:
-1370435568.18

I'm not sure what the units are but the UDP and IP checksums occupy a large
percentage of the processing time.  I disabled the UDP checksum and was able
to increase the throughput to 6.0 Mbps which is still not as high as I need.

Paul Randall
Delta Information Systems


-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Problem compiling newnet with timing stats

[Andrew Lunn]

> I was hoping that the timing stats would shed some light on where
> the performance bottleneck is.  Running timing stats on the OpenBSD
> stack, I got the following results from sending 1024 UDP packets:

The stats themselves can add quite a bit of overhead for some of the
operations. When comparing OpenBSD and FreeBSD did you have them
turned off for OpenBSD?

> I'm not sure what the units are but the UDP and IP checksums occupy a large
> percentage of the processing time.  I disabled the UDP checksum and was able
> to increase the throughput to 6.0 Mbps which is still not as high as I need.

Be careful here. Things do get corrupt and passed the ethernet
checksum. Sun Microsystems had problems with there NFS servers getting
corrupt when they disabled UDP checksums. A better idea may be to
rewrite the code in assembly. Also look back in the mail
archive. There were a couple of posts about this a long time ago.

What's your platform again? I made some saving for ARM and i82559. See
bugzilla.redhat.com. Its not responding at the moment so i cannot give
you an exact reference. Search for my name and it should be obvious. I
optimized the memcpy from PCI space to normal space. The PCI space is
none cachable, none bufferable. Doing none word aligned reads or
writes to PCI space is very expensive compared to normal ram. So i
wrote a special memcpy function which does some bit twiddling so that
the PCI read/write is always word aligned. This gave between 2 and 3
times speedup for the memcpy in the driver. The code is not
generic. It assume little endian. Hence its not been merged into the
eCos repository. 

      Andrew

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss

Re: [ECOS] Problem compiling newnet with timing stats

[Gary Thomas]

On Mon, 2002-06-17 at 10:05, Andrew Lunn wrote:
> > I was hoping that the timing stats would shed some light on where
> > the performance bottleneck is.  Running timing stats on the OpenBSD
> > stack, I got the following results from sending 1024 UDP packets:
> 
> The stats themselves can add quite a bit of overhead for some of the
> operations. When comparing OpenBSD and FreeBSD did you have them
> turned off for OpenBSD?
> 
> > I'm not sure what the units are but the UDP and IP checksums occupy a large
> > percentage of the processing time.  I disabled the UDP checksum and was able
> > to increase the throughput to 6.0 Mbps which is still not as high as I need.
> 
> Be careful here. Things do get corrupt and passed the ethernet
> checksum. Sun Microsystems had problems with there NFS servers getting
> corrupt when they disabled UDP checksums. A better idea may be to
> rewrite the code in assembly. Also look back in the mail
> archive. There were a couple of posts about this a long time ago.
> 

The FreeBSD stack has a much better structure for improvement in just 
this area.  There are hooks for breaking the checksum calculations down 
and providing optimized code based on the architecture (currently only 
"C" code is used).

> What's your platform again? I made some saving for ARM and i82559. See
> bugzilla.redhat.com. Its not responding at the moment so i cannot give
> you an exact reference. Search for my name and it should be obvious. I
> optimized the memcpy from PCI space to normal space. The PCI space is
> none cachable, none bufferable. Doing none word aligned reads or
> writes to PCI space is very expensive compared to normal ram. So i
> wrote a special memcpy function which does some bit twiddling so that
> the PCI read/write is always word aligned. This gave between 2 and 3
> times speedup for the memcpy in the driver. The code is not
> generic. It assume little endian. Hence its not been merged into the
> eCos repository. 

Alas these changes won't help Paul much as he's working on a PowerPC 
based platform and the ethernet controller is part of the chip.

Another thing to consider is the other half of the connection.  How fast
is it?  Do you know anything about it's performance?  You can have a
blazingly fast network adapter on one end but the overall performance
ends up being miserable if it's connected to a slower device.

There may be other available tuning's, for example the number of device
send and receive buffers, etc.  

There are a whole lot of variables here, and only by examining them all 
and possibly making some adjustments can the performance be changed by 
much.




-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss