[ECOS] ISR to DSR delay?

[ECOS] ISR to DSR delay?

[Stefan Sommerfeld]

Hi,

I working with latest eCos on a XScale system and i noticed a very high 
delay between the ISR and the corresponding after some time. The system is 
not idle and does a lot of things (including multiple threads and irq's). I 
see a delay between ISR and DSR over 30 ms.

I checked the system's DSR, to make sure there's no DSR which runs very 
long and i looked for possible cyg_scheduler_lock() calls. What i wonder 
about is, i see that sometimes a quick cyg_scheduler_lock() happen which 
between the long ISR/DSR delay. So it looks like scheduling is active and 
running but the DSR is not called.

What could influence the ISR to DSR delay besides scheduler_lock and still 
active DSR? I thought a DSR will be called as soon as possible. Could this 
be a wrong eCos kernel setup?

Any help will be usefull....

Bye...

BTW: Why is "c++ empty delete function" a default option? I took me a day 
to find out why my system consumes a lot memory. 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ISR to DSR delay?

[Andrew Lunn]

On Thu, Oct 13, 2005 at 02:42:58PM +0200, Stefan Sommerfeld wrote:
> BTW: Why is "c++ empty delete function" a default option? I took me a day 
> to find out why my system consumes a lot memory. 

Its a philisophical thing. 

Small footprint embedded systems, which is what eCos is targeted to,
tend to use static allocated memory. That way they know they have
enough memory either at compile time, or just after the system has
started running when it allocates all the memory it needs for the rest
of its life. This also makes the code simpler and smaller. You don't
need to worry about running out of memory if you survive the startup
phase. It is not normal to free memory since objects live until the
system crashes/reboots. In such systems delete() is just a waste of
space, but the linker is unlikely to be able to throw it away. So an
empty implementation is used.

Now i guess you are not using eCos for what its primarily targeted
to. If you are using C++ with dynamically allocated objects you are
probably on a big fat footprint embedded system.

What i suppose we could do to help people in your situation is that if
INFRA_DEBUG is enabled add a counter to count the number of times
delete() is called. If it exceeds 100 throw an assert. When the assert
goes off and you start investigating it, it will quickly become
obvious what you need to do....

        Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ISR to DSR delay?

[Stefan Sommerfeld]

Hi,

> On Thu, Oct 13, 2005 at 02:42:58PM +0200, Stefan Sommerfeld wrote:
>> BTW: Why is "c++ empty delete function" a default option? I took me a 
>> day
>> to find out why my system consumes a lot memory.
>
> Its a philisophical thing.
>
> Small footprint embedded systems, which is what eCos is targeted to,
> tend to use static allocated memory. That way they know they have
> enough memory either at compile time, or just after the system has
> started running when it allocates all the memory it needs for the rest
> of its life. This also makes the code simpler and smaller. You don't
> need to worry about running out of memory if you survive the startup
> phase. It is not normal to free memory since objects live until the
> system crashes/reboots. In such systems delete() is just a waste of
> space, but the linker is unlikely to be able to throw it away. So an
> empty implementation is used.

Of course, i agree it's an important function for embedded system, but i 
would suggest that it's not by default enabled.

> Now i guess you are not using eCos for what its primarily targeted
> to. If you are using C++ with dynamically allocated objects you are
> probably on a big fat footprint embedded system.

I'm using eCos as a RTOS ... XScale 520MHz with 64MB RAM :)

>
> What i suppose we could do to help people in your situation is that if
> INFRA_DEBUG is enabled add a counter to count the number of times
> delete() is called. If it exceeds 100 throw an assert. When the assert
> goes off and you start investigating it, it will quickly become
> obvious what you need to do....

That would be an option ....

Bye... 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ISR to DSR delay?

[Andrew Lunn]

On Thu, Oct 13, 2005 at 03:23:57PM +0200, Stefan Sommerfeld wrote:
> Hi,
> 
> >On Thu, Oct 13, 2005 at 02:42:58PM +0200, Stefan Sommerfeld wrote:
> >>BTW: Why is "c++ empty delete function" a default option? I took me a 
> >>day
> >>to find out why my system consumes a lot memory.
> >
> >Its a philisophical thing.
> >
> >Small footprint embedded systems, which is what eCos is targeted to,
> >tend to use static allocated memory. That way they know they have
> >enough memory either at compile time, or just after the system has
> >started running when it allocates all the memory it needs for the rest
> >of its life. This also makes the code simpler and smaller. You don't
> >need to worry about running out of memory if you survive the startup
> >phase. It is not normal to free memory since objects live until the
> >system crashes/reboots. In such systems delete() is just a waste of
> >space, but the linker is unlikely to be able to throw it away. So an
> >empty implementation is used.
> 
> Of course, i agree it's an important function for embedded system, but i 
> would suggest that it's not by default enabled.
> 
> >Now i guess you are not using eCos for what its primarily targeted
> >to. If you are using C++ with dynamically allocated objects you are
> >probably on a big fat footprint embedded system.
> 
> I'm using eCos as a RTOS ... XScale 520MHz with 64MB RAM :)

So a very very fat footprint system. I expect most systems running
eCos are around 1MB-2MBs of RAM.

> >
> >What i suppose we could do to help people in your situation is that if
> >INFRA_DEBUG is enabled add a counter to count the number of times
> >delete() is called. If it exceeds 100 throw an assert. When the assert
> >goes off and you start investigating it, it will quickly become
> >obvious what you need to do....
> 
> That would be an option ....

I will add it to my ToDo list.

        Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ISR to DSR delay?

[Andrew Lunn]

On Thu, Oct 13, 2005 at 02:42:58PM +0200, Stefan Sommerfeld wrote:
> Hi,
> 
> I working with latest eCos on a XScale system and i noticed a very high 
> delay between the ISR and the corresponding after some time. The system is 
> not idle and does a lot of things (including multiple threads and irq's). I 
> see a delay between ISR and DSR over 30 ms.
> 
> I checked the system's DSR, to make sure there's no DSR which runs very 
> long and i looked for possible cyg_scheduler_lock() calls. What i wonder 
> about is, i see that sometimes a quick cyg_scheduler_lock() happen which 
> between the long ISR/DSR delay. So it looks like scheduling is active and 
> running but the DSR is not called.
> 
> What could influence the ISR to DSR delay besides scheduler_lock and still 
> active DSR? I thought a DSR will be called as soon as possible. Could this 
> be a wrong eCos kernel setup?

DSRs are only allowed to run with it is safe to run a DSR. This in
fact means that when the schedular is locked DSRs cannot run. Once the
schedular is unlocked the DSR will get to run. 

So the schedular is probably locked when the ISR happens and you see a
delay. When the ISR exits the DSR does not get to run. When you see a
cyg_scheduler_lock() this will actually be a nested lock inside
another lock. So the DSR will only get to run when both locks are
unlocked. This might actually be telling you something. A double lock
suggests something more complex is going on. It might be worth seeing
if a single lock gives you an acceptable delay but a double lock is
unacceptable. If so you can then just find the double lock case and
simply the code. Otherwise you need to investigate all locks and find
onces which are taking too long.

        Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] ISR to DSR delay?

[Stefan Sommerfeld]

Hi,

Progress update:

I'm still searching, but find a strange thing: In the irq_end routine 
(intr.cxx:323) there's a Cyg_Scheduler::unlock(), which should run pending 
dsr's. I think there're to possible things which this function could to:
Decrease the scheduler lock and return and Decrease and call unlock_inner() 
to call pending dsr's. What i see is that this ::unlock() take sometimes 
more than 6ms, but the unlock_inner() is always fast (some microseconds). 
So what could be the cause? Maybe a context switch while unlocking? Any 
hints?

Bye...

----- Original Message ----- 
From: "Andrew Lunn" <andrew@lunn.ch>
To: "Stefan Sommerfeld" <sommerfeld@mikrom.de>
Cc: <ecos-discuss@ecos.sourceware.org>
Sent: Donnerstag, 13. Oktober 2005 14:14
Subject: Re: [ECOS] ISR to DSR delay?


> On Thu, Oct 13, 2005 at 02:42:58PM +0200, Stefan Sommerfeld wrote:
>> Hi,
>>
>> I working with latest eCos on a XScale system and i noticed a very high
>> delay between the ISR and the corresponding after some time. The system 
>> is
>> not idle and does a lot of things (including multiple threads and 
>> irq's). I
>> see a delay between ISR and DSR over 30 ms.
>>
>> I checked the system's DSR, to make sure there's no DSR which runs very
>> long and i looked for possible cyg_scheduler_lock() calls. What i wonder
>> about is, i see that sometimes a quick cyg_scheduler_lock() happen which
>> between the long ISR/DSR delay. So it looks like scheduling is active 
>> and
>> running but the DSR is not called.
>>
>> What could influence the ISR to DSR delay besides scheduler_lock and 
>> still
>> active DSR? I thought a DSR will be called as soon as possible. Could 
>> this
>> be a wrong eCos kernel setup?
>
> DSRs are only allowed to run with it is safe to run a DSR. This in
> fact means that when the schedular is locked DSRs cannot run. Once the
> schedular is unlocked the DSR will get to run.
>
> So the schedular is probably locked when the ISR happens and you see a
> delay. When the ISR exits the DSR does not get to run. When you see a
> cyg_scheduler_lock() this will actually be a nested lock inside
> another lock. So the DSR will only get to run when both locks are
> unlocked. This might actually be telling you something. A double lock
> suggests something more complex is going on. It might be worth seeing
> if a single lock gives you an acceptable delay but a double lock is
> unacceptable. If so you can then just find the double lock case and
> simply the code. Otherwise you need to investigate all locks and find
> onces which are taking too long.
>
>        Andrew
>
> -- 
> Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
> and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss
> 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss