[ECOS] Non-stationary DSR and thread resuming latencies

[ECOS] Non-stationary DSR and thread resuming latencies

[Nikolay Marusov]

Dear all!

Does anyboby know why DSR latency and thead resuming time may SUDDENLY increase?
I have tested 2 main variants: immediate handler function call from
DSR and 2-stage handler invokation (resume handler thread in DSR).
Please see pictures below:
http://vo.nfi.kiae.ru/pub/eCos/peaks_82.png - DSR response time
http://vo.nfi.kiae.ru/pub/eCos/peaks_85.png - DSR response time
http://vo.nfi.kiae.ru/pub/eCos/peaks_83.png - Thread resuming from DSR
http://vo.nfi.kiae.ru/pub/eCos/peaks_84.png - Thread resuming from DSR
http://vo.nfi.kiae.ru/pub/eCos/peaks_86.png - DSR response time
without high-priority ISR (other interrupt channel)

I can latch time with microsecond resolution using FPGA periphery. In
our application the irregularity of rensponse time may be critical.
Beside all, it is interesting what may delay THE MOST PRIOTITY
interrupt handler and thread? Maybe eCos scheduler is not so
determinate?

Thanks,
Nikolay.

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] Non-stationary DSR and thread resuming latencies

[Christophe Coutand]

What about having a single interrupt source, will the ISR response time
be constant then?

Christophe


-----Original Message-----
From: ecos-discuss-owner@ecos.sourceware.org
[mailto:ecos-discuss-owner@ecos.sourceware.org] On Behalf Of Nikolay
Marusov
Sent: 20. mai 2010 07:54
To: ecos-discuss@ecos.sourceware.org
Subject: [ECOS] Non-stationary DSR and thread resuming latencies

Dear all!

Does anyboby know why DSR latency and thead resuming time may SUDDENLY
increase?
I have tested 2 main variants: immediate handler function call from
DSR and 2-stage handler invokation (resume handler thread in DSR).
Please see pictures below:
http://vo.nfi.kiae.ru/pub/eCos/peaks_82.png - DSR response time
http://vo.nfi.kiae.ru/pub/eCos/peaks_85.png - DSR response time
http://vo.nfi.kiae.ru/pub/eCos/peaks_83.png - Thread resuming from DSR
http://vo.nfi.kiae.ru/pub/eCos/peaks_84.png - Thread resuming from DSR
http://vo.nfi.kiae.ru/pub/eCos/peaks_86.png - DSR response time
without high-priority ISR (other interrupt channel)

I can latch time with microsecond resolution using FPGA periphery. In
our application the irregularity of rensponse time may be critical.
Beside all, it is interesting what may delay THE MOST PRIOTITY
interrupt handler and thread? Maybe eCos scheduler is not so
determinate?

Thanks,
Nikolay.

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss


--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Non-stationary DSR and thread resuming latencies

[Sergei Gavrikov]

On Thu, 20 May 2010, Nikolay Marusov wrote:
> Does anyboby know why DSR latency and thead resuming time may SUDDENLY
> increase?

Hi Nikolay,

There are important notes about influences on performance in the eCos
documentation:
http://ecos.sourceware.org/docs-latest/ref/kernel-characterization.html

So, what would pend your DSRs you have to dig by yourself.

[snip]

> Beside all, it is interesting what may delay THE MOST PRIOTITY
> interrupt handler and thread? Maybe eCos scheduler is not so
> determinate?

The eCos `tm_basic` test is a good source to get own testcase and get a
clue. First, what DSR/ISR latency/deviation you get for real-time clock
timer in the test for your target? (RT ISR handler has the highest ISR
priority by default).

Import the below (to measure and RT clock ISR latency), build and run
`tm_basic` test on your target.

   cdl_option CYGVAR_KERNEL_COUNTERS_CLOCK_LATENCY {
       user_value 1
   };

Perhaps, you need to increase NSAMPLES value to get more continuous run
of the test. Note: you may get differ results for the differ types of
startup (ROM, RAM. GDB, etc.) and differ compiler optimization levels.

Well, tm_basic with alone RT ISR/DSR != your_app. But looking on its
output (ideal world) you will be know what you would expect for real
world. I'm sorry, we can only guess about your DSR "issue". For example,
if your program has continuous sections where scheduler locks itself
those can pend your DSR sometimes, etc. Try bitmap scheduler if you can.
Investigate! :-)

Sergei

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Non-stationary DSR and thread resuming latencies

[Sergei Gavrikov]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4191 bytes --]

Please keep your replies on the mailing list(s) so that all may benefit.

On Tue, 25 May 2010, Nikolay Marusov wrote:
> Thank you, Sergei. Very useful reply.

Hi Nikolay,

> I cannot understand why the variance in some cases in such big?
>
> Benchmarking results:
>
> Board: Embedded Artists EA2468
> CPU:   LPC2468 (ARM7TDMI core), 48 MHz

[cut RedBoot's go]

> Reading the hardware clock takes 20 'ticks' overhead
> ... this value will be factored out of all other measurements
> Clock interrupt took   52.24 microseconds (2507 raw clock ticks)

[snip default tm prolog]

>                                 Confidence
>     Ave     Min     Max     Var  Ave  Min  Function
>  ======  ======  ======  ====== ========== ========
>    8.61    8.29    9.02    0.12   60%  10% Resume [suspended low prio]
> thread --------+
>    4.96    4.77    5.23    0.08   54%  15% Resume [runnable low prio]
> thread             |
>   87.37   27.19 3829.29  116.93   98%  98% Resume [high priority]
> thread    ------------ Very stange!

Your ASCII-art seems to be broken :-) But, I'm still seeing your narrows
and your question.

> Timing complete - 15675 ms total

And had not you changed default RT-clock resolution?

[eb40 table from eCos user guide is cut, that paste was broken at all]

> The results must be comparable. But what I see? High priority thread
> resuming delay may differ in 40 times!  I ask for some experience
> about what eCos packages may delay the sheduler in such dramatic way?

Any I/O package which uses locks intensively; broken hardware (spurious
interrupts, for example). *But*, when you build `tm_basic` test you use
a minimal set of the packages (How do you configure/build it?). Is that
~4mS delay on resume got for the default config options and `default`
template ? As far as I saw on top your mail RedBoot's `go` command, you
use RAM startup, don't you?  As I said, it would be interesting to know
for yourself what results for ROM startup are? Build `tm_basic` test for
ROM startup too. Certainly, if you can revert back RedBoot then. And
even build a set of the tm tests for a matrix

Optim: -O2, -O0, ?-Os?

Startup: ROM, RAM, ?ROMRAM?

eCos assertions: Disabled, Enabled

with *default* eCos config settings except that clock latency option.
And compare the results. BTW: among other things you can fill-in the
same matrix (for yourself) for eCos `dhrystone` test, and last thing:
check the bus timing value(s) for external RAM in your HAL startup.

Sergei

> Nikolay.
>
> 2010/5/20 Sergei Gavrikov <sergei.gavrikov@gmail.com>:
>> On Thu, 20 May 2010, Nikolay Marusov wrote:
>>>
>>> Does anyboby know why DSR latency and thead resuming time may SUDDENLY
>>> increase?
>>
>> Hi Nikolay,
>>
>> There are important notes about influences on performance in the eCos
>> documentation:
>> http://ecos.sourceware.org/docs-latest/ref/kernel-characterization.html
>>
>> So, what would pend your DSRs you have to dig by yourself.
>>
>> [snip]
>>
>>> Beside all, it is interesting what may delay THE MOST PRIOTITY
>>> interrupt handler and thread? Maybe eCos scheduler is not so
>>> determinate?
>>
>> The eCos `tm_basic` test is a good source to get own testcase and get a
>> clue. First, what DSR/ISR latency/deviation you get for real-time clock
>> timer in the test for your target? (RT ISR handler has the highest ISR
>> priority by default).
>>
>> Import the below (to measure and RT clock ISR latency), build and run
>> `tm_basic` test on your target.
>>
>>  cdl_option CYGVAR_KERNEL_COUNTERS_CLOCK_LATENCY {
>>      user_value 1
>>  };
>>
>> Perhaps, you need to increase NSAMPLES value to get more continuous run
>> of the test. Note: you may get differ results for the differ types of
>> startup (ROM, RAM. GDB, etc.) and differ compiler optimization levels.
>>
>> Well, tm_basic with alone RT ISR/DSR != your_app. But looking on its
>> output (ideal world) you will be know what you would expect for real
>> world. I'm sorry, we can only guess about your DSR "issue". For example,
>> if your program has continuous sections where scheduler locks itself
>> those can pend your DSR sometimes, etc. Try bitmap scheduler if you can.
>> Investigate! :-)
>>
>> Sergei

[-- Attachment #2: Type: text/plain, Size: 148 bytes --]

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss