Re: [ECOS] Thread activation disturbed by lower priority threads]

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Alois Z.]

Hi,
> 
> How many mutex's does your lower priority thread hold? From
> packages/kernel/current/src/sched/sched.cxx
> 
> void Cyg_SchedThread::clear_inherited_priority()
> {
>     CYG_REPORT_FUNCTION();
> 
> #ifdef CYGSEM_KERNEL_SYNCH_MUTEX_PRIORITY_INVERSION_PROTOCOL_SIMPLE
> 
>     // A simple implementation of priority inheritance/ceiling
>     // protocols.  The simplification in this algorithm is that we do
>     // not reduce our priority until we have freed all mutexes
>     // claimed. Hence we can continue to run at an artificially high
>     // priority even when we should not.  However, since nested
>     // mutexes are rare, the thread we have inherited from is likely
>     // to be locking the same mutexes we are, and mutex claim periods
>     // should be very short, the performance difference between this
>     // and a more complex algorithm should be negligible. The most
>     // important advantage of this algorithm is that it is fast and
>     // deterministic.
> 
> Does this explain what you see?
>     
I should have always only one mutex claimed per thread. And always only for a very short period. What if several threads waiting for one mutex with priorities from 2 to 6 and the tread with priority 7 is holding the mutex. First of all will there be a reschedule after the mutex is realeased? 

Alois

-- 
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! 
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Pieter-Jan Busschaert]

[-- Attachment #1: Type: text/plain, Size: 1941 bytes --]

> I should have always only one mutex claimed per thread. And always only for a very short
> period. What if several threads waiting for one mutex with priorities from 2 to 6 and the
> tread with priority 7 is holding the mutex. First of all will there be a reschedule after the
> mutex is realeased?
>
> Alois


Hello,

To have more detailed info about this, you could turn on kernel
instrumentation and enable CLOCK, MUTEX and SCHEDULER events. For more
info, see here :
http://ecos.sourceware.org/docs-latest/user-guide/kernel-instrumentation.html

In your ecos.ecc file, these values have to be changed (you can
copy/paste this and use ecosconfig import) :

%------

cdl_option CYGPKG_KERNEL_INSTRUMENT {
    user_value 1
};

cdl_option CYGNUM_KERNEL_INSTRUMENT_BUFFER_SIZE {
    user_value 0x1000
};

cdl_option CYGDBG_KERNEL_INSTRUMENT_BUFFER_WRAP {
    user_value 0
};

cdl_component CYGDBG_KERNEL_INSTRUMENT_BUILD_HOST_DUMP {
    user_value 1
};

%------

This last option creates a binary ( [ecos.ecc
path]/install/bin/dump_instr ) which can be used to examine the
instrumentation buffer. I had to modify the source a little bit to
account for little endian / big endian differences, support for
wrapped around buffers and continuous time (ie: no time reset after
each clock IRQ). Attached is my source file (
ecos\packages\kernel\host\instr\dump_instr.c )

Ofcourse, you still have to get the instrumentation buffer from your
target. If you have filesystem and ftp support, this can be as easy as
:

FILE* fp;
fp = fopen("/D/etc/instrument.raw", "w");
if (fp != NULL)
{
  fwrite((void*) instrument_buffer, sizeof(instrument_buffer[0]),
instrument_buffer_size, fp);
  fclose(fp);
  int wrap = (instrument_buffer_pointer - &instrument_buffer[0]);
  diag_printf("Instrumentation buffer wrap_around = %d\r\n", wrap);
}


Then you can see very easily which mutexes are locked/unlocked and why
threads are waiting for them.


Pieter-Jan

[-- Attachment #2: dump_instr.c --]
[-- Type: text/plain, Size: 6097 bytes --]


#include <pkgconf/kernel.h>
#include <cyg/kernel/ktypes.h>         // base kernel types
#include <cyg/kernel/instrmnt.h>

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

#define CYG_INSTRUMENT_CLOCK_TICK_START (CYG_INSTRUMENT_EVENT_CLOCK_TICK_START|CYG_INSTRUMENT_CLASS_CLOCK)
#define CYG_INSTRUMENT_CLOCK_ISR (CYG_INSTRUMENT_EVENT_CLOCK_ISR|CYG_INSTRUMENT_CLASS_CLOCK)

// -------------------------------------------------------------------------
// Instrumentation record.

struct Instrument_Record
{
    CYG_WORD16  type;                   // record type
    CYG_WORD16  thread;                 // current thread id
    CYG_WORD    timestamp;              // 32 bit timestamp
    CYG_WORD    arg1;                   // first arg
    CYG_WORD    arg2;                   // second arg
};

// -------------------------------------------------------------------------

static long rtc_resolution[] = CYGNUM_KERNEL_COUNTERS_RTC_RESOLUTION;
static CYG_WORD64 ns_per_tick;
static CYG_WORD64 ps_per_hal_clock;
static CYG_WORD64 ticks = -1;

#ifdef CYGDBG_KERNEL_INSTRUMENT_MSGS
#define CYGDBG_KERNEL_INSTRUMENT_MSGS_DEFINE_TABLE
#include <cyg/kernel/instrument_desc.h>
#define NELEM(x) (sizeof(x)/sizeof*(x))


externC char * cyg_instrument_msg(CYG_WORD16 type)
{
  struct instrument_desc_s *record;
  struct instrument_desc_s *end_record;
  CYG_WORD cl, event;

  record = instrument_desc;
  end_record = &instrument_desc[NELEM(instrument_desc)-1];
  cl = type & 0xff00;
  event = type & 0x00ff;

  while ((record != end_record) && (record->num != cl)) {
    record++;
  }

  if (record->num == cl) {
    record++;
    while ((record != end_record) && (record->num != event) &&
           (record->num < 0xff)) {
      record++;
    }

    if (record->num == event) {
      return (record->msg);
    }
  }
  return("Unknown event");
}
#endif // CYGDBG_KERNEL_INSTRUMENT_MSGS

void usage(char *myname) 
{
  fprintf(stderr,"Usage: %s <filename> [start item (in case of wraparound)]\n",myname);
  fprintf(stderr,"where filename is that of the instrumentation data\n");
}

uint16_t swap2bytes(uint16_t input)
{
  uint16_t result = 0;
  result += (input & 0xff00) >> 8;
  result += (input & 0x00ff) << 8;
  return result;
}

uint32_t swap4bytes(uint32_t input)
{
  uint32_t result = 0;
  result += (input & 0xff000000) >> 24;
  result += (input & 0x00ff0000) >> 8;
  result += (input & 0x0000ff00) << 8;
  result += (input & 0x000000ff) << 24;
  return result;
}

/* Return the time in ns */
CYG_WORD64 cvt_time(CYG_WORD timestamp) 
{
  return (  (ticks * ns_per_tick * 1000) + 
            ((CYG_WORD64)timestamp * ps_per_hal_clock)  );
}

void find_first_tick(FILE* file, int firstitem)
{
  struct Instrument_Record record;
  bool wrapped = false;

  fseek(file, sizeof(record)*firstitem, SEEK_SET);
  int cnt = firstitem;

  while (!feof(file)) {
    if (fread(&record,sizeof(record),1,file) == 0) {    // EOF reached, wrap around
      if (!wrapped)
      {
        fseek(file, 0, SEEK_SET);
        cnt = 0;
        wrapped = true;
        continue;
      }
      break;
    }

    if (wrapped && cnt == firstitem)                    // end of buffer reached
    {
      break;
    }

    if (record.type == 0) {
      break;
    }

    record.type = swap2bytes(record.type);
    record.thread = swap2bytes(record.thread);
    record.timestamp = swap4bytes(record.timestamp);
    record.arg1 = swap4bytes(record.arg1);
    record.arg2 = swap4bytes(record.arg2);

    if (record.type == CYG_INSTRUMENT_CLOCK_TICK_START) {
      ticks = ((CYG_WORD64)record.arg2 << 32) + ((CYG_WORD64)record.arg1);
      printf("first tick found on item %d => %d\r\n", cnt, ticks);
      ticks--;
      return;
    }

    cnt++;
  }
  printf("first tick not found\r\n");
}

int main(int argc, char * argv[]) 
{
  FILE * file;
  char * filename;
  struct Instrument_Record record;
  int cnt=0;
  int firstitem = 0;
  bool wrapped = false;

  ns_per_tick = 1000000/rtc_resolution[1];
  ps_per_hal_clock = ns_per_tick * 1000 / CYGNUM_KERNEL_COUNTERS_RTC_PERIOD;

  if (argc != 2 && argc != 3) {
    usage(argv[0]);
    exit(1);
  }

  filename = argv[1];

  if (argc == 3)
  {
    firstitem = atoi(argv[2]);
  }

  file = fopen(filename, "r");
  if (!file) {
    fprintf(stderr,"Error opening file %s: ",filename);
    perror("");
    exit(1);
  }

  find_first_tick(file, firstitem);

  fseek(file, sizeof(record)*firstitem, SEEK_SET);
  cnt = firstitem;

  while (!feof(file)) {
    if (fread(&record,sizeof(record),1,file) == 0) {    // EOF reached, wrap around
      if (!wrapped)
      {
        fseek(file, 0, SEEK_SET);
        cnt = 0;
        wrapped = true;
        continue;
      }
      break;
    }

    if (wrapped && cnt == firstitem)                    // end of buffer reached
    {
      break;
    }

    if (record.type == 0) {
      break;
    }

    record.type = swap2bytes(record.type);
    record.thread = swap2bytes(record.thread);
    record.timestamp = swap4bytes(record.timestamp);
    record.arg1 = swap4bytes(record.arg1);
    record.arg2 = swap4bytes(record.arg2);


    if (record.type == CYG_INSTRUMENT_CLOCK_TICK_START)
    {
      if (ticks != ((CYG_WORD64)record.arg2 << 32) + ((CYG_WORD64)record.arg1))
      {
        printf("tick count error : ticks = %lld, should be %lld\r\n", 
          ticks, ((CYG_WORD64)record.arg2 << 32) + ((CYG_WORD64)record.arg1));
        ticks = ((CYG_WORD64)record.arg2 << 32) + ((CYG_WORD64)record.arg1);
      }
    }


#ifdef CYGDBG_KERNEL_INSTRUMENT_MSGS 
    printf("%4d Record type (0x%04x): %-20s thread %2d, ",
           cnt++,record.type,cyg_instrument_msg(record.type), 
           record.thread);
#else
    printf("%4d Record type 0x%04x, thread %2d, ",
           cnt++,record.type, record.thread);
#endif
    printf("time %10lld, arg1 0x%08x, arg2 0x%08x\n",
           cvt_time(record.timestamp), record.arg1,
           record.arg2);

    if (record.type == CYG_INSTRUMENT_CLOCK_ISR)
    {
      printf("tick increase\r\n");
      ticks++;
    }
  }

  fclose(file);
  printf("end-of-instrumentation\r\n");
  return (0);
}

[-- Attachment #3: Type: text/plain, Size: 148 bytes --]

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

> This last option creates a binary ( [ecos.ecc
> path]/install/bin/dump_instr ) which can be used to examine the
> instrumentation buffer. I had to modify the source a little bit to
> account for little endian / big endian differences, support for
> wrapped around buffers and continuous time (ie: no time reset after
> each clock IRQ). Attached is my source file (
> ecos\packages\kernel\host\instr\dump_instr.c )

Could you produce a clean patch which i can commit to CVS?

      Thanks
        Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Pieter-Jan Busschaert]

On 17/08/07, Andrew Lunn <andrew@lunn.ch> wrote:
>
> Could you produce a clean patch which i can commit to CVS?

I thought about that, but decided not to, because I hard-coded the
big-endian / little-endian swapping. I could add a CDL option to
enable / disable this swapping, or I could submit a clean patch
without swapping at all. What do you prefer ?


Pieter-Jan

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

On Fri, Aug 17, 2007 at 10:46:15AM +0200, Pieter-Jan Busschaert wrote:
> On 17/08/07, Andrew Lunn <andrew@lunn.ch> wrote:
> >
> > Could you produce a clean patch which i can commit to CVS?
> 
> I thought about that, but decided not to, because I hard-coded the
> big-endian / little-endian swapping. I could add a CDL option to
> enable / disable this swapping, or I could submit a clean patch
> without swapping at all. What do you prefer ?

This is a host tool. So you have command line parameters. -b and -l.
Or maybe it is easier to just have -s for swap. Then you don't need to
worry about the host endien's.

You also have access to the target header files. So you might be able
to see the definition of CYG_BYTEORDER. 

   Thanks
        Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Alois Zoitl]

Hi

> To have more detailed info about this, you could turn on kernel
> instrumentation and enable CLOCK, MUTEX and SCHEDULER events. For more
> info, see here :
> http://ecos.sourceware.org/docs-latest/user-guide/kernel-instrumentation.html

I tried a little bit with the kernel instrumentation, but could not get 
any results I could handle.
What i did was to further investigate which of my mutexes was the 
problematic one. And it is the one
I initiali thought will make a problem.

This mutex locks at some stage each of my threads. As soon as I remove 
this mutex (with special test apps this is not a problem) the scheduling 
is as expected. As soon as I add the mutex I get the problem again. Even 
the thread which is disturbed will access the mutex later in its 
execution its activation is delayed. It looks as this mutex gets an very 
high priority although I use priority inheritance.

For my current test I found it that priority inversion will not be a big 
problem. At least currently. So i'm fine without a priority inversion 
protocol.
Nonetheless Its ok for me to further investigate the problem if I can 
help to improve eCos.

cheers
Alil

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] Thread activation disturbed by lower priority threads]

[Alois Z.]

Hi, 

as I got no response to me questions (see below) I may have to add a few things for clarification.

First of all I'm running an an AT91M5580A processor (thy phytec board). I changed the ecos settings so that the timer tick is now 1ms. The reason for this is that I need such a small tick for my application. Does this anyhow influence the scheduling algorithm. Are there settings that need to be adjusted appart from denominator, nominator and timesclice value?

I did more measurements and found out that the timer DSR is really stable. even more stable than on some other systems (non ecos) I'm using. The problem is that the time between posting on the semaphore (the thread is waiting on) until the thread starts executing  is varying largly. It seems that it is prolonged by other execution elements. And this even when the thread under question is the thread with the highest priority.

would be great if this clearifies my problem a little bit more.

Thanks,
Alois

Hi,

In a larger project I have a problem when doing a timed activation of threads.

In the timer alarm handler I check if any thread is ready for activation and post on a semaphore the thread is waiting for.

This works fine when I have only one or two such timed threads the activation jitter of the thread (i.e. the jitter when the threads starts its execution) is ok. But when I add more threads this jitter grows tremendously. Each thread has an own priority. And event the thread with with the highest priority (in my case 2) has this large jitter (more than half of the cycle time).

Is there a way to improve the behavior of the activation. Do I the timed activation the wrong way. Can I improve the timing behavior of the scheduler through the real-time clock settings.

By the way I use the multilevel scheduler.

Thanks,
Alois



-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

On Tue, Aug 07, 2007 at 04:02:34PM +0200, Alois Z. wrote:
> Hi, 
> 
> as I got no response to me questions (see below) I may have to add a
> few things for clarification.
>
> First of all I'm running an an AT91M5580A processor (thy phytec
> board). I changed the ecos settings so that the timer tick is now
> 1ms. The reason for this is that I need such a small tick for my
> application. Does this anyhow influence the scheduling
> algorithm. Are there settings that need to be adjusted appart from
> denominator, nominator and timesclice value?
> 
> I did more measurements and found out that the timer DSR is really
> stable. even more stable than on some other systems (non ecos) I'm
> using. The problem is that the time between posting on the semaphore
> (the thread is waiting on) until the thread starts executing is
> varying largly. It seems that it is prolonged by other execution
> elements. And this even when the thread under question is the thread
> with the highest priority.  would be great if this clearifies my
> problem a little bit more.

If it is the highest priority runnable thread, as soon as the DSR
finished it should get to run. The only exception i can think of is if
some other thread has the scheduler locked. This would prevent a
context switch until the scheduler was unlocked.

How to you do your timing between the DSR timer and thread running? 

Does this high priority thread need to acquire a mutex etc? It could
be that something else has the mutex. So it has to wait for it to be
released. Priority inversion then happens. The lower priority thread
which holds the mutex gets boosted in priority to the priority of the
waiting thread. This should allow the low priority thread to finish
what it is doing and release the mutex. However there is one
wrinkle. eCos only undoes priority inversion when the thread releases
all its mutex, not just the mutex of interest. 

    Andrew


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

RE: [ECOS] Thread activation disturbed by lower priority threads]

[Paul D. DeRocco]

> From: Andrew Lunn
> 
> If it is the highest priority runnable thread, as soon as the 
> DSR finished it should get to run. The only exception i can 
> think of is if some other thread has the scheduler locked. 
> This would prevent a context switch until the scheduler was unlocked.


Doesn't locking the scheduler also prevent the DSR from running?

-- 

Ciao,               Paul D. DeRocco
Paul                mailto:pderocco@ix.netcom.com 


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

On Tue, Aug 07, 2007 at 10:35:14AM -0700, Paul D. DeRocco wrote:
> > From: Andrew Lunn
> > 
> > If it is the highest priority runnable thread, as soon as the 
> > DSR finished it should get to run. The only exception i can 
> > think of is if some other thread has the scheduler locked. 
> > This would prevent a context switch until the scheduler was unlocked.
> 
> 
> Doesn't locking the scheduler also prevent the DSR from running?

Err, um, correct. Sorry, i was wrong.

     Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

On Wed, Aug 08, 2007 at 09:58:10AM +0200, Alois Z. wrote:
> 
> -------- Original-Nachricht --------
> Datum: Tue, 7 Aug 2007 16:40:47 +0200
> Von: Andrew Lunn <andrew@lunn.ch>
> An: "Alois Z." <alois@gmx.at>
> CC: ecos-discuss@ecos.sourceware.org
> Betreff: Re: [ECOS] Thread activation disturbed by lower priority threads]
> 
> > On Tue, Aug 07, 2007 at 04:02:34PM +0200, Alois Z. wrote:
> > > Hi, 
> > > 
> > > as I got no response to me questions (see below) I may have to add a
> > > few things for clarification.
> > >
> > > First of all I'm running an an AT91M5580A processor (thy phytec
> > > board). I changed the ecos settings so that the timer tick is now
> > > 1ms. The reason for this is that I need such a small tick for my
> > > application. Does this anyhow influence the scheduling
> > > algorithm. Are there settings that need to be adjusted appart from
> > > denominator, nominator and timesclice value?
> > > 
> > > I did more measurements and found out that the timer DSR is really
> > > stable. even more stable than on some other systems (non ecos) I'm
> > > using. The problem is that the time between posting on the semaphore
> > > (the thread is waiting on) until the thread starts executing is
> > > varying largly. It seems that it is prolonged by other execution
> > > elements. And this even when the thread under question is the thread
> > > with the highest priority.  would be great if this clearifies my
> > > problem a little bit more.
> > 
> > If it is the highest priority runnable thread, as soon as the DSR
> > finished it should get to run. The only exception i can think of is if
> > some other thread has the scheduler locked. This would prevent a
> > context switch until the scheduler was unlocked.
> > 
> > How to you do your timing between the DSR timer and thread running? 
> 

> I just set bits in both and can than see the timing on an
> oscilloscope. This works really good and I did the same measurements
> on different boards.

Do you set the bits just after the semaphore operations, or later,
when it does the real work? I'm just thinking about the mutex issue.
 
> > Does this high priority thread need to acquire a mutex etc? It could
> > be that something else has the mutex. So it has to wait for it to be
> > released. Priority inversion then happens. The lower priority thread
> > which holds the mutex gets boosted in priority to the priority of the
> > waiting thread. This should allow the low priority thread to finish
> > what it is doing and release the mutex. However there is one
> > wrinkle. eCos only undoes priority inversion when the thread releases
> > all its mutex, not just the mutex of interest. 
> > 
> >     Andrew
> > 

> There is may be a mutex the high priority thread has to wait for. It
> is just one and typically the lock time is rather
> short. Unfortunatly every thread will use this mutex so maybe thats
> the reason for my problem. As I think now of it it may be a bad
> design, but because of other constraints it will not be possible to
> remove this mutex. By the way it works on other real-time operating
> systems (e.g. ThreadX). So I should think on the riority inversion
> protocol for the mutexes, i'm right?

Just for the purpose of testing a theory, take out the mutex. If the
timing gets better, you know the mutex is the problem. 

You might also want to look at kernel instrumentation. 
http://ecos.sourceware.org/docs-latest/user-guide/kernel-instrumentation.html

        Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Alois Zoitl]

Hi,

thanks you definitely pointed me into the right direction. The problem 
is located with the mutexes. I removed all mutexes in questions and set 
my timing measurement points directly after the semaphore that is in 
charge of activating my threads.After making more measurements and 
playing a little bit around I found it that when I set the priority 
inversion protocol to none (using cyg_mutex_set_protocol). The timing is 
as expected. So I thought it could be that before I was using priority 
ceiling which would be an explanation for the delay as every time a 
mutex is gathered the thread will get priorty 0. So I changed to 
priority inheritance. This from my point of view should do the job as I 
like to have it done.
But when using priority inheritance I get the same bad timing as in the 
beginning.

And as longer I think I don't know why the tread activation of the 
highest priority thread is prolonged by threads holding a mutex with 
priority inheritance where each of the treads that my also get this 
mutex has a lower priority. So I'm completly confuesed. Any Ideas what 
could be the problem or what I could do?

For my current tests no priority inversion protocol is just fine, but 
for further more complected tests I think i will need something like 
priority inheritance so it would be nice to have it.

Thanks,
Alois

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

On Thu, Aug 16, 2007 at 10:04:54PM +0200, Alois Zoitl wrote:
> Hi,
>
> thanks you definitely pointed me into the right direction. The problem is 
> located with the mutexes. I removed all mutexes in questions and set my 
> timing measurement points directly after the semaphore that is in charge of 
> activating my threads.After making more measurements and playing a little 
> bit around I found it that when I set the priority inversion protocol to 
> none (using cyg_mutex_set_protocol). The timing is as expected. So I 
> thought it could be that before I was using priority ceiling which would be 
> an explanation for the delay as every time a mutex is gathered the thread 
> will get priorty 0. So I changed to priority inheritance. This from my 
> point of view should do the job as I like to have it done.
> But when using priority inheritance I get the same bad timing as in the 
> beginning.
>
> And as longer I think I don't know why the tread activation of the highest 
> priority thread is prolonged by threads holding a mutex with priority 
> inheritance where each of the treads that my also get this mutex has a 
> lower priority. So I'm completly confuesed. Any Ideas what could be the 
> problem or what I could do?
>
> For my current tests no priority inversion protocol is just fine, but for 
> further more complected tests I think i will need something like priority 
> inheritance so it would be nice to have it.

How many mutex's does your lower priority thread hold? From
packages/kernel/current/src/sched/sched.cxx

void Cyg_SchedThread::clear_inherited_priority()
{
    CYG_REPORT_FUNCTION();

#ifdef CYGSEM_KERNEL_SYNCH_MUTEX_PRIORITY_INVERSION_PROTOCOL_SIMPLE

    // A simple implementation of priority inheritance/ceiling
    // protocols.  The simplification in this algorithm is that we do
    // not reduce our priority until we have freed all mutexes
    // claimed. Hence we can continue to run at an artificially high
    // priority even when we should not.  However, since nested
    // mutexes are rare, the thread we have inherited from is likely
    // to be locking the same mutexes we are, and mutex claim periods
    // should be very short, the performance difference between this
    // and a more complex algorithm should be negligible. The most
    // important advantage of this algorithm is that it is fast and
    // deterministic.

Does this explain what you see?
    
    Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Thread activation disturbed by lower priority threads]

[Andrew Lunn]

On Fri, Aug 17, 2007 at 09:28:49AM +0200, Alois Z. wrote:
> Hi,
> > 
> > How many mutex's does your lower priority thread hold? From
> > packages/kernel/current/src/sched/sched.cxx
> > 
> > void Cyg_SchedThread::clear_inherited_priority()
> > {
> >     CYG_REPORT_FUNCTION();
> > 
> > #ifdef CYGSEM_KERNEL_SYNCH_MUTEX_PRIORITY_INVERSION_PROTOCOL_SIMPLE
> > 
> >     // A simple implementation of priority inheritance/ceiling
> >     // protocols.  The simplification in this algorithm is that we do
> >     // not reduce our priority until we have freed all mutexes
> >     // claimed. Hence we can continue to run at an artificially high
> >     // priority even when we should not.  However, since nested
> >     // mutexes are rare, the thread we have inherited from is likely
> >     // to be locking the same mutexes we are, and mutex claim periods
> >     // should be very short, the performance difference between this
> >     // and a more complex algorithm should be negligible. The most
> >     // important advantage of this algorithm is that it is fast and
> >     // deterministic.
> > 
> > Does this explain what you see?
> >     

> I should have always only one mutex claimed per thread. And always
> only for a very short period. What if several threads waiting for
> one mutex with priorities from 2 to 6 

The queue of threads waiting for the mutex can either be in FIFO
order, or sorted into decreasing priority order. FIFO is fast where as
sorting the queue takes more time. I think the default is FIFO. Take a
look at CYGIMP_KERNEL_SCHED_SORTED_QUEUES.

> and the tread with priority 7 is holding the mutex. First of all
> will there be a reschedule after the mutex is realeased?

Yes, as soon as the higher priority thread is runnable, and the
scheduler is unlocked, there will be a context switch to the higher
priority thread.

          Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss